-- Logs begin at Sat 2022-10-15 20:01:13 UTC, end at Sat 2022-10-15 22:33:46 UTC. -- Oct 15 20:36:52.212052 ncdv-master-1 systemd[1]: Starting Devstack devstack@n-api.service... Oct 15 20:36:52.220801 ncdv-master-1 devstack@n-api.service[122247]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 15 20:36:52 2022] *** Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: nodename: ncdv-master-1 Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: machine: x86_64 Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: clock source: unix Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: pcre jit disabled Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: detected number of CPU cores: 4 Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: current working directory: / Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: detected binary path: /usr/bin/uwsgi-core Oct 15 20:36:52.222791 ncdv-master-1 devstack@n-api.service[122247]: your processes number limit is 47680 Oct 15 20:36:52.223556 ncdv-master-1 devstack@n-api.service[122247]: your memory page size is 4096 bytes Oct 15 20:36:52.223556 ncdv-master-1 devstack@n-api.service[122247]: detected max file descriptor number: 2048 Oct 15 20:36:52.223556 ncdv-master-1 devstack@n-api.service[122247]: lock engine: pthread robust mutexes Oct 15 20:36:52.223556 ncdv-master-1 devstack@n-api.service[122247]: thunder lock: enabled Oct 15 20:36:52.223556 ncdv-master-1 devstack@n-api.service[122247]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 15 20:36:52.223556 ncdv-master-1 devstack@n-api.service[122247]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 15 20:36:52.267359 ncdv-master-1 devstack@n-api.service[122247]: Python main interpreter initialized at 0x55a63525c9c0 Oct 15 20:36:52.267359 ncdv-master-1 devstack@n-api.service[122247]: python threads support enabled Oct 15 20:36:52.267359 ncdv-master-1 devstack@n-api.service[122247]: your server socket listen backlog is limited to 100 connections Oct 15 20:36:52.267359 ncdv-master-1 devstack@n-api.service[122247]: your mercy for graceful operations on workers is 90 seconds Oct 15 20:36:52.267747 ncdv-master-1 devstack@n-api.service[122247]: mapped 403077 bytes (393 KB) for 2 cores Oct 15 20:36:52.267827 ncdv-master-1 devstack@n-api.service[122247]: *** Operational MODE: preforking *** Oct 15 20:36:52.268020 ncdv-master-1 devstack@n-api.service[122247]: *** uWSGI is running in multiple interpreter mode *** Oct 15 20:36:52.268020 ncdv-master-1 devstack@n-api.service[122247]: spawned uWSGI master process (pid: 122247) Oct 15 20:36:52.268230 ncdv-master-1 systemd[1]: Started Devstack devstack@n-api.service. Oct 15 20:36:52.269428 ncdv-master-1 devstack@n-api.service[122247]: spawned uWSGI worker 1 (pid: 122248, cores: 1) Oct 15 20:36:52.269708 ncdv-master-1 devstack@n-api.service[122247]: spawned uWSGI worker 2 (pid: 122249, cores: 1) Oct 15 20:36:52.269820 ncdv-master-1 devstack@n-api.service[122247]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 15 20:36:53.573615 ncdv-master-1 devstack@n-api.service[122248]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 15 20:36:53.574859 ncdv-master-1 devstack@n-api.service[122249]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 15 20:36:54.251446 ncdv-master-1 devstack@n-api.service[122249]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:36:54.251446 ncdv-master-1 devstack@n-api.service[122249]: from cryptography.utils import int_from_bytes Oct 15 20:36:54.251751 ncdv-master-1 devstack@n-api.service[122249]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:36:54.251751 ncdv-master-1 devstack@n-api.service[122249]: from cryptography.utils import int_from_bytes Oct 15 20:36:54.253762 ncdv-master-1 devstack@n-api.service[122248]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:36:54.253762 ncdv-master-1 devstack@n-api.service[122248]: from cryptography.utils import int_from_bytes Oct 15 20:36:54.254076 ncdv-master-1 devstack@n-api.service[122248]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:36:54.254076 ncdv-master-1 devstack@n-api.service[122248]: from cryptography.utils import int_from_bytes Oct 15 20:36:56.854519 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 20:36:56.855085 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 20:36:56.855292 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 20:36:56.855659 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 20:36:56.855913 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 20:36:56.856299 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.856622 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.856946 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.857300 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.857629 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.857939 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.858259 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.858576 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.858894 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 20:36:56.859250 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.859250 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.859404 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 20:36:56.859485 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.859693 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 20:36:56.859787 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.860056 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 20:36:56.860143 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.860312 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 20:36:56.860406 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.860657 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.860753 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.860970 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.861092 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.861301 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.861387 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.861623 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.861741 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.861928 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.862042 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.862268 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.862421 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.862597 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.862883 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863032 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863192 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863312 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863628 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863744 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863850 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.863947 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.864335 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.864477 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.864666 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.864753 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.864975 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.865085 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.865311 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.865400 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.865653 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.865752 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.865935 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.866029 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.866257 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.866398 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.866565 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.866648 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.866922 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.867052 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.867243 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.867350 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.867652 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.867738 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.867960 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868046 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868266 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868362 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868585 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868671 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868892 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.868981 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.869784 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.869893 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.870098 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.870248 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.870424 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.870573 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.870740 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.870898 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.871136 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.871235 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.871455 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.871559 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.871769 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.871856 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.872083 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.872179 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.872399 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.872511 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.872703 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.872823 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873007 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873162 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873281 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873486 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873570 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873810 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.873905 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874119 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874205 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874447 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874581 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874737 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874824 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.874984 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.875071 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.875298 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.875393 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.875615 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.875721 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.875926 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876012 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876232 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876330 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876536 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876621 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876839 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.876923 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.877168 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.877290 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.877424 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.877522 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.877735 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.877831 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.878055 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.878152 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.878371 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.878467 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.878684 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.878771 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879002 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879097 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879254 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879339 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879586 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879671 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879910 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.879998 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.880219 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.880304 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.880522 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.880607 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.880831 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.880928 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.881169 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.881262 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.881479 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.881592 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.881787 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.881889 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.882095 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.882204 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.882399 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.882504 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.882704 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.882795 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883010 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883103 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883329 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883423 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883651 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883743 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.883954 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.884039 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.884202 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.884356 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.884504 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.884599 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.884842 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.885484 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.885803 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.885947 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.886114 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.886200 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.886422 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.886507 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.886728 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.886823 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887028 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887128 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887336 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887433 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887640 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887726 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.887946 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888032 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888252 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888337 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888558 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888644 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888862 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.888946 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.889136 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.889225 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.889450 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.889537 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.889758 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.889851 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890067 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890161 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890376 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890461 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890687 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890782 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.890991 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891106 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891290 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891375 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891595 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891681 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891899 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.891994 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.892207 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.892292 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.892512 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.892595 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.892817 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.892979 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.893147 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.893235 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.893451 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.893551 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.893756 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.893852 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894060 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894157 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894364 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894448 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894670 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894862 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.894986 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895073 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895285 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895371 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895589 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895683 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895893 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.895989 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.896201 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.896287 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.896502 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.896596 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.896805 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.896891 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.897130 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.897230 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.897435 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.897543 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.897737 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.897829 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898039 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898182 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898339 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898432 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898637 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898723 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.898947 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899038 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899262 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899350 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899570 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899654 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899873 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.899958 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.900178 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.900264 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.900520 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.900644 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.900852 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.900944 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.901207 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.901306 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.901512 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.901597 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.901820 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.901920 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.902129 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.902217 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.902435 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.902525 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.902740 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.902837 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.903042 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.903140 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.903362 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.903448 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.903684 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.903778 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.903991 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.904075 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.904295 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.904380 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.904601 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.904687 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.904915 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.904999 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:36:56.905350 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.905450 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.905607 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.905693 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.905925 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906058 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906240 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906327 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906557 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906643 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906865 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.906957 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.907181 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.907294 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.907501 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.907629 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.907816 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.907916 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.908123 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.908219 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.908441 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.908534 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.908755 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.908848 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.909089 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.909185 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.909420 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.909514 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.909736 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.909835 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910044 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910154 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910353 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910455 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910660 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910753 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.910981 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.911083 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.911289 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.911381 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.911596 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.911690 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.911911 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912020 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912220 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912313 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912532 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912638 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912837 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.912943 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.913169 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.913263 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.913480 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.913586 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.913786 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.913889 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.914093 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.914183 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.914423 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.914528 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.914736 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.914849 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915046 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915144 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915359 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915444 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915667 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915754 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.915983 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.916113 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.916317 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.916411 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.916625 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.916729 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.916929 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917015 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917255 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917344 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917571 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917706 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917858 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.917945 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.918163 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.918258 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.918463 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.918549 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.918779 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.918872 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.919100 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.919194 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.919400 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.919492 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.919728 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.919822 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920032 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920127 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920338 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920424 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920644 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920729 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.920950 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921077 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921352 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921462 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921590 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921678 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921894 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.921993 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.922199 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.922285 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.922509 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.922594 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.922815 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.922909 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.923114 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.923200 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.923433 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.923518 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.923736 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.923823 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924039 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924133 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924349 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924435 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924657 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924742 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.924964 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925069 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925298 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925396 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925603 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925689 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925913 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.925998 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.926222 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.926321 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.926524 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.926623 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.926846 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.926940 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.927155 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.927253 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.927459 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.927544 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.927779 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.927874 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.928085 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.928180 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.928393 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.928478 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.928693 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.928784 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929000 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929117 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929327 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929426 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929629 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929721 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.929937 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930038 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930240 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930325 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930542 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930629 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930845 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.930932 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.931149 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.931243 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.931455 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.931579 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.932281 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.932379 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.932602 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.932718 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.932911 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933028 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933237 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933363 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933540 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933673 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933839 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.933987 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.934179 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.934294 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.934482 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.934602 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.934821 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.934921 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.935152 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.935238 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.935466 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.935561 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.935786 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.935881 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.936092 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.936177 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.936399 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.936484 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.936704 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.936790 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937010 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937152 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937342 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937429 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937644 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937729 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.937951 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938036 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938255 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938339 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938569 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938656 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938876 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.938961 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.939195 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.939281 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.939503 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.939590 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.939819 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.939905 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.940120 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.940215 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.940432 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.940531 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.940754 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.940839 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.941084 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.941187 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.941402 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.941505 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.941713 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.941810 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942019 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942113 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942330 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942416 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942637 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942723 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.942944 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.943039 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.943249 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.943334 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.943554 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.943648 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.943871 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944033 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944176 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944261 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944480 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944563 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944783 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.944869 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.945125 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.945213 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.945434 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.945531 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.945749 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.945836 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.946051 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.946137 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.946366 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.946462 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.946691 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.946779 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.947014 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.947103 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.947390 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.947485 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.947766 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.947853 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.948090 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.948175 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.948399 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.948496 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.948707 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.948794 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949017 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949141 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949343 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949443 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949646 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949732 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.949952 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950038 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950267 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950363 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950578 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950663 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950882 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.950976 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.951185 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.951281 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.951494 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.951581 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.951799 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.951888 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.952099 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.952186 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.952400 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.952488 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.952708 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.952801 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.953025 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.953166 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.953358 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.953444 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.953675 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.953805 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954002 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954138 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954292 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954378 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954593 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954678 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954912 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.954999 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.955217 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.955373 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.955519 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.955606 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.955821 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.955909 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.956120 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.956206 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.956425 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.956518 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.956731 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.956817 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.957073 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.957163 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.957385 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.957527 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.957700 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.957787 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958004 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958147 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958316 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958412 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958621 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958714 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.958925 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959020 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959243 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959344 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959553 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959647 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959857 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.959941 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.960161 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.960247 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.960463 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.960547 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.960769 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.960853 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.961097 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.961188 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.961420 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.961517 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.961726 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.961856 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.962033 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.962121 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.962339 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.962424 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.962637 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.962722 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.963479 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.963615 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.963786 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.963895 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.964185 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.964348 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.964505 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.964678 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.964816 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.964991 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.965145 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.965334 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.965451 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.965680 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.965782 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.965996 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.966097 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.966325 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.966425 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.966651 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.966745 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.966964 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967049 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967265 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967359 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967568 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967660 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967868 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.967960 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.968170 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.968266 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.968477 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.968735 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.968821 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.969087 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.969188 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.969391 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.969486 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.969698 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.969785 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970001 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970097 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970314 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970416 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970617 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970704 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.970920 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971013 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971231 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971362 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971531 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971618 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971838 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.971924 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.972139 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.972235 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.972458 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.972544 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.972766 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.972850 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973084 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973173 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973390 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973488 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973690 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973777 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.973997 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.974082 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.974303 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.974395 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.974608 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.974702 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.974922 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975015 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975230 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975315 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975549 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975644 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975853 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.975938 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.976160 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.976252 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.976465 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.976560 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.976768 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.976853 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977088 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977189 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977398 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977495 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977702 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977787 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.977998 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.978088 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.978306 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.978403 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.978703 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.978798 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979062 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979149 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979370 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979460 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979672 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979787 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.979977 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.980101 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.980277 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.980409 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.980592 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.980733 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.980899 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981070 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981210 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981367 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981515 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981670 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981823 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.981980 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.982128 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.982288 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.982426 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.982589 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.982734 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.982900 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983035 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983209 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983349 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983518 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983651 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983828 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.983953 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.984131 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.984252 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.984438 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.984557 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.984740 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.984863 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985075 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985175 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985389 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985485 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985692 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985778 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.985995 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.986080 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.986306 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.986405 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.986616 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.986703 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.986919 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987004 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987228 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987320 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987533 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987626 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987846 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.987930 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.988150 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.988244 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.988449 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.988534 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.988751 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.988843 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.989062 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.989448 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.989639 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.989726 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.989941 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990027 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990247 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990335 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990546 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990670 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990846 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.990933 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.991146 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.991234 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.991444 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.991530 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.991743 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.991828 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992042 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992172 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992351 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992437 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992650 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992743 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.992952 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993059 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993276 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993363 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993596 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993693 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993901 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.993986 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.994719 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.994818 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995026 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995242 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995336 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995418 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995630 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995724 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.995935 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996036 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996239 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996326 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996544 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996630 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996847 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.996941 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.997182 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.997273 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.997498 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.997585 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.997803 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.997899 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.998103 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.998209 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.998405 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.998531 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.998722 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.998810 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999031 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999118 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999339 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999425 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999657 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999746 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:56.999975 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.000067 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.000293 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.000388 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.000596 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.000672 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.000903 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.001051 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.001234 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.001375 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.001540 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.001682 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.001846 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002001 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002151 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002314 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002461 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002630 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002768 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.002935 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003074 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003240 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003380 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003562 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003685 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003875 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.003987 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.004187 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.004293 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.004511 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.004597 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.004836 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.004969 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.005291 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.005394 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.005497 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.005583 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.005804 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.005899 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.006108 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.006195 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.006408 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.006501 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.006714 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.006811 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007015 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007102 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007323 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007454 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007635 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007761 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.007944 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008068 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008253 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008370 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008563 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008676 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008868 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.008983 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.009201 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.009321 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.009509 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.009631 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.009885 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.010007 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.010285 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.010494 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.010591 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.010807 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.010895 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.011116 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.011202 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.011426 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.011513 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.011727 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.011815 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012026 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012126 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012333 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012480 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012652 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012793 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.012953 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.013287 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.013431 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.013549 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.013657 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.013743 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.013880 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014019 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014160 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014335 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014477 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014649 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014887 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.014980 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.015194 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.015359 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.015493 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.015672 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.015902 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.015998 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.016210 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.016297 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.016516 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.016609 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.016833 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.016928 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.017174 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.017266 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.017502 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.017589 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.017807 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.017907 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.018119 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.018207 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.018425 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.018511 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.018746 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.018831 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019051 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019151 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019357 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019442 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019676 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019763 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.019980 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.020075 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.020301 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.020388 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.020622 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.020764 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.020934 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.021096 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.021276 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.021408 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.021685 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.021771 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022005 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022093 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022312 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022438 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022630 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022718 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.022945 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023031 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023243 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023330 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023545 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023642 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023849 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.023935 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.024149 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.024242 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.024449 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.024545 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.024767 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.024854 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.025086 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.025176 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.025913 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.026022 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.026258 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.026350 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.026588 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.026731 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027002 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027097 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027328 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027414 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027641 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027739 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.027949 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028039 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028271 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028358 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028577 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028662 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028884 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.028972 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.029219 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.029307 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.029522 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.029606 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.029827 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.029913 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.030136 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.030221 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.030437 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.030524 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.030740 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.030827 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031040 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031128 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031349 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031448 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031654 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031740 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.031958 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032045 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032278 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032377 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032581 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032667 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032894 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.032986 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.033230 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.033328 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.033552 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.033639 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.033868 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.033952 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.034171 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.034266 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.034485 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.034570 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.034790 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.034874 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035090 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035176 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035391 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035477 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035692 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035775 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.035989 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036074 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036289 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036430 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036588 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036675 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036891 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.036979 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.037215 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.037304 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.037515 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.037601 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.037820 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.037946 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.038137 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.038232 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.038435 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.038531 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.038751 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.038838 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039056 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039141 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039355 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039461 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039657 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039745 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.039958 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.040043 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.040306 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.040404 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.040608 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.040694 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.040920 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.041151 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.041317 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.041421 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.041632 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.041753 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.041937 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.042072 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.042244 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.042389 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.042551 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.042697 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.042879 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043000 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043183 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043293 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043504 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043609 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043813 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.043907 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.044118 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.044213 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.044423 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.044516 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.044726 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.044820 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.045090 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.045189 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.045407 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.045623 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.045732 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.045949 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046048 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046262 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046347 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046573 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046662 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046882 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.046974 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.047192 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.047280 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.047501 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.047589 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.047816 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.047905 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.048124 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.048217 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.048430 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.048518 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.048737 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.048837 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.049143 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.049252 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.049468 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.049565 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.049786 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.049883 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.050117 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.050308 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.050436 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.050635 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.050776 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.050955 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.051090 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.051275 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.051420 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.051592 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.051732 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.051907 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052056 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052222 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052438 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052585 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052719 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052851 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.052988 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.053167 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.053316 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.053452 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.053587 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.053774 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.053911 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054075 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054221 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054382 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054518 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054689 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054826 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.054997 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.055135 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.055308 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.055446 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.055618 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.055710 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.055938 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.056025 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.056269 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.056362 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.056586 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.056738 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.056923 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.057011 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.057262 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.057353 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.057579 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.057683 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.057897 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.058096 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.058221 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.058407 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.058495 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.058736 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.058832 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.059057 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.059182 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.059378 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.059480 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.059693 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.059780 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060004 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060090 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060319 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060418 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060620 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060708 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.060921 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061015 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061260 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061369 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061567 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061654 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061866 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.061960 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.062183 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.062271 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.062490 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.062711 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.062805 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.063027 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.063137 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.063362 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.063456 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.063690 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.063932 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064038 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064238 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064326 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064554 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064648 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064880 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.064973 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.065229 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.065319 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.065544 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.065628 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.065847 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.065942 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.066153 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.066239 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.066457 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.066550 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.066762 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.066870 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067072 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067167 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067373 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067459 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067674 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067769 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.067980 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.068067 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.068288 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.068508 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.068595 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.068853 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.068940 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.069193 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.069330 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.069495 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.069596 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.069817 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.069905 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.070121 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.070208 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.070422 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.070521 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.070722 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.070809 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071028 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071116 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071330 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071424 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071628 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071759 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.071931 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.072021 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.072232 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.072932 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.073092 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.073181 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.073403 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.073489 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.073710 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.073800 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074012 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074106 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074317 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074403 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074621 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074707 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.074947 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075042 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075253 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075340 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075559 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075647 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075859 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.075952 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.076160 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.076246 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.076462 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.076548 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.076779 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.076920 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.077090 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.077195 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.077397 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.077485 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.077706 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.077791 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078010 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078111 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078317 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078450 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078620 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078871 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.078965 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.079208 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.079296 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.079546 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.079632 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.079854 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.079938 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.080178 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.080266 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.080481 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.080567 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.080786 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.080885 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.081114 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.081282 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.081438 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.081549 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.081750 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.081915 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082060 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082179 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082369 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082489 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082679 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082835 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.082986 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.083151 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.083295 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.083604 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.083690 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.083911 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.083998 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.084217 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.084326 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.084524 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.084664 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.084831 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085000 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085147 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085335 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085459 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085644 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085766 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.085949 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.086076 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.086274 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.086413 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.086585 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.086729 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.086895 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.087025 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.087204 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.087342 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.087516 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.087657 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.087827 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.088545 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.088693 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.088845 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.089101 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.089193 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.089421 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.089510 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.089735 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.089832 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090046 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090133 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090357 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090443 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090667 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090754 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.090977 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.091288 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.091374 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.091609 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.091705 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.091933 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092027 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092256 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092343 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092567 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092662 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092882 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.092969 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.093228 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.093317 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.093539 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.093626 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.093848 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.093933 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.094156 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.094248 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.094466 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.094559 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.094783 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.094877 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.095113 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.095200 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.095427 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.095514 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.095729 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.095855 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.096033 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.096336 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.096422 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.096643 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.096730 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.096962 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097122 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097298 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097387 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097602 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097696 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097904 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.097990 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.098210 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.098299 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.098514 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.098600 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.098817 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.098903 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.099118 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.099215 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.099420 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.099507 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.099726 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.099825 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100027 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100145 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100327 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100501 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100627 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100810 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.100927 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.101146 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.101257 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.101461 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.101546 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.101773 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.101861 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.102094 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.102180 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.102406 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.102502 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.102712 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.102798 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103013 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103097 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103312 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103407 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103656 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103749 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.103965 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.104168 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.104272 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.104479 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.104614 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.104803 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.104894 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.105136 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.105290 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.105461 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.105571 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.105766 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.105856 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106073 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106178 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106378 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106470 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106682 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106774 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.106988 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107087 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107341 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107449 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107678 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107762 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107860 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.107955 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.108169 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.108282 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.108475 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.108561 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.108805 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.108947 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.109137 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.109229 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.109464 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.109552 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.109788 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.109876 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.110094 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.110181 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.110416 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.110500 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.110727 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.110815 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111044 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111130 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111359 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111446 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111670 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111761 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.111980 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.112131 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.112296 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.112383 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.112605 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.112693 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.112914 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113001 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113249 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113350 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113557 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113644 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113873 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.113961 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.114444 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.114545 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.114753 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.114841 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.115068 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.115155 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.115382 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.115478 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.115707 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.115807 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116026 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116112 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116334 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116420 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116651 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116736 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.116962 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.117100 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.117318 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.117415 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.117631 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.117762 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.117942 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118033 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118252 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118340 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118564 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118651 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118874 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.118974 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.119202 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.119297 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.119446 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 20:36:57.119609 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.120054 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.120375 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.120692 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.121022 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.121383 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.121702 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.122029 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.122357 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.122704 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.123018 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.123366 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.123694 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.124009 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.124322 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.124631 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:36:57.124871 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 20:36:57.184993 ncdv-master-1 devstack@n-api.service[122249]: INFO dbcounter [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Registered counter for database nova_api Oct 15 20:36:57.190713 ncdv-master-1 devstack@n-api.service[122248]: INFO dbcounter [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Registered counter for database nova_api Oct 15 20:36:57.206737 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_db.sqlalchemy.engines [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.211000 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_db.sqlalchemy.engines [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.213960 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writer thread running {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.218225 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writer thread running {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.269744 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.context [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=122249) load_cells /opt/stack/nova/nova/context.py:464}} Oct 15 20:36:57.273584 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:36:57.274066 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:36:57.274169 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.context [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=122248) load_cells /opt/stack/nova/nova/context.py:464}} Oct 15 20:36:57.276184 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:36:57.276911 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:36:57.277373 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:36:57.278458 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_concurrency.lockutils [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:36:57.278927 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_concurrency.lockutils [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=122248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:36:57.279343 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:36:57.280470 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_concurrency.lockutils [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:36:57.281223 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_concurrency.lockutils [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:36:57.281323 ncdv-master-1 devstack@n-api.service[122249]: INFO dbcounter [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Registered counter for database nova_cell0 Oct 15 20:36:57.281663 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_concurrency.lockutils [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:36:57.283633 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_concurrency.lockutils [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122248) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:36:57.285288 ncdv-master-1 devstack@n-api.service[122249]: INFO dbcounter [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Registered counter for database nova_cell1 Oct 15 20:36:57.285639 ncdv-master-1 devstack@n-api.service[122248]: INFO dbcounter [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Registered counter for database nova_cell0 Oct 15 20:36:57.289578 ncdv-master-1 devstack@n-api.service[122248]: INFO dbcounter [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Registered counter for database nova_cell1 Oct 15 20:36:57.293168 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_db.sqlalchemy.engines [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.298170 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_db.sqlalchemy.engines [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.298534 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_db.sqlalchemy.engines [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.300145 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_db.sqlalchemy.engines [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.303721 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writer thread running {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.305220 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writer thread running {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.308561 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writer thread running {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.439755 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writer thread running {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.450379 ncdv-master-1 devstack@n-api.service[122249]: INFO dbcounter [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Registered counter for database nova_cell0 Oct 15 20:36:57.455042 ncdv-master-1 devstack@n-api.service[122248]: INFO dbcounter [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Registered counter for database nova_cell0 Oct 15 20:36:57.459242 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_db.sqlalchemy.engines [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.463926 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writer thread running {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.464480 ncdv-master-1 devstack@n-api.service[122248]: DEBUG oslo_db.sqlalchemy.engines [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:36:57.468829 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writer thread running {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:36:57.920382 ncdv-master-1 devstack@n-api.service[122249]: WARNING keystonemiddleware.auth_token [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:36:57.922651 ncdv-master-1 devstack@n-api.service[122248]: WARNING keystonemiddleware.auth_token [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:36:57.927729 ncdv-master-1 devstack@n-api.service[122249]: WARNING keystonemiddleware.auth_token [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:36:57.929978 ncdv-master-1 devstack@n-api.service[122248]: WARNING keystonemiddleware.auth_token [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:36:58.152940 ncdv-master-1 devstack@n-api.service[122249]: WARNING keystonemiddleware.auth_token [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:36:58.156677 ncdv-master-1 devstack@n-api.service[122248]: WARNING keystonemiddleware.auth_token [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:36:58.158990 ncdv-master-1 devstack@n-api.service[122249]: WARNING keystonemiddleware.auth_token [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:36:58.162705 ncdv-master-1 devstack@n-api.service[122248]: WARNING keystonemiddleware.auth_token [None req-935ebbbf-5f44-47a9-a991-6db885b743ed None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:36:58.164252 ncdv-master-1 devstack@n-api.service[122249]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a63525c9c0 pid: 122249 (default app) Oct 15 20:36:58.166939 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:36:58.168110 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] 10.222.0.16 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.002306 Oct 15 20:36:58.168509 ncdv-master-1 devstack@n-api.service[122248]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a63525c9c0 pid: 122248 (default app) Oct 15 20:36:58.168657 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 788 bytes} [Sat Oct 15 20:36:58 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:26.027240 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:26.028160 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001424 Oct 15 20:38:26.028790 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:26.041407 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_api:SELECT=3 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:26.042677 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_cell1:SELECT=3 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:26.043388 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_cell0:SELECT=3 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:26.044147 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:26.937914 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-79b62f78-2eab-434a-af44-81b9d6e7fd1a admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:26.972909 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-79b62f78-2eab-434a-af44-81b9d6e7fd1a admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.936707 Oct 15 20:38:26.973711 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 1/3] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 20:38:26 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 939 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:29.917400 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:29.918416 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-3f09ef50-3565-4b44-bfe2-5cec2e914f94 None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Oct 15 20:38:29.918968 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 3/4] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:30.349329 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-5f3bcd93-74ef-4b99-a356-6e878ba9e6af admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "cirros256", "ram": 256, "disk": 1, "swap": 0, "rxtx_factor": 1.0, "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:30.397202 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-5f3bcd93-74ef-4b99-a356-6e878ba9e6af admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.471398 Oct 15 20:38:30.397961 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 2/5] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:29 2022] POST /compute/v2.1/flavors => generated 407 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:30.412330 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_api:SELECT=3 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:30.413954 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_cell0:SELECT=3 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:30.414864 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_cell1:SELECT=3 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:30.415173 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:30.626460 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-75fc70b8-e4c9-4626-96c8-60546dc7f8c8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:30.698787 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-75fc70b8-e4c9-4626-96c8-60546dc7f8c8 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.291544 Oct 15 20:38:30.699369 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 4/6] 10.222.0.16 () {64 vars in 1336 bytes} [Sat Oct 15 20:38:30 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:33.757490 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-5f3bcd93-74ef-4b99-a356-6e878ba9e6af admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:33.759194 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-5f3bcd93-74ef-4b99-a356-6e878ba9e6af admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002187 Oct 15 20:38:33.760350 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 3/7] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:33 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:34.695347 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e01a3c72-efae-4861-8e74-4335b0781c03 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 5, "ram": 512, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "ds512M", "id": "d1", "os-flavor-access:is_public": true, "vcpus": 1}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:34.808854 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-e01a3c72-efae-4861-8e74-4335b0781c03 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.971154 Oct 15 20:38:34.808854 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 5/8] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:33 2022] POST /compute/v2.1/flavors => generated 404 bytes in 973 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:34.809135 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-d13067a3-66db-4d57-9c89-3f5a32a4fd6c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:34.809135 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-d13067a3-66db-4d57-9c89-3f5a32a4fd6c admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.059438 Oct 15 20:38:34.809135 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 4/9] 10.222.0.16 () {64 vars in 1336 bytes} [Sat Oct 15 20:38:34 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:38.184353 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e01a3c72-efae-4861-8e74-4335b0781c03 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:38.185524 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-e01a3c72-efae-4861-8e74-4335b0781c03 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001463 Oct 15 20:38:38.193946 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:38 2022] GET /compute/v2.1 => generated 388 bytes in 10 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:38.672783 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-9a32cb1b-2a28-4a3c-8903-e2bc6670ce39 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 10, "ram": 1024, "name": "ds1G", "os-flavor-access:is_public": true, "vcpus": 1, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d2", "rxtx_factor": 1.0}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:38.692219 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-9a32cb1b-2a28-4a3c-8903-e2bc6670ce39 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.498013 Oct 15 20:38:38.693604 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 5/11] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:38 2022] POST /compute/v2.1/flavors => generated 404 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:38.705330 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-0f9ad7c3-f636-461b-a2b0-270967882ec8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:38.730132 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-0f9ad7c3-f636-461b-a2b0-270967882ec8 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028333 Oct 15 20:38:38.730737 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 7/12] 10.222.0.16 () {64 vars in 1336 bytes} [Sat Oct 15 20:38:38 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:41.917885 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-9a32cb1b-2a28-4a3c-8903-e2bc6670ce39 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:41.918998 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-9a32cb1b-2a28-4a3c-8903-e2bc6670ce39 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001840 Oct 15 20:38:41.919596 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 6/13] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:41 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:42.344320 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-76e8b716-0145-4dc3-a912-e468a021c569 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 10, "name": "ds2G", "vcpus": 2, "rxtx_factor": 1.0, "ram": 2048, "id": "d3"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:42.368476 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-76e8b716-0145-4dc3-a912-e468a021c569 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.442481 Oct 15 20:38:42.369012 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 8/14] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:41 2022] POST /compute/v2.1/flavors => generated 404 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:42.381499 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e6d397b4-33a9-4cf0-905d-cd2b695f442d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:42.410753 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e6d397b4-33a9-4cf0-905d-cd2b695f442d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032995 Oct 15 20:38:42.411982 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 7/15] 10.222.0.16 () {64 vars in 1336 bytes} [Sat Oct 15 20:38:42 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:45.350909 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-76e8b716-0145-4dc3-a912-e468a021c569 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:45.351828 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-76e8b716-0145-4dc3-a912-e468a021c569 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Oct 15 20:38:45.352228 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 9/16] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:38:45.828378 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e7964f01-8020-4020-9c6d-c08ab51848fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d4", "disk": 20, "vcpus": 4, "ram": 4096, "name": "ds4G"}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:45.847180 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e7964f01-8020-4020-9c6d-c08ab51848fd admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.488872 Oct 15 20:38:45.848029 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 8/17] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:45 2022] POST /compute/v2.1/flavors => generated 404 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:45.860934 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-343eed9e-6369-40b6-bf25-0d54db78c061 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:45.886846 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-343eed9e-6369-40b6-bf25-0d54db78c061 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029657 Oct 15 20:38:45.887468 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 10/18] 10.222.0.16 () {64 vars in 1336 bytes} [Sat Oct 15 20:38:45 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:48.978910 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e7964f01-8020-4020-9c6d-c08ab51848fd admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:48.980275 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e7964f01-8020-4020-9c6d-c08ab51848fd admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001967 Oct 15 20:38:48.980782 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 9/19] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:49.424149 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-92ac9a86-2463-477d-bcb9-1d0b8d3dfdbe admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:49.451827 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-92ac9a86-2463-477d-bcb9-1d0b8d3dfdbe admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.462901 Oct 15 20:38:49.452370 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 11/20] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 20:38:48 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:38:52.476966 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e7964f01-8020-4020-9c6d-c08ab51848fd admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:52.478476 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e7964f01-8020-4020-9c6d-c08ab51848fd admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002056 Oct 15 20:38:52.479136 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 10/21] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:52 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:38:52.923108 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-0fc728e9-04d7-40ad-bfce-beac3865b6b9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "disk": 1, "vcpus": 1, "name": "m1.tiny", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "1"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:52.944008 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-0fc728e9-04d7-40ad-bfce-beac3865b6b9 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.456817 Oct 15 20:38:52.944654 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 12/22] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:52 2022] POST /compute/v2.1/flavors => generated 402 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:52.956917 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-069410bf-5dff-468c-9ddc-8471b6c69e7f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:53.002941 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-069410bf-5dff-468c-9ddc-8471b6c69e7f admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.049256 Oct 15 20:38:53.003849 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 11/23] 10.222.0.16 () {64 vars in 1333 bytes} [Sat Oct 15 20:38:52 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:56.054135 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-0fc728e9-04d7-40ad-bfce-beac3865b6b9 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:56.055057 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-0fc728e9-04d7-40ad-bfce-beac3865b6b9 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001441 Oct 15 20:38:56.055455 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:38:56.459977 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-d7d188e4-1d57-4a72-811d-26054830a86e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.small", "disk": 20, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "2", "ram": 2048, "rxtx_factor": 1.0, "vcpus": 1}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:56.465539 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=6,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:38:56.481059 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-d7d188e4-1d57-4a72-811d-26054830a86e admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.419577 Oct 15 20:38:56.485068 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 12/25] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:56 2022] POST /compute/v2.1/flavors => generated 405 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:38:56.494141 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-47ad84dd-6325-49c9-a5f8-51b3281eb939 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:38:56.517621 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-47ad84dd-6325-49c9-a5f8-51b3281eb939 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027148 Oct 15 20:38:56.518313 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 14/26] 10.222.0.16 () {64 vars in 1333 bytes} [Sat Oct 15 20:38:56 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:38:59.529969 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-d7d188e4-1d57-4a72-811d-26054830a86e admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:38:59.531161 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-d7d188e4-1d57-4a72-811d-26054830a86e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Oct 15 20:38:59.531687 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 13/27] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:38:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:00.015558 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-56e8c484-4370-4cfd-83df-ef29c86d8f29 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "id": "3", "rxtx_factor": 1.0, "swap": 0, "disk": 40, "ram": 4096, "vcpus": 2}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:00.036076 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-56e8c484-4370-4cfd-83df-ef29c86d8f29 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.497236 Oct 15 20:39:00.036629 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 15/28] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:38:59 2022] POST /compute/v2.1/flavors => generated 406 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:39:00.048667 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e36388d4-ba6b-4568-bb51-f40ce339fddf admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:00.075192 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e36388d4-ba6b-4568-bb51-f40ce339fddf admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030302 Oct 15 20:39:00.076091 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 14/29] 10.222.0.16 () {64 vars in 1333 bytes} [Sat Oct 15 20:39:00 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:39:03.062216 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-56e8c484-4370-4cfd-83df-ef29c86d8f29 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:03.063236 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-56e8c484-4370-4cfd-83df-ef29c86d8f29 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Oct 15 20:39:03.063650 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 16/30] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:39:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:03.533560 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-a69ed1e4-bf7a-4edb-8eb7-189f7fc4e9d8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.large", "disk": 80, "os-flavor-access:is_public": true, "vcpus": 4, "rxtx_factor": 1.0, "id": "4", "swap": 0, "ram": 8192, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:03.558736 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-a69ed1e4-bf7a-4edb-8eb7-189f7fc4e9d8 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.487949 Oct 15 20:39:03.559399 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 15/31] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:39:03 2022] POST /compute/v2.1/flavors => generated 405 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:39:03.571864 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-88fcdaad-6063-49ff-9169-f00cde252198 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:03.576671 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:03.597330 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-88fcdaad-6063-49ff-9169-f00cde252198 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028951 Oct 15 20:39:03.598081 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 17/32] 10.222.0.16 () {64 vars in 1333 bytes} [Sat Oct 15 20:39:03 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:39:06.532529 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-a69ed1e4-bf7a-4edb-8eb7-189f7fc4e9d8 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:06.533701 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-a69ed1e4-bf7a-4edb-8eb7-189f7fc4e9d8 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001542 Oct 15 20:39:06.534112 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 16/33] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:39:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:06.989065 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-003046e8-3bef-4003-8f1f-8c8512d5e40a admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "ram": 16384, "disk": 160, "name": "m1.xlarge", "id": "5", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "vcpus": 8, "rxtx_factor": 1.0}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:07.021075 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-003046e8-3bef-4003-8f1f-8c8512d5e40a admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.480575 Oct 15 20:39:07.021633 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 18/34] 10.222.0.16 () {64 vars in 1283 bytes} [Sat Oct 15 20:39:06 2022] POST /compute/v2.1/flavors => generated 408 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 20:39:07.033834 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e0693b30-354d-45f1-b5d0-17cdeba44d02 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:07.062380 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e0693b30-354d-45f1-b5d0-17cdeba44d02 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032152 Oct 15 20:39:07.063234 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 17/35] 10.222.0.16 () {64 vars in 1333 bytes} [Sat Oct 15 20:39:07 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:39:26.538759 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-003046e8-3bef-4003-8f1f-8c8512d5e40a admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:26.539764 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-003046e8-3bef-4003-8f1f-8c8512d5e40a admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Oct 15 20:39:26.540174 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 19/36] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:39:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:26.550594 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=17,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:26.969616 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-948a27b2-1ca7-4345-ac37-8d7b1031ef62 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:26.991116 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-948a27b2-1ca7-4345-ac37-8d7b1031ef62 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.444975 Oct 15 20:39:26.992019 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 20:39:26 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:39:28.911077 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-003046e8-3bef-4003-8f1f-8c8512d5e40a admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:28.912081 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-003046e8-3bef-4003-8f1f-8c8512d5e40a admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001423 Oct 15 20:39:28.912484 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 20/38] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:29.517076 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-f3121e78-6197-4fae-8809-880891408f6e admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:29.523209 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.wsgi [None req-f3121e78-6197-4fae-8809-880891408f6e admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-f3121e78-6197-4fae-8809-880891408f6e admin admin] Request method failure captured: Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Accept: application/json Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Accept-Encoding: gzip, deflate Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Connection: keep-alive Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Content-Length: 0 Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Host: 10.222.0.16 Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Openstack-Api-Version: compute 2.92 Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: Openstack-System-Scope: None Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: User-Agent: python-novaclient Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Auth-Token: gAAAAABjSxqA2D_KRJJ9o6rgXGZcYKXWeZY2QRmdsZBaIooZIVtsPghQeSQ4ePtNj6_MQwWcUCvVieAJ99VFZtK5v-JHUINQgrEwiXJ4VNAiWriDDLCDgKg76ywc-CgfFwIQjYBs64SpAXmY9B4O9wev0QsdseU8IPOVfDoUb7TfVNXef1ziYpU Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Domain-Id: None Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Domain-Name: None Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Identity-Status: Confirmed Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Is-Admin-Project: True Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Openstack-Nova-Api-Version: 2.92 Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Domain-Id: default Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Domain-Name: Default Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Name: admin Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Role: reader,member,admin Oct 15 20:39:29.532732 ncdv-master-1 devstack@n-api.service[122248]: X-Roles: reader,member,admin Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant: admin Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant-Name: admin Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-User: admin Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-User-Domain-Id: default Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-User-Domain-Name: Default Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-User-Id: f9ecef8e2dba4f429d37c67479eb320e Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: X-User-Name: admin Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: method: > Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: exception: Flavor 42 could not be found. Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: action_args: {'id': '42'} Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:39:29.534622 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:29.535689 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.537215 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-f3121e78-6197-4fae-8809-880891408f6e admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=122248) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:39:29.538523 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-f3121e78-6197-4fae-8809-880891408f6e admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.400629 Oct 15 20:39:29.538523 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 19/39] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:39:29 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 402 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:29.544353 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-d65bc85a-070d-45ff-9c22-f8ab6fedec75 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:29.548397 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:39:29.568172 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-d65bc85a-070d-45ff-9c22-f8ab6fedec75 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027360 Oct 15 20:39:29.568718 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 21/40] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Oct 15 20:39:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:39:29.579286 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-12894d34-c2eb-4e16-a5c3-92a4d0a097c2 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:29.603958 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-12894d34-c2eb-4e16-a5c3-92a4d0a097c2 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.029178 Oct 15 20:39:29.604503 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 20/41] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Oct 15 20:39:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:39:29.614681 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-400a9076-0f37-4bb8-a08a-484365a55a6f admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:29.620281 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.wsgi [None req-400a9076-0f37-4bb8-a08a-484365a55a6f admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-400a9076-0f37-4bb8-a08a-484365a55a6f admin admin] Request method failure captured: Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Accept: application/json Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Accept-Encoding: gzip, deflate Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Connection: keep-alive Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Content-Length: 0 Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Host: 10.222.0.16 Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Openstack-Api-Version: compute 2.92 Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: Openstack-System-Scope: None Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: User-Agent: python-novaclient Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Auth-Token: gAAAAABjSxqA2D_KRJJ9o6rgXGZcYKXWeZY2QRmdsZBaIooZIVtsPghQeSQ4ePtNj6_MQwWcUCvVieAJ99VFZtK5v-JHUINQgrEwiXJ4VNAiWriDDLCDgKg76ywc-CgfFwIQjYBs64SpAXmY9B4O9wev0QsdseU8IPOVfDoUb7TfVNXef1ziYpU Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Domain-Id: None Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Domain-Name: None Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Identity-Status: Confirmed Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Is-Admin-Project: True Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Openstack-Nova-Api-Version: 2.92 Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Domain-Id: default Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Domain-Name: Default Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Name: admin Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Role: reader,member,admin Oct 15 20:39:29.623530 ncdv-master-1 devstack@n-api.service[122249]: X-Roles: reader,member,admin Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant: admin Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant-Name: admin Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-User: admin Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-User-Domain-Id: default Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-User-Domain-Name: Default Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-User-Id: f9ecef8e2dba4f429d37c67479eb320e Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: X-User-Name: admin Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: method: > Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: exception: Flavor 42 could not be found. Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: action_args: {'id': '42'} Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:39:29.625022 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:29.626000 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:29.627070 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-400a9076-0f37-4bb8-a08a-484365a55a6f admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=122249) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:39:29.628586 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-400a9076-0f37-4bb8-a08a-484365a55a6f admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014608 Oct 15 20:39:29.628586 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 22/42] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:39:29 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:31.435844 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-12894d34-c2eb-4e16-a5c3-92a4d0a097c2 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:31.436843 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-12894d34-c2eb-4e16-a5c3-92a4d0a097c2 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Oct 15 20:39:31.437273 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 21/43] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:31.971259 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e2d5d31f-213a-4702-90ea-5b29d1ce92db admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:31.976202 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.wsgi [None req-e2d5d31f-213a-4702-90ea-5b29d1ce92db admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e2d5d31f-213a-4702-90ea-5b29d1ce92db admin admin] Request method failure captured: Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Accept: application/json Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Accept-Encoding: gzip, deflate Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Connection: keep-alive Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Content-Length: 0 Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Host: 10.222.0.16 Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Openstack-Api-Version: compute 2.92 Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: Openstack-System-Scope: None Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: User-Agent: python-novaclient Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Auth-Token: gAAAAABjSxqD71ZmrjUwgKhBcvIrw9DiXGKyJB9iAoCME8k5PShEHteTDRJqhnZoUxvY8h2e7A_m3IHfswxG7Cnw5Fpe2Y7SbxYfsWt1Dz5pnAGqRU1NIR8BcKonOa4_CcB3A7P8TKAx7Ilm9afaKcIRKjjaOwIJ7zdM3Ol1i2CYyCSFLWThdrw Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Domain-Id: None Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Domain-Name: None Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Identity-Status: Confirmed Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Is-Admin-Project: True Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Openstack-Nova-Api-Version: 2.92 Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Domain-Id: default Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Domain-Name: Default Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Name: admin Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Role: reader,member,admin Oct 15 20:39:31.977731 ncdv-master-1 devstack@n-api.service[122249]: X-Roles: reader,member,admin Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant: admin Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant-Name: admin Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-User: admin Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-User-Domain-Id: default Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-User-Domain-Name: Default Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-User-Id: f9ecef8e2dba4f429d37c67479eb320e Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: X-User-Name: admin Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: method: > Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: exception: Flavor 84 could not be found. Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: action_args: {'id': '84'} Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:39:31.979171 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:31.980062 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:31.981484 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e2d5d31f-213a-4702-90ea-5b29d1ce92db admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=122249) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:39:31.982700 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-e2d5d31f-213a-4702-90ea-5b29d1ce92db admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.395547 Oct 15 20:39:31.982700 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 23/44] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:39:31 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 397 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:31.989676 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-43621798-1569-4d87-9311-94505f3c8a72 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:32.005228 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-43621798-1569-4d87-9311-94505f3c8a72 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020025 Oct 15 20:39:32.005685 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 22/45] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Oct 15 20:39:31 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:39:32.015062 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e0447409-ea90-4330-a421-ba582fcadeaa admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:32.031426 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-e0447409-ea90-4330-a421-ba582fcadeaa admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019878 Oct 15 20:39:32.031942 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Oct 15 20:39:32 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:39:32.041277 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-a03717c0-9420-4ec2-b16b-2ccdfb71e521 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:32.046228 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.wsgi [None req-a03717c0-9420-4ec2-b16b-2ccdfb71e521 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-a03717c0-9420-4ec2-b16b-2ccdfb71e521 admin admin] Request method failure captured: Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Accept: application/json Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Accept-Encoding: gzip, deflate Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Connection: keep-alive Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Content-Length: 0 Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Host: 10.222.0.16 Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Openstack-Api-Version: compute 2.92 Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: Openstack-System-Scope: None Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: User-Agent: python-novaclient Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Auth-Token: gAAAAABjSxqD71ZmrjUwgKhBcvIrw9DiXGKyJB9iAoCME8k5PShEHteTDRJqhnZoUxvY8h2e7A_m3IHfswxG7Cnw5Fpe2Y7SbxYfsWt1Dz5pnAGqRU1NIR8BcKonOa4_CcB3A7P8TKAx7Ilm9afaKcIRKjjaOwIJ7zdM3Ol1i2CYyCSFLWThdrw Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Domain-Id: None Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Domain-Name: None Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Identity-Status: Confirmed Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Is-Admin-Project: True Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Openstack-Nova-Api-Version: 2.92 Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Domain-Id: default Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Domain-Name: Default Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Name: admin Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Role: reader,member,admin Oct 15 20:39:32.047723 ncdv-master-1 devstack@n-api.service[122248]: X-Roles: reader,member,admin Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant: admin Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant-Name: admin Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-User: admin Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-User-Domain-Id: default Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-User-Domain-Name: Default Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-User-Id: f9ecef8e2dba4f429d37c67479eb320e Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: X-User-Name: admin Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: method: > Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: exception: Flavor 84 could not be found. Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: action_args: {'id': '84'} Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:39:32.048949 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:32.050263 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:32.051598 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-a03717c0-9420-4ec2-b16b-2ccdfb71e521 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=122248) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:39:32.052818 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-a03717c0-9420-4ec2-b16b-2ccdfb71e521 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011495 Oct 15 20:39:32.052818 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 23/47] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:39:32 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:33.814539 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-e0447409-ea90-4330-a421-ba582fcadeaa admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:33.816143 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-e0447409-ea90-4330-a421-ba582fcadeaa admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002033 Oct 15 20:39:33.816538 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 25/48] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:33 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:34.346936 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-466bd8e4-77f9-44d2-b740-ed0fd36d0c35 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:34.352771 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.wsgi [None req-466bd8e4-77f9-44d2-b740-ed0fd36d0c35 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-466bd8e4-77f9-44d2-b740-ed0fd36d0c35 admin admin] Request method failure captured: Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Accept: application/json Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Accept-Encoding: gzip, deflate Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Connection: keep-alive Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Content-Length: 0 Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Host: 10.222.0.16 Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Openstack-Api-Version: compute 2.92 Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: Openstack-System-Scope: None Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: User-Agent: python-novaclient Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Auth-Token: gAAAAABjSxqFsMLIkNxNLAW5tzs_HzS8mg2ou1Aak_UrGZh5JLcTMk0Jlnu4nZHd-iUQbsT275lQZ9u7_t1WzfQLX6P3q6m4Wai3BD9Onv62d2Iu0oFlMdEAcxV7jjbnofKvUwxcSnMUfpb2pTyVUecbvyVf0BXjLW_EJZFdeVjSI9voIFKGej0 Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Domain-Id: None Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Domain-Name: None Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Identity-Status: Confirmed Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Is-Admin-Project: True Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Openstack-Nova-Api-Version: 2.92 Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Domain-Id: default Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Domain-Name: Default Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Project-Name: admin Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Role: reader,member,admin Oct 15 20:39:34.354292 ncdv-master-1 devstack@n-api.service[122248]: X-Roles: reader,member,admin Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant: admin Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-Tenant-Name: admin Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-User: admin Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-User-Domain-Id: default Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-User-Domain-Name: Default Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-User-Id: f9ecef8e2dba4f429d37c67479eb320e Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: X-User-Name: admin Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: method: > Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: exception: Flavor 451 could not be found. Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: action_args: {'id': '451'} Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:39:34.355735 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:34.356570 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.357883 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-466bd8e4-77f9-44d2-b740-ed0fd36d0c35 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=122248) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:39:34.359244 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-466bd8e4-77f9-44d2-b740-ed0fd36d0c35 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.392597 Oct 15 20:39:34.359244 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 24/49] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 20:39:33 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 394 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:34.365693 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-03109140-a540-4657-9084-0fb68b1fb760 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:34.380384 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-03109140-a540-4657-9084-0fb68b1fb760 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018661 Oct 15 20:39:34.380905 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 26/50] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Oct 15 20:39:34 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:39:34.390225 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:34.405910 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019189 Oct 15 20:39:34.406434 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 25/51] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Oct 15 20:39:34 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:39:34.415569 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-738b085a-aac0-4fbc-b706-3114e6d8456c admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:34.420412 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.wsgi [None req-738b085a-aac0-4fbc-b706-3114e6d8456c admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-738b085a-aac0-4fbc-b706-3114e6d8456c admin admin] Request method failure captured: Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Accept: application/json Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Accept-Encoding: gzip, deflate Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Connection: keep-alive Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Content-Length: 0 Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Host: 10.222.0.16 Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Openstack-Api-Version: compute 2.92 Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: Openstack-System-Scope: None Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: User-Agent: python-novaclient Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Auth-Token: gAAAAABjSxqFsMLIkNxNLAW5tzs_HzS8mg2ou1Aak_UrGZh5JLcTMk0Jlnu4nZHd-iUQbsT275lQZ9u7_t1WzfQLX6P3q6m4Wai3BD9Onv62d2Iu0oFlMdEAcxV7jjbnofKvUwxcSnMUfpb2pTyVUecbvyVf0BXjLW_EJZFdeVjSI9voIFKGej0 Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Domain-Id: None Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Domain-Name: None Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Identity-Status: Confirmed Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Is-Admin-Project: True Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Openstack-Nova-Api-Version: 2.92 Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Domain-Id: default Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Domain-Name: Default Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Project-Name: admin Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Role: reader,member,admin Oct 15 20:39:34.421917 ncdv-master-1 devstack@n-api.service[122249]: X-Roles: reader,member,admin Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38e66271254b4380bfa7d7e1ccea54cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant: admin Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant-Id: 38e66271254b4380bfa7d7e1ccea54cc Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-Tenant-Name: admin Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-User: admin Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-User-Domain-Id: default Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-User-Domain-Name: Default Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-User-Id: f9ecef8e2dba4f429d37c67479eb320e Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: X-User-Name: admin Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: method: > Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: exception: Flavor 451 could not be found. Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: action_args: {'id': '451'} Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:39:34.423404 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:34.424923 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: ERROR nova.api.openstack.wsgi Oct 15 20:39:34.426314 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-738b085a-aac0-4fbc-b706-3114e6d8456c admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=122249) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:39:34.427516 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-738b085a-aac0-4fbc-b706-3114e6d8456c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011141 Oct 15 20:39:34.427516 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 27/52] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 20:39:34 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:39:36.214181 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:36.215180 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001423 Oct 15 20:39:36.215585 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:36.747055 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-6fed872a-54ee-4a45-ac70-5642da43e765 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:36.766699 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-6fed872a-54ee-4a45-ac70-5642da43e765 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.404080 Oct 15 20:39:36.767246 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 28/54] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Oct 15 20:39:36 2022] POST /compute/v2.1/flavors => generated 404 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:39:38.547322 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:38.548319 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001418 Oct 15 20:39:38.548769 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:39.081638 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-bb3ba565-7ea8-493e-bc6e-31d245538c47 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:39.099635 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-bb3ba565-7ea8-493e-bc6e-31d245538c47 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.403675 Oct 15 20:39:39.100184 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 29/56] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Oct 15 20:39:38 2022] POST /compute/v2.1/flavors => generated 405 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:39:40.908199 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:40.909799 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002109 Oct 15 20:39:40.910188 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 28/57] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:41.440311 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-83089be7-36bf-4555-968b-d0afc1c68f42 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:41.458483 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-83089be7-36bf-4555-968b-d0afc1c68f42 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.402305 Oct 15 20:39:41.459013 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 30/58] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Oct 15 20:39:41 2022] POST /compute/v2.1/flavors => generated 407 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:39:43.261639 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:43.262670 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Oct 15 20:39:43.263086 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 29/59] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:43.796210 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-d2c0f49c-2fb2-4245-a724-1f1ad5cbaf33 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:39:43.815522 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-d2c0f49c-2fb2-4245-a724-1f1ad5cbaf33 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.404716 Oct 15 20:39:43.816066 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 31/60] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Oct 15 20:39:43 2022] POST /compute/v2.1/flavors => generated 409 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:39:45.606101 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:45.607098 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001425 Oct 15 20:39:45.607484 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 30/61] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:39:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:39:46.143813 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-78f008c4-7138-4ffb-9617-414a56d3c90c admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:39:46.164916 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-78f008c4-7138-4ffb-9617-414a56d3c90c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.409002 Oct 15 20:39:46.165485 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 32/62] 10.222.0.16 () {60 vars in 1202 bytes} [Sat Oct 15 20:39:45 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:40:51.321966 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:40:51.322892 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-00d46881-0563-4aa3-b8c3-505d9b308eff admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Oct 15 20:40:51.323299 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 31/63] 10.222.0.16 () {54 vars in 931 bytes} [Sat Oct 15 20:40:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:40:51.331551 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:51.726645 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-d37e9f3f-75dd-4b5d-858c-a3c57d0860cc admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:40:51.748531 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-d37e9f3f-75dd-4b5d-858c-a3c57d0860cc admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.417786 Oct 15 20:40:51.749081 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 33/64] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 20:40:51 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:40:51.769717 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_api:SELECT=14 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:40:51.773451 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-241c720e-d766-4b93-8a16-71714390a1ca admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:40:51.781511 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-241c720e-d766-4b93-8a16-71714390a1ca admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013406 Oct 15 20:40:51.782032 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 32/65] 10.222.0.16 () {60 vars in 1283 bytes} [Sat Oct 15 20:40:51 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:40:51.793786 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-11ec2b17-eb83-4f92-a26a-8f3a60bb47cf admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:40:51.801928 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-11ec2b17-eb83-4f92-a26a-8f3a60bb47cf admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011757 Oct 15 20:40:51.802451 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 34/66] 10.222.0.16 () {60 vars in 1286 bytes} [Sat Oct 15 20:40:51 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:40:51.814082 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e7058623-54be-4130-91db-b979a9c94f64 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:40:51.821634 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e7058623-54be-4130-91db-b979a9c94f64 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011381 Oct 15 20:40:51.822343 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 33/67] 10.222.0.16 () {60 vars in 1286 bytes} [Sat Oct 15 20:40:51 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:40:51.833963 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-d3f19637-40cc-43f2-a890-cc5f2035c67c admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:40:51.840991 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-d3f19637-40cc-43f2-a890-cc5f2035c67c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010707 Oct 15 20:40:51.842121 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 35/68] 10.222.0.16 () {60 vars in 1283 bytes} [Sat Oct 15 20:40:51 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:46:18.297398 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e7058623-54be-4130-91db-b979a9c94f64 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:46:18.298454 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e7058623-54be-4130-91db-b979a9c94f64 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001553 Oct 15 20:46:18.298847 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:46:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:46:18.453331 ncdv-master-1 devstack@n-api.service[122249]: DEBUG dbcounter [-] [122249] Writing DB stats nova_api:SELECT=6 {{(pid=122249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:46:18.595416 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:46:18.600064 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:46:18.600555 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:46:18.600977 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:46:18.601713 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:46:18.602157 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:46:18.602595 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:46:18.639970 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-4f949dd2-b004-439f-9d07-350fecabe567 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.92 time: 0.188008 Oct 15 20:46:18.640524 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 36/70] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 20:46:18 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:46:20.262970 ncdv-master-1 devstack@n-api.service[122248]: DEBUG nova.api.openstack.wsgi [None req-e7058623-54be-4130-91db-b979a9c94f64 admin admin] Calling method '>' {{(pid=122248) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:46:20.264024 ncdv-master-1 devstack@n-api.service[122248]: INFO nova.api.openstack.requestlog [None req-e7058623-54be-4130-91db-b979a9c94f64 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Oct 15 20:46:20.264416 ncdv-master-1 devstack@n-api.service[122248]: [pid: 122248|app: 0|req: 35/71] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Oct 15 20:46:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:46:20.469128 ncdv-master-1 devstack@n-api.service[122249]: DEBUG nova.api.openstack.wsgi [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Calling method '>' {{(pid=122249) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:46:20.472543 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:46:20.473051 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:46:20.473480 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:46:20.474803 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:46:20.475243 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:46:20.475653 ncdv-master-1 devstack@n-api.service[122249]: DEBUG oslo_concurrency.lockutils [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122249) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:46:20.499312 ncdv-master-1 devstack@n-api.service[122249]: INFO nova.api.openstack.requestlog [None req-c22c1b75-4dde-4bfe-930d-859105df8454 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.92 time: 0.081781 Oct 15 20:46:20.499884 ncdv-master-1 devstack@n-api.service[122249]: [pid: 122249|app: 0|req: 37/72] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 20:46:20 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:47:05.778015 ncdv-master-1 systemd[1]: Stopping Devstack devstack@n-api.service... Oct 15 20:47:05.779005 ncdv-master-1 devstack@n-api.service[122247]: Sat Oct 15 20:47:05 2022 - graceful shutdown triggered... Oct 15 20:47:05.779542 ncdv-master-1 devstack@n-api.service[122248]: Gracefully killing worker 1 (pid: 122248)... Oct 15 20:47:05.779999 ncdv-master-1 devstack@n-api.service[122249]: Gracefully killing worker 2 (pid: 122249)... Oct 15 20:47:05.923626 ncdv-master-1 devstack@n-api.service[122248]: DEBUG dbcounter [-] [122248] Writing DB stats nova_api:SELECT=4 {{(pid=122248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:06.780128 ncdv-master-1 devstack@n-api.service[122247]: worker 1 buried after 1 seconds Oct 15 20:47:06.780642 ncdv-master-1 devstack@n-api.service[122247]: worker 2 buried after 1 seconds Oct 15 20:47:06.780642 ncdv-master-1 devstack@n-api.service[122247]: goodbye to uWSGI. Oct 15 20:47:06.781845 ncdv-master-1 systemd[1]: devstack@n-api.service: Succeeded. Oct 15 20:47:06.782406 ncdv-master-1 systemd[1]: Stopped Devstack devstack@n-api.service. Oct 15 20:47:06.785171 ncdv-master-1 systemd[1]: Starting Devstack devstack@n-api.service... Oct 15 20:47:06.826995 ncdv-master-1 devstack@n-api.service[138673]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 15 20:47:06 2022] *** Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: nodename: ncdv-master-1 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: machine: x86_64 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: clock source: unix Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: pcre jit disabled Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: detected number of CPU cores: 4 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: current working directory: / Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: detected binary path: /usr/bin/uwsgi-core Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: your processes number limit is 47680 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: your memory page size is 4096 bytes Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: detected max file descriptor number: 2048 Oct 15 20:47:06.829307 ncdv-master-1 devstack@n-api.service[138673]: lock engine: pthread robust mutexes Oct 15 20:47:06.830326 ncdv-master-1 devstack@n-api.service[138673]: thunder lock: enabled Oct 15 20:47:06.830326 ncdv-master-1 devstack@n-api.service[138673]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 15 20:47:06.830326 ncdv-master-1 devstack@n-api.service[138673]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 15 20:47:06.881061 ncdv-master-1 devstack@n-api.service[138673]: Python main interpreter initialized at 0x55b2a2a3c9c0 Oct 15 20:47:06.881061 ncdv-master-1 devstack@n-api.service[138673]: python threads support enabled Oct 15 20:47:06.881061 ncdv-master-1 devstack@n-api.service[138673]: your server socket listen backlog is limited to 100 connections Oct 15 20:47:06.881061 ncdv-master-1 devstack@n-api.service[138673]: your mercy for graceful operations on workers is 90 seconds Oct 15 20:47:06.881512 ncdv-master-1 devstack@n-api.service[138673]: mapped 403077 bytes (393 KB) for 2 cores Oct 15 20:47:06.881512 ncdv-master-1 devstack@n-api.service[138673]: *** Operational MODE: preforking *** Oct 15 20:47:06.881849 ncdv-master-1 devstack@n-api.service[138673]: *** uWSGI is running in multiple interpreter mode *** Oct 15 20:47:06.881849 ncdv-master-1 devstack@n-api.service[138673]: spawned uWSGI master process (pid: 138673) Oct 15 20:47:06.882078 ncdv-master-1 systemd[1]: Started Devstack devstack@n-api.service. Oct 15 20:47:06.883094 ncdv-master-1 devstack@n-api.service[138673]: spawned uWSGI worker 1 (pid: 138688, cores: 1) Oct 15 20:47:06.883094 ncdv-master-1 devstack@n-api.service[138673]: spawned uWSGI worker 2 (pid: 138689, cores: 1) Oct 15 20:47:06.883284 ncdv-master-1 devstack@n-api.service[138673]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 15 20:47:08.197783 ncdv-master-1 devstack@n-api.service[138688]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 15 20:47:08.267324 ncdv-master-1 devstack@n-api.service[138689]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 15 20:47:08.959880 ncdv-master-1 devstack@n-api.service[138688]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:47:08.959880 ncdv-master-1 devstack@n-api.service[138688]: from cryptography.utils import int_from_bytes Oct 15 20:47:08.960210 ncdv-master-1 devstack@n-api.service[138688]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:47:08.960210 ncdv-master-1 devstack@n-api.service[138688]: from cryptography.utils import int_from_bytes Oct 15 20:47:09.036789 ncdv-master-1 devstack@n-api.service[138689]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:47:09.036789 ncdv-master-1 devstack@n-api.service[138689]: from cryptography.utils import int_from_bytes Oct 15 20:47:09.037272 ncdv-master-1 devstack@n-api.service[138689]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 20:47:09.037272 ncdv-master-1 devstack@n-api.service[138689]: from cryptography.utils import int_from_bytes Oct 15 20:47:11.422664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 20:47:11.423274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 20:47:11.423469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 20:47:11.423947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 20:47:11.424251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 20:47:11.424654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.425065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.425482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.425810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.426211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.427310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.427745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.428437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.428833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.429261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.429663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.430094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.430475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.430836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.431161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.431491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.431804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.432921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.433273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.433833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.434502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.434820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.435235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.435591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.435917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.436237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.436636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.436998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.437418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.437800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.438506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.438844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.439285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.439689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.440346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.440851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.441207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.441543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.441886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.442206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.442546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.442875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.443202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.443530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.443888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.444137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.444455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.444782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.445121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.445434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.445689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.446014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.446329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.446654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.446984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.448727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.448727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.448976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.449325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.449649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.449967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.450282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.450597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.450910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.451368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.451546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.451843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.452170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.452495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.452824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.453212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.453806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.454086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.454344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.454658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.455009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.455334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.455647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.455971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.456296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.456607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.456918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.457262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.457587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.457917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.458226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.458600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.459077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.459398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.459781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.460043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.460358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.460677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.460974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.461377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.461641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.461931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.462243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.462556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.462869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.463205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.463522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.463958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.464126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.464450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.464775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.465262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.465431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.465712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.466022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.466345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.466655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.466963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.467288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.467599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.467908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.468231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.468566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.469174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.469505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.469814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.470124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.470512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.470787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.471065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.471375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.471721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.471987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.472294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.472600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.472906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.473235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.473559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.474107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.474287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.474488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.474796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.475119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.475462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.475800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.476147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.476465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.477072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.477412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.477735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.478052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.478370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.478699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.479243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.479569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.479887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.480210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.480535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.480904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.481212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.481539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.481870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.482173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.482495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.482827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.483152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.483465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.483811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.484318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.484501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.484795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.485151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.485469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.485784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.486101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.486434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.486816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.487139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.487463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.487779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.488106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.488422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.488737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.489223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.489512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.489832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.490172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.490492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.490820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.491142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.491463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.491814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.492100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.492416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.492731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.493128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.493418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.493726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.494038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.494968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.495184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.495518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.495828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.496168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.496478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.496792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.497127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.497443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.497753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.498069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.498382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.498726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.499018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.499436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.499752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.500080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.500391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.500701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.501019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.501355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.501673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.501993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.502304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.502631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.502948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.503262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.503603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.503937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.504246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.506997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.507149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.507383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.507693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.508002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.509009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.509009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.509009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.509761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.509761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.510718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.510915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.511317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.511577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.511894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.512283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.512581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.512902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.513325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.513643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.513963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.514280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.514680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.514906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.515316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.515547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.515863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.516187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.516505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.516833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.517172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.517499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.517811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.518135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.518481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.518820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.519141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.519694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.519862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.520122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.520437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.520754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.521102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.521426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.521729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.522043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.522391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.522675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.523020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.523308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.523628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.523935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.524251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.524584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.524917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.525254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.525574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.525930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.526093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 20:47:11.526585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 20:47:11.526738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.526738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.527039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.527192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 20:47:11.527350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.527504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 20:47:11.527656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.527806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 20:47:11.527958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.528102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.528251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.528413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.528571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.528715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.528879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.529023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.529248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.529401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.529573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.530016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.530164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.530317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.530467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.530620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.530764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.530906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.531055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.531211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.531363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.531537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.531765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.531915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.532067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.532219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.532466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.532704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.532852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.533003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.533189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.533346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.533490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.533649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.533800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.533949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.534091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.534243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.534401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.534401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.534682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.535020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.535192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.535344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.535498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.535745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.535894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.536035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.536185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.536342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.536486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.536628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.536786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.536937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.537109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.537284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.537429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.537580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.537723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.537864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.538005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.538160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.538302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.538444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.538587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.538728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.538883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.539033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.539228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.539384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.539384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.539655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.540088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.540249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.540402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.540545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.540695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.540847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.540997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.541165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.541868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.542103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.542270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.542442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.542669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.542888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.542888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.543163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.543163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.543516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.543670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.543814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.543955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.544109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.544266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.544418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.544575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.544726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.544884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.545252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.545405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.545557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.545557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.545946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.546112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.546257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.546411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.546553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.546694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.546845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.546989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.547134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.547276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.547435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.547584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.547739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.547881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.548036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.548185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.548336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.548492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.548642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.548784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.548926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.549102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.549295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.549453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.549596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.549747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.549917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.550346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.550490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.550642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.550792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.550934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.551082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.551226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.551368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.551519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.551663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.551811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.551961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.552121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.552347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.552511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.552681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.552681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.553165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.553165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.553465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.553618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.553776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.553927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.554070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.554212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.554383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.554527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.554676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.554827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.554970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.555112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.555440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.556317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.556483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.556638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.556835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.557082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.557745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.557900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.558044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.558196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.558339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.558491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.558642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.558798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.558941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.559098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.559255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.560439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.560601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.560759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.560912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.561155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.561313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.561458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.561611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.561763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.561917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.562060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.562221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.562395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.562619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.562777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.563041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.563284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.563434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.563584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.563728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.563878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.564031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.564175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.564326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.564477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.564620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.564771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.564925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.565099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.565260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.565643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.565806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.565962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.566114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.566283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.566535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.566684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.566834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.566995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.567147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.567289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.567457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.567601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.567789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.567939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.568109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.568262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.568409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.568559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.568711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.568857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.568936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.569021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.569135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.569222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.569302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.569419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.569545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.569652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.569740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.570055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.570193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.570365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.570675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.570778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.570871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.570956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.571038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.571119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.571213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.571306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.571400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.571491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.571581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.571664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.571754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.571754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.571996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.572092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.572183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.572183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.572353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.572353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.572561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.573202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.573472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.573568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.573650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.573805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.573903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.573995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.574180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 20:47:11.574363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.574959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.575946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.576925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.577918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.578130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.578230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.578447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.578539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.578768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.578865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.579073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.579176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.579387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.579534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.579705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.579830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.580965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.581123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.581307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.581398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.581618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.581709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.581936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.582981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.583193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.583283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.583501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.583618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.583810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.583911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.584116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.584215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.584426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.584526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.584732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.584832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.585061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.585202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.585378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.585516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.585692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.585829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.586996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.587098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.587309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.587400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.587632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.587724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.587949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.588040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.588798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.588975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.589147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.589240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.589456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.589609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.589765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.589866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.590079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.590170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.590390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.590482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.590744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.590858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.591959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.592056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.592269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.592365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.592587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.592678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.592901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.593112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.593262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.593354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.593587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.594394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.594487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.594568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.594650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.594836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.594974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.595912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.596045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.596234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.596356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.596550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.596674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.596916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.597947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.598172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.598261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.598501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.598589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.598822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.598911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.599137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.599217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.599465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.599553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.599771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.599875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.600103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.600191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.600440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.600522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.600763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.600918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.601127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.601234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.601395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.601478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.601963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.602982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.603219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.603306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.603527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.603670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.603873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.603973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.604171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.604253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.604483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.604677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.604776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.604885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.605111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.605236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.605444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.605536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.605753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.605871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.606080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.606176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.606402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.606507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.606732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.606830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.607061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.607166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.607374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.607523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.607704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.607806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.608942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.609055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.609278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.609451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.609594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.609712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.609914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.610005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.610239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.610335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.610553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.610658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.610877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.611024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.611186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.611322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.611534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.611856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.611947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.612173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.612273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.612495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.612586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.612807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.612907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.613146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.613254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.613455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.613604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.613767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.613955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.614086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.614280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.614373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.614608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.614700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.614926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.615955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.616167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.616263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.616471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.616566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.616793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.616900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.617130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.617251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.617444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.617538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.617750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.617859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.618057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.618155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.618373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.618478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.618697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.618803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.619006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.619105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.619323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.619455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.619675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.619866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.620066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.620209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.620394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.620528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.620716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.620847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.621065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.621200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.621384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.621666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.621773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.621989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.622161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.622322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.622416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.622603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.622702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.622913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.623003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.623227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.623333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.623545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.623796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.623887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.624135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.624237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.624455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.624548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.624769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.624859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.625098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.625207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.625410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.625503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.625718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.625817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.626977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.627078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.627300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.627393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.627633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.627738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.627923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.628016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.628249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.628331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.628559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.628647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.628866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.629938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.630170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.630263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.630554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.630646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.630794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.630883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.631104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.631480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.631735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.632969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.633242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.633574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.633903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.634228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.634386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.634545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.634741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.634923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.635595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.635845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.635938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.636229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.636329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.636558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.636655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.636876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.636976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.637222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.637341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.637544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.637656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.637852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.638025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.638167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.638384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.638483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.638705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.639043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.639141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.639374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.639473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.639698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.639786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.640991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.641129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.641330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.641439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.641648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.641753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.641966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.642066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.642277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.642370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.642605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.642697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.642919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.643930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.644140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.644379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.644484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.644737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.644840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.645092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.645201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.645372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.645471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.645681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.645780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.646933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.647997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.648201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.648307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.648505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.648612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.648820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.648918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.649165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.649274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.649477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.649578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.649792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.649883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.650102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.650200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.650433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.650536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.651383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.651509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.651702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.651845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.651947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.652969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.653204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.653296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.653515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.653607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.653828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.653928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.654135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.654224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.654446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.654556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.654750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.654837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.655066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.655166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.655415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.655513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.655720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.655811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.656963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.657108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.657312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.657424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.657629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.657735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.657960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.658072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.658280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.658390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.658595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.658685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.658914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.659015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.659237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.659415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.659535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.659783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.660683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.660789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.660875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.660948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.661985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.662996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.663216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.663298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.663529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.663622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.663843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.663925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.664149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.664238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.664473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.664555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.664769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.664861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.665094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.665228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.665415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.665730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.665992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.666113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.666416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.666542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.666734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.666867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.667063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.667157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.667378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.667500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.667700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.667844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.668981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.669095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.669317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.669430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.669629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.669739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.669989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.670089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.670301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.670449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.670634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.670737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.670952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.671052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.671265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.671346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.671625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.671762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.672923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.673015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.673276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.673382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.673595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.673697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.673910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.674985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.675415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.675510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.675599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.675683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.675807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.675917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.676131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.676269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.676438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.676529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.676744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.676843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.677079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.677184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.677402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.677493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.677715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.677832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.678951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.679180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.679277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.679497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.679588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.679818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.679921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.680136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.680229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.680438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.680522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.680746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.680849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.681083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.681203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.681377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.681472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.681691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.681787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.682046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.682195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.682460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.682566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.682770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.682873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.683082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.683175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.683391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.683483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.683700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.683808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.684007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.684143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.684344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.684451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.684683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.684774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.685003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.685118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.685359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.685484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.685680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.685788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.686963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.687054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.687286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.687388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.687601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.687700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.687939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.688962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.689213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.689311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.689524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.689615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.689836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.689926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.690146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.690237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.690471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.690571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.690779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.690872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.691104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.691203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.691423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.691513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.691751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.691856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.692077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.692176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.692394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.692528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.692728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.692830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.693060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.693156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.693382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.693482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.693699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.693801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.694969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.695085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.695286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.695387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.695608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.695716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.695928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.696986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.697246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.697345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.698088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.698466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.698575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.698652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.698749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.698981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.699096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.699400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.699720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.700043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.700151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.700353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.700458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.700598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 20:47:11.700731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.701087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.701407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.701721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.702039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.702430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.702657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.703127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.703493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.703764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.704085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.704400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.704711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.705026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.705378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.705697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.706015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.706559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.706674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.706996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.707310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.707639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.707955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.708265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.708577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.708887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.709335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.709653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.709987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.710324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.710661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.710976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.711317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.711647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.711974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.712383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.712725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.713081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.714105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.714424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.714740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.715053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.715364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.715675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.715991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.716299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.716611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.716923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.717275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.717585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.717914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.718228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.718535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.718843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.719152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.719498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.719809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.720146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.720461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.720790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.721147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.721527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.721792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.722111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.722433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.722751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.723069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.723381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.723700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.724016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.724336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.724896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.725256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.725575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.725911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.726211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.726812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.727122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.727439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.727752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.728116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.728380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.728775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.729089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.729420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.729756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.730086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.730408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.730729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.731047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.731408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.731726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.732033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.732352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.732673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.732992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.733450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.733768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.734089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.734407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.734724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.735038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.735358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.735679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.736010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.736348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.736668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.736988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.737334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.737655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.738187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.738504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.738890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.739166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.739493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.739812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.740138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.740467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.740777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.741326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.741443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.741724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.742035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.742360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.742673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.742988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.743298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.743608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.744147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.744714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.745070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.745395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.745726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.746048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.746359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.746685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.747033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.747300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.747619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.747934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.748244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.748553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.748868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.749220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.749530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.749843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.750152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.750487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.750797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.751125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.751506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.751719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.752025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.752331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.752640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.752945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.753284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.753593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.753907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.754213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.754698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.755121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.755443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.755753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.756084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.756395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.756706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.757030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.757366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.757696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.758013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.758343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.758678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.758998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.759548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.759893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.760751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.761121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.761604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.761776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.762072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.762384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.762716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.763031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.763350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.763663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.763979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.764295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.764608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.764922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.765263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.765579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.765892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.766208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.766522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.766836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.767149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.767473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.767817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.768138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.768460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.768780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.769123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.769441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.769754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.770067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.770225 ncdv-master-1 devstack@n-api.service[138688]: INFO dbcounter [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Registered counter for database nova_api Oct 15 20:47:11.770402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.770690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.771087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.771418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.771801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.772066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.772381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.772705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.773042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.773450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.773693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.774019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.774366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.774650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.774963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.775296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.775659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.776521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.776848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.777178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.777526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.777794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.778274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.778599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.778915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.779223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.779534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.779844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.780151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.780464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.780772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.781104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.781417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.781871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.782050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.782321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.782626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.782932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.783241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.783586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.783851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.784156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.784657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.784974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.785310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.785649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.785989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.786309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.786658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.787653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_db.sqlalchemy.engines [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138688) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:11.793816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.793816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.794121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.794238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.794637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.794984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.795371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.795709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.796103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.796455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.796910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.797435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.797794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.798172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.798528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.798881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.799325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.799738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.800097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.800431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.800764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.801192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.801535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.801874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writer thread running {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:11.802008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.802416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.802754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.803102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.803471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.803869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.804205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.804570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.804922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.805276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.805607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.805929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 20:47:11.806186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 20:47:11.867666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.context [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=138688) load_cells /opt/stack/nova/nova/context.py:464}} Oct 15 20:47:11.872014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:11.872627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:11.874360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:11.875226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:11.875896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:11.878111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:11.879140 ncdv-master-1 devstack@n-api.service[138689]: INFO dbcounter [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Registered counter for database nova_api Oct 15 20:47:11.880558 ncdv-master-1 devstack@n-api.service[138688]: INFO dbcounter [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Registered counter for database nova_cell0 Oct 15 20:47:11.885990 ncdv-master-1 devstack@n-api.service[138688]: INFO dbcounter [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Registered counter for database nova_cell1 Oct 15 20:47:11.893458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_db.sqlalchemy.engines [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138688) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:11.898513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_db.sqlalchemy.engines [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138688) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:11.900728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_db.sqlalchemy.engines [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138689) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:11.905138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writer thread running {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:11.908543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writer thread running {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:11.971017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.context [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=138689) load_cells /opt/stack/nova/nova/context.py:464}} Oct 15 20:47:11.974957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:11.975446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:11.977337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:11.978101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:11.978564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:11.980669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:11.982748 ncdv-master-1 devstack@n-api.service[138689]: INFO dbcounter [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Registered counter for database nova_cell0 Oct 15 20:47:11.986982 ncdv-master-1 devstack@n-api.service[138689]: INFO dbcounter [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Registered counter for database nova_cell1 Oct 15 20:47:11.994335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_db.sqlalchemy.engines [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138689) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:11.997696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_db.sqlalchemy.engines [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138689) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:12.001878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writer thread running {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:12.049058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writer thread running {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:12.059878 ncdv-master-1 devstack@n-api.service[138688]: INFO dbcounter [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Registered counter for database nova_cell0 Oct 15 20:47:12.066693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_db.sqlalchemy.engines [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138688) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:12.071314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writer thread running {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:12.145526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writer thread running {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:12.155925 ncdv-master-1 devstack@n-api.service[138689]: INFO dbcounter [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Registered counter for database nova_cell0 Oct 15 20:47:12.162729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_db.sqlalchemy.engines [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138689) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 20:47:12.167123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writer thread running {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 20:47:12.508564 ncdv-master-1 devstack@n-api.service[138688]: WARNING keystonemiddleware.auth_token [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:47:12.517645 ncdv-master-1 devstack@n-api.service[138688]: WARNING keystonemiddleware.auth_token [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:47:12.611889 ncdv-master-1 devstack@n-api.service[138689]: WARNING keystonemiddleware.auth_token [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:47:12.620076 ncdv-master-1 devstack@n-api.service[138689]: WARNING keystonemiddleware.auth_token [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:47:12.746245 ncdv-master-1 devstack@n-api.service[138688]: WARNING keystonemiddleware.auth_token [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:47:12.752327 ncdv-master-1 devstack@n-api.service[138688]: WARNING keystonemiddleware.auth_token [None req-db33f45f-0526-4cea-a41a-42d4ef5ceba3 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:47:12.757588 ncdv-master-1 devstack@n-api.service[138688]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b2a2a3c9c0 pid: 138688 (default app) Oct 15 20:47:12.855387 ncdv-master-1 devstack@n-api.service[138689]: WARNING keystonemiddleware.auth_token [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 15 20:47:12.862941 ncdv-master-1 devstack@n-api.service[138689]: WARNING keystonemiddleware.auth_token [None req-612fef57-73cf-4790-8488-d9f56e1d2c3f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 15 20:47:12.868467 ncdv-master-1 devstack@n-api.service[138689]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b2a2a3c9c0 pid: 138689 (default app) Oct 15 20:47:54.011899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.013678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.015093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.017324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.208635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.210308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.211548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:54.212806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:55.394442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Calling method '.version_select at 0x7fe75550e700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:55.405956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=138688) load_cells /opt/stack/nova/nova/compute/api.py:319}} Oct 15 20:47:55.407001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:55.407559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:55.408079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:55.438599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:55.439148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:55.439660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:55.467106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:55.467657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:55.468324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:55.476214 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8993e387-9de0-4511-a69a-56da1290b2e4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1423 microversion: 2.41 time: 1.477487 Oct 15 20:47:55.489344 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1/1] 10.222.0.16 () {60 vars in 1208 bytes} [Sat Oct 15 20:47:53 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1423 bytes in 1484 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:47:55.731286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-49c8a417-e1ae-487d-80ac-406547fb1539 tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] Calling method '.version_select at 0x7fe75550d5e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:55.763906 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-49c8a417-e1ae-487d-80ac-406547fb1539 tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 172 microversion: 2.57 time: 1.580726 Oct 15 20:47:55.764500 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1/2] 10.222.0.16 () {60 vars in 1226 bytes} [Sat Oct 15 20:47:54 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 172 bytes in 1583 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 20:47:55.797713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-06edeca0-ca52-4907-a949-5977876e4f6d tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-11340920"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:55.821281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-06edeca0-ca52-4907-a949-5977876e4f6d tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Generated uuid b0b1f891-fa5f-4420-a22b-0004dfb231c5 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:47:55.932982 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-06edeca0-ca52-4907-a949-5977876e4f6d tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 244 microversion: 2.41 time: 0.138961 Oct 15 20:47:55.934237 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2/3] 10.222.0.16 () {62 vars in 1205 bytes} [Sat Oct 15 20:47:55 2022] POST /compute/v2.1/os-aggregates => generated 244 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:55.941610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b3094ef2-b027-4180-9576-f98ce21d2200 tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] Action: 'update', calling method: .version_select at 0x7fe75537a280>, body: {"quota_class_set": {"cores": 120, "instances": 110, "key_pairs": 200, "metadata_items": 228, "ram": 51300, "server_group_members": 110, "server_groups": 110}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:56.166158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae6dd19a-5bd2-4f91-a94d-25a6a57068d3 tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsV261TestJSON-flavor-2049250414", "id": 686426055, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:56.290451 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae6dd19a-5bd2-4f91-a94d-25a6a57068d3 tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 465 microversion: 2.61 time: 0.807154 Oct 15 20:47:56.300917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2/4] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:47:55 2022] POST /compute/v2.1/flavors => generated 465 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:56.306439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02cf10b6-38b5-4310-a1c2-f873476015f4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1900685042"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:56.348596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b3094ef2-b027-4180-9576-f98ce21d2200 tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 159 microversion: 2.57 time: 0.410852 Oct 15 20:47:56.349326 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 3/5] 10.222.0.16 () {62 vars in 1247 bytes} [Sat Oct 15 20:47:55 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 159 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:56.566079 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02cf10b6-38b5-4310-a1c2-f873476015f4 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-aggregates/1" status: 200 len: 299 microversion: 2.41 time: 0.272402 Oct 15 20:47:56.573917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3/6] 10.222.0.16 () {62 vars in 1210 bytes} [Sat Oct 15 20:47:56 2022] PUT /compute/v2.1/os-aggregates/1 => generated 299 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:56.580199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-014d3010-d19c-481b-b565-7e6faf1cf6ba tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] Calling method '.version_select at 0x7fe7552dc820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:56.619494 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-014d3010-d19c-481b-b565-7e6faf1cf6ba tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 176 microversion: 2.57 time: 0.043371 Oct 15 20:47:56.620371 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4/7] 10.222.0.16 () {60 vars in 1226 bytes} [Sat Oct 15 20:47:56 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 176 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:56.853647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-44cc6d83-0afa-4a84-8a15-61f5ee644f78 tempest-FlavorsV261TestJSON-601841843 tempest-FlavorsV261TestJSON-601841843-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:56.900755 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-44cc6d83-0afa-4a84-8a15-61f5ee644f78 tempest-FlavorsV261TestJSON-601841843 tempest-FlavorsV261TestJSON-601841843-project] 10.222.0.16 "GET /compute/v2.1/flavors/686426055" status: 200 len: 465 microversion: 2.61 time: 0.550261 Oct 15 20:47:56.901944 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 4/8] 10.222.0.16 () {60 vars in 1196 bytes} [Sat Oct 15 20:47:56 2022] GET /compute/v2.1/flavors/686426055 => generated 465 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:56.914063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a17651af-3a6c-440b-b377-b5733995b77a tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:56.937806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a17651af-3a6c-440b-b377-b5733995b77a tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/1" status: 200 len: 299 microversion: 2.41 time: 0.034446 Oct 15 20:47:56.939031 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 5/9] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:47:56 2022] GET /compute/v2.1/os-aggregates/1 => generated 299 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:56.943946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ab0bdada-7abd-4a91-be5d-7f17bf24d927 tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] Action: 'update', calling method: .version_select at 0x7fe7553d8040>, body: {"quota_class_set": {"cores": 20, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "server_group_members": 10, "server_groups": 10}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:57.057927 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ab0bdada-7abd-4a91-be5d-7f17bf24d927 tempest-QuotaClassesAdmin257Test-1143951036 tempest-QuotaClassesAdmin257Test-1143951036-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 155 microversion: 2.57 time: 0.117515 Oct 15 20:47:57.060034 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 6/10] 10.222.0.16 () {62 vars in 1247 bytes} [Sat Oct 15 20:47:56 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 155 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:57.278902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4c74564-6643-4af9-a4f6-312fc5b84514 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": "1024", "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-852858772", "id": 2004168586, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:57.303219 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4c74564-6643-4af9-a4f6-312fc5b84514 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 429 microversion: 2.1 time: 0.681421 Oct 15 20:47:57.304638 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 5/11] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:56 2022] POST /compute/v2.1/flavors => generated 429 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:57.310251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb5da955-9a68-4d81-9398-56e02eb38f8d tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] Action: 'update', calling method: .version_select at 0x7fe7558c75e0>, body: {"flavor": {"description": "new"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:57.389569 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb5da955-9a68-4d81-9398-56e02eb38f8d tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/686426055" status: 200 len: 466 microversion: 2.61 time: 0.082945 Oct 15 20:47:57.390370 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 6/12] 10.222.0.16 () {62 vars in 1216 bytes} [Sat Oct 15 20:47:57 2022] PUT /compute/v2.1/flavors/686426055 => generated 466 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:57.397681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ead1b718-2b1c-4ead-b4bb-23f6eceecf2c tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:57.416175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ead1b718-2b1c-4ead-b4bb-23f6eceecf2c tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:57.417321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ead1b718-2b1c-4ead-b4bb-23f6eceecf2c tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:57.417912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ead1b718-2b1c-4ead-b4bb-23f6eceecf2c tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:57.489509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '.version_select at 0x7fe7553d80d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:57.501304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=138689) load_cells /opt/stack/nova/nova/compute/api.py:319}} Oct 15 20:47:57.506732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:57.507618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:57.508275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:57.584979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:57.585808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:57.586946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:57.631350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:57.635822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:57.636934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:57.654500 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2afdee8a-5253-43cb-8b3e-4c516d0a7b96 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1503 microversion: 2.1 time: 0.592736 Oct 15 20:47:57.655534 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 20:47:57 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1503 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:47:58.127670 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ead1b718-2b1c-4ead-b4bb-23f6eceecf2c tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/1/action" status: 200 len: 314 microversion: 2.41 time: 0.735797 Oct 15 20:47:58.128285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 7/14] 10.222.0.16 () {62 vars in 1232 bytes} [Sat Oct 15 20:47:57 2022] POST /compute/v2.1/os-aggregates/1/action => generated 314 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:58.135357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bfaaa0b-30b4-4c8d-997e-6a6a01af8d87 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.140054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.145314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:58.145798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:58.146247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:58.147930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:58.147930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:58.148308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:58.211693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9bfaaa0b-30b4-4c8d-997e-6a6a01af8d87 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2004168586" status: 202 len: 0 microversion: 2.1 time: 0.081858 Oct 15 20:47:58.212353 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 8/15] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:47:58 2022] DELETE /compute/v2.1/flavors/2004168586 => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:47:58.217005 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2fabdf5d-7dd4-46f4-8493-03ee45a6d09c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.558142 Oct 15 20:47:58.227752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f72d5d13-cd12-46f1-b686-6288c9c17a7b tempest-FlavorsV261TestJSON-601841843 tempest-FlavorsV261TestJSON-601841843-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.227934 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:47:57 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:58.227934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ffe75e33-1362-4362-a81c-6f21b41f10a8 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:58.240110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ffe75e33-1362-4362-a81c-6f21b41f10a8 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:58.240619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ffe75e33-1362-4362-a81c-6f21b41f10a8 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:58.241318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ffe75e33-1362-4362-a81c-6f21b41f10a8 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:58.291655 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f72d5d13-cd12-46f1-b686-6288c9c17a7b tempest-FlavorsV261TestJSON-601841843 tempest-FlavorsV261TestJSON-601841843-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6247 microversion: 2.61 time: 0.078204 Oct 15 20:47:58.292833 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 9/17] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors/detail => generated 6247 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:47:58.314018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b996c1d4-e837-45c8-a621-d46a3f2777e5 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.357534 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b996c1d4-e837-45c8-a621-d46a3f2777e5 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5406 microversion: 2.1 time: 0.062967 Oct 15 20:47:58.358983 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 10/18] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors/detail => generated 5406 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:47:58.370460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b47e0252-3695-4d2f-bde5-40f32576ad3d tempest-FlavorsV261TestJSON-601841843 tempest-FlavorsV261TestJSON-601841843-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.402504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b47e0252-3695-4d2f-bde5-40f32576ad3d tempest-FlavorsV261TestJSON-601841843 tempest-FlavorsV261TestJSON-601841843-project] 10.222.0.16 "GET /compute/v2.1/flavors" status: 200 len: 3154 microversion: 2.61 time: 0.040370 Oct 15 20:47:58.403869 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 11/19] 10.222.0.16 () {60 vars in 1166 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors => generated 3154 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:47:58.417290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1e0306d3-4ed7-4ead-a863-e62691d49fbd tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-794693052", "id": 1866722497, "swap": 1024, "rxtx_factor": 2, "OS-FLV-EXT-DATA:ephemeral": 10, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:58.468507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1e0306d3-4ed7-4ead-a863-e62691d49fbd tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 416 microversion: 2.1 time: 0.062922 Oct 15 20:47:58.476655 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 12/20] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:58 2022] POST /compute/v2.1/flavors => generated 416 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:58.481978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81ae9fba-0ea5-4977-b86f-9a2b50daf320 tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.503187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81ae9fba-0ea5-4977-b86f-9a2b50daf320 tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/686426055" status: 202 len: 0 microversion: 2.61 time: 0.025140 Oct 15 20:47:58.507443 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 13/21] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Oct 15 20:47:58 2022] DELETE /compute/v2.1/flavors/686426055 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:47:58.512392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18ee802f-3569-46a7-86a6-81610659e9b7 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.572035 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18ee802f-3569-46a7-86a6-81610659e9b7 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5397 microversion: 2.1 time: 0.048978 Oct 15 20:47:58.573157 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 14/22] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors/detail => generated 5397 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:47:58.586406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61c04d48-609e-4d03-8acc-bf4a1c353ba4 tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.638077 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61c04d48-609e-4d03-8acc-bf4a1c353ba4 tempest-FlavorsV261TestJSON-1507464908 tempest-FlavorsV261TestJSON-1507464908-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6237 microversion: 2.61 time: 0.058069 Oct 15 20:47:58.651559 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 15/23] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors/detail => generated 6237 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:47:58.666790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f16ec11-ace3-4051-9c08-8f72ad77a342 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.715645 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f16ec11-ace3-4051-9c08-8f72ad77a342 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1866722497" status: 202 len: 0 microversion: 2.1 time: 0.062684 Oct 15 20:47:58.716710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 16/24] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:47:58 2022] DELETE /compute/v2.1/flavors/1866722497 => generated 0 bytes in 64 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:47:58.725507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b942341-d150-4af5-b8ec-4b338f1efcf3 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.747102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b942341-d150-4af5-b8ec-4b338f1efcf3 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.026314 Oct 15 20:47:58.748354 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 17/25] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:47:58.772560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26c244dd-a57d-4118-990d-27937001625c tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-1488751617", "id": 727509496, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:58.805737 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26c244dd-a57d-4118-990d-27937001625c tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 426 microversion: 2.1 time: 0.036719 Oct 15 20:47:58.806350 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 18/26] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:58 2022] POST /compute/v2.1/flavors => generated 426 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:58.823681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6af473b-042e-49e4-9076-bc8b64189080 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.825754 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ffe75e33-1362-4362-a81c-6f21b41f10a8 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/1/action" status: 200 len: 299 microversion: 2.41 time: 0.606342 Oct 15 20:47:58.831705 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 9/27] 10.222.0.16 () {62 vars in 1232 bytes} [Sat Oct 15 20:47:58 2022] POST /compute/v2.1/os-aggregates/1/action => generated 299 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:47:58.841169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31be5f2b-da96-40e1-ab78-c02b78197638 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.855206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6af473b-042e-49e4-9076-bc8b64189080 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/727509496" status: 202 len: 0 microversion: 2.1 time: 0.038840 Oct 15 20:47:58.856762 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 19/28] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:47:58 2022] DELETE /compute/v2.1/flavors/727509496 => generated 0 bytes in 42 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:47:58.868621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32adebdd-dfb5-476c-8b89-f93c135e3056 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:58.908325 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31be5f2b-da96-40e1-ab78-c02b78197638 tempest-AggregatesAdminTestV241-1883899033 tempest-AggregatesAdminTestV241-1883899033-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/1" status: 200 len: 0 microversion: 2.41 time: 0.072446 Oct 15 20:47:58.910350 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 10/29] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 20:47:58 2022] DELETE /compute/v2.1/os-aggregates/1 => generated 0 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:58.930785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-465f9a61-a3a8-4d47-8ccd-88a4608b38ac tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"availability_zone": "tempest-test_az-720847984", "name": "tempest-test_aggregate-127817627"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:58.943287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32adebdd-dfb5-476c-8b89-f93c135e3056 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.081432 Oct 15 20:47:58.947644 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 20/30] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:58 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:47:58.971987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-383575ef-012f-4f3f-a236-34dfa7474a8e tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-746261602", "id": null, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:58.993482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-465f9a61-a3a8-4d47-8ccd-88a4608b38ac tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 4dd4d653-f5ac-4249-b565-b5437422c1ce for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:47:59.011939 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-383575ef-012f-4f3f-a236-34dfa7474a8e tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 506 microversion: 2.1 time: 0.043450 Oct 15 20:47:59.012813 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 21/31] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:58 2022] POST /compute/v2.1/flavors => generated 506 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:47:59.027193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c6185c4-288e-4764-a79b-6921f8808c7f tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:59.054972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c6185c4-288e-4764-a79b-6921f8808c7f tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/59f29d89-d102-4e40-b94d-b43e888ad648" status: 202 len: 0 microversion: 2.1 time: 0.033178 Oct 15 20:47:59.058392 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 22/32] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:47:59 2022] DELETE /compute/v2.1/flavors/59f29d89-d102-4e40-b94d-b43e888ad648 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:47:59.068087 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-465f9a61-a3a8-4d47-8ccd-88a4608b38ac tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 220 microversion: 2.1 time: 0.142263 Oct 15 20:47:59.076007 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 11/33] 10.222.0.16 () {60 vars in 1165 bytes} [Sat Oct 15 20:47:58 2022] POST /compute/v2.1/os-aggregates => generated 220 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:59.082822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b60a1a6f-3846-4e45-8904-87e439853ba2 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:59.086697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-375572ce-03e1-48fe-b66c-167126c1df73 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:59.115603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b60a1a6f-3846-4e45-8904-87e439853ba2 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.052311 Oct 15 20:47:59.116652 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 23/34] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:59 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:47:59.121448 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-375572ce-03e1-48fe-b66c-167126c1df73 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 300 microversion: 2.1 time: 0.042546 Oct 15 20:47:59.122221 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 12/35] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:47:59 2022] GET /compute/v2.1/os-aggregates => generated 300 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:59.135041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-632ef63d-55d8-4c2c-9073-8ff26ceea474 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:59.143224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-96fe2413-975e-42b3-8fb0-47afbcde0f41 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-1543309200", "id": "1f63ffac-8ecf-470d-b15f-8a293635e59f", "os-flavor-access:is_public": "True"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:59.143858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=6,nova_api:UPDATE=2,nova_api:DELETE=15 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:59.150893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-632ef63d-55d8-4c2c-9073-8ff26ceea474 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:47:59.151858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-632ef63d-55d8-4c2c-9073-8ff26ceea474 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:47:59.152472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-632ef63d-55d8-4c2c-9073-8ff26ceea474 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:47:59.215171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=11,nova_api:UPDATE=14,nova_api:DELETE=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:47:59.227424 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-96fe2413-975e-42b3-8fb0-47afbcde0f41 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 507 microversion: 2.1 time: 0.091210 Oct 15 20:47:59.228381 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 13/36] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:59 2022] POST /compute/v2.1/flavors => generated 507 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:47:59.241118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-84c33eac-061a-41c4-b3f9-3968646fab52 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:59.308593 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-84c33eac-061a-41c4-b3f9-3968646fab52 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1f63ffac-8ecf-470d-b15f-8a293635e59f" status: 202 len: 0 microversion: 2.1 time: 0.067453 Oct 15 20:47:59.309762 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 14/37] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:47:59 2022] DELETE /compute/v2.1/flavors/1f63ffac-8ecf-470d-b15f-8a293635e59f => generated 0 bytes in 74 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:47:59.318708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-db524d7f-1305-4877-ab2e-c5e50331744d tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:59.359526 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-db524d7f-1305-4877-ab2e-c5e50331744d tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.044843 Oct 15 20:47:59.374117 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 15/38] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:59 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:47:59.387206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d88f34a3-052f-49d6-be09-96f507f90437 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-569284121", "id": 1218188067, "os-flavor-access:is_public": "True"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:47:59.437190 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d88f34a3-052f-49d6-be09-96f507f90437 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 413 microversion: 2.1 time: 0.049834 Oct 15 20:47:59.439611 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 16/39] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:59 2022] POST /compute/v2.1/flavors => generated 413 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:59.459092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d2ad9eeb-4614-45d2-854b-99ceda2cf340 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:47:59.481231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-632ef63d-55d8-4c2c-9073-8ff26ceea474 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/2/action" status: 200 len: 312 microversion: 2.1 time: 0.350865 Oct 15 20:47:59.482386 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 24/40] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:47:59 2022] POST /compute/v2.1/os-aggregates/2/action => generated 312 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:47:59.483905 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d2ad9eeb-4614-45d2-854b-99ceda2cf340 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/1218188067" status: 200 len: 413 microversion: 2.1 time: 0.034513 Oct 15 20:47:59.492696 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 17/41] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:47:59 2022] GET /compute/v2.1/flavors/1218188067 => generated 413 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:00.211883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AggregatesAdminTestJSON-server-751816934", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "availability_zone": "tempest-test_az-720847984"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:00.233831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:00.240795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-681f849f-db79-4da8-b431-f2535080f04c tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.257326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:00.260658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:00.263268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:00.264003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:00.264548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:00.310108 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-681f849f-db79-4da8-b431-f2535080f04c tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5394 microversion: 2.1 time: 0.816501 Oct 15 20:48:00.312902 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 25/42] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:47:59 2022] GET /compute/v2.1/flavors/detail => generated 5394 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:00.332418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4f74058-7adc-42c7-b8bb-98705b48c380 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.368357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4f74058-7adc-42c7-b8bb-98705b48c380 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1218188067" status: 202 len: 0 microversion: 2.1 time: 0.038875 Oct 15 20:48:00.375216 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 26/43] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:48:00 2022] DELETE /compute/v2.1/flavors/1218188067 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:48:00.391030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50c354db-70f5-44de-838b-dd10045afa6a tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.445994 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50c354db-70f5-44de-838b-dd10045afa6a tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.068975 Oct 15 20:48:00.458463 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 27/44] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:00 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:00.470557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20fafa58-1919-46af-bd2b-a8fd452af059 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-1097146150", "id": 2037668667, "os-flavor-access:is_public": "False"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:00.498186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20fafa58-1919-46af-bd2b-a8fd452af059 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 430 microversion: 2.1 time: 0.031101 Oct 15 20:48:00.498822 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 28/45] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:00 2022] POST /compute/v2.1/flavors => generated 430 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:00.527179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8f89123-1a06-4d07-8f67-e189c9567216 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] Action: 'create', calling method: >, body: {"server": {"name": "test", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "2037668667"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:00.564672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a8f89123-1a06-4d07-8f67-e189c9567216 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] HTTP exception thrown: Flavor 2037668667 could not be found. Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8f89123-1a06-4d07-8f67-e189c9567216 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] Request method failure captured: Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 107 Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx5Mqy6NuVH2RRxsGkXfJqJ4p-4d3zU7Tn15kUV490h2xF9KsoeRE_ZdR2dhdgBlk32fHU-Ezqljc6Pp-GmMeMtveHHf3fmkao5yyeg8dPpXRfDi76m4awUyYJlbFK7l9e-yqarSE6t4tJcmJI8dL7d7zEX4MkMN1BY5zh-PAKNPPY Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 732d361e9c7041cea306ecc4805ffdaf Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsAdminTestJSON-1554255855 Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:00.594175 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/732d361e9c7041cea306ecc4805ffdaf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_732d361e9c7041cea306ecc4805ffdaf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/732d361e9c7041cea306ecc4805ffdaf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/732d361e9c7041cea306ecc4805ffdaf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsAdminTestJSON-1554255855 Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 732d361e9c7041cea306ecc4805ffdaf Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsAdminTestJSON-1554255855 Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsAdminTestJSON-1554255855-project Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6b5a19d928dc47e48486a58359778635 Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsAdminTestJSON-1554255855-project Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "test", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "2037668667"}} Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:00.595182 ncdv-master-1 devstack@n-api.service[138688]: exception: Flavor 2037668667 could not be found. Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'test', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '2037668667'}}} Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 778, in create Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id( Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 2037668667 could not be found. Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:00.595776 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor 2037668667 could not be found. Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:00.596744 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 2037668667 could not be found. Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8f89123-1a06-4d07-8f67-e189c9567216 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] Returning 400 to user: Flavor 2037668667 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8f89123-1a06-4d07-8f67-e189c9567216 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 81 microversion: 2.1 time: 0.079147 Oct 15 20:48:00.599177 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 29/46] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:00 2022] POST /compute/v2.1/servers => generated 81 bytes in 83 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:00.611748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0d53c40-8869-435f-9830-450593466933 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.653666 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0d53c40-8869-435f-9830-450593466933 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2037668667" status: 202 len: 0 microversion: 2.1 time: 0.046600 Oct 15 20:48:00.655127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 30/47] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:48:00 2022] DELETE /compute/v2.1/flavors/2037668667 => generated 0 bytes in 54 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:48:00.672651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-857f0f7d-b939-447f-9634-fddd2575fb30 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.710545 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-857f0f7d-b939-447f-9634-fddd2575fb30 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.041434 Oct 15 20:48:00.721302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 31/48] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:00 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:00.739122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a9c8e73-92e8-4aec-8fc3-0842651143a1 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-2031161386", "id": 2020818967, "os-flavor-access:is_public": "False"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:00.799014 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a9c8e73-92e8-4aec-8fc3-0842651143a1 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 415 microversion: 2.1 time: 0.065592 Oct 15 20:48:00.799618 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 32/49] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:00 2022] POST /compute/v2.1/flavors => generated 415 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:00.815194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e257190-d65e-4372-a4ab-1d6abbaf55d1 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-548649057", "id": 241288529, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:00.875098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e257190-d65e-4372-a4ab-1d6abbaf55d1 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 410 microversion: 2.1 time: 0.064094 Oct 15 20:48:00.876282 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 33/50] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:00 2022] POST /compute/v2.1/flavors => generated 410 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:00.889654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb2681ee-0063-433a-a5d1-de149dbfa193 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.928583 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb2681ee-0063-433a-a5d1-de149dbfa193 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=f" status: 200 len: 418 microversion: 2.1 time: 0.043415 Oct 15 20:48:00.930874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 34/51] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Oct 15 20:48:00 2022] GET /compute/v2.1/flavors/detail?is_public=f => generated 418 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:00.953018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de807f59-1591-4451-be34-55debe803202 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.966047 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de807f59-1591-4451-be34-55debe803202 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=false" status: 200 len: 418 microversion: 2.1 time: 0.021647 Oct 15 20:48:00.967039 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 35/52] 10.222.0.16 () {58 vars in 1177 bytes} [Sat Oct 15 20:48:00 2022] GET /compute/v2.1/flavors/detail?is_public=false => generated 418 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:00.982913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66d24837-089d-4346-a5ba-b9872a87c9ab tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:00.999367 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66d24837-089d-4346-a5ba-b9872a87c9ab tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=no" status: 200 len: 418 microversion: 2.1 time: 0.022303 Oct 15 20:48:01.000228 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 36/53] 10.222.0.16 () {58 vars in 1171 bytes} [Sat Oct 15 20:48:00 2022] GET /compute/v2.1/flavors/detail?is_public=no => generated 418 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:01.016975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b8889ac-0c56-4455-b421-aa9727c5c61a tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.030619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b8889ac-0c56-4455-b421-aa9727c5c61a tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=0" status: 200 len: 418 microversion: 2.1 time: 0.018665 Oct 15 20:48:01.032043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 37/54] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail?is_public=0 => generated 418 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:01.051284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46444ae5-6ce7-4a1d-b827-d5c314281acc tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.120382 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46444ae5-6ce7-4a1d-b827-d5c314281acc tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=t" status: 200 len: 5391 microversion: 2.1 time: 0.078793 Oct 15 20:48:01.122890 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 38/55] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail?is_public=t => generated 5391 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:01.145107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8f6af64-337a-4274-a6fb-cb4960b89918 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.184963 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8f6af64-337a-4274-a6fb-cb4960b89918 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=true" status: 200 len: 5391 microversion: 2.1 time: 0.043778 Oct 15 20:48:01.185975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 39/56] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail?is_public=true => generated 5391 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:01.211167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc244b9c-8db3-4fce-bf03-d41150619314 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.241545 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc244b9c-8db3-4fce-bf03-d41150619314 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=yes" status: 200 len: 5391 microversion: 2.1 time: 0.037015 Oct 15 20:48:01.243901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 40/57] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail?is_public=yes => generated 5391 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:01.262209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d24b3cf-427a-48f9-bd85-7b5536c1d8a7 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.285410 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d24b3cf-427a-48f9-bd85-7b5536c1d8a7 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=1" status: 200 len: 5391 microversion: 2.1 time: 0.027119 Oct 15 20:48:01.286534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 41/58] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail?is_public=1 => generated 5391 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:01.306467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fa5a1f0-3ad0-4e70-9182-49e0025ab06b tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.326175 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fa5a1f0-3ad0-4e70-9182-49e0025ab06b tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/241288529" status: 202 len: 0 microversion: 2.1 time: 0.023686 Oct 15 20:48:01.331611 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 42/59] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:48:01 2022] DELETE /compute/v2.1/flavors/241288529 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:48:01.344211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97618732-0a63-4fba-9254-f152b6bb6c28 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.376777 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97618732-0a63-4fba-9254-f152b6bb6c28 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.043211 Oct 15 20:48:01.405879 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 43/60] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:01.441709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07940814-a285-4fcd-8728-137ae69a5c48 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.490693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07940814-a285-4fcd-8728-137ae69a5c48 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2020818967" status: 202 len: 0 microversion: 2.1 time: 0.083344 Oct 15 20:48:01.496192 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 44/61] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:48:01 2022] DELETE /compute/v2.1/flavors/2020818967 => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:48:01.504074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:01.519599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-151ef829-9535-46eb-aa5f-660cdba1ebe4 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.587220 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-151ef829-9535-46eb-aa5f-660cdba1ebe4 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.089376 Oct 15 20:48:01.589174 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 45/62] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:01.628392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0085adf3-9e8d-4592-a766-80c38aaf435a tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-840624625", "id": 152318815, "os-flavor-access:is_public": "False"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:01.670784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0085adf3-9e8d-4592-a766-80c38aaf435a tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 411 microversion: 2.1 time: 0.053322 Oct 15 20:48:01.683558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 46/63] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:01 2022] POST /compute/v2.1/flavors => generated 411 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:01.700154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea74cbc4-69fb-4c26-8489-58c80ff2369d tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.755238 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea74cbc4-69fb-4c26-8489-58c80ff2369d tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.069846 Oct 15 20:48:01.760180 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 47/64] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:01.779865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4098b09a-0c23-495a-b86e-bc625d9f93af tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.824626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4098b09a-0c23-495a-b86e-bc625d9f93af tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.049690 Oct 15 20:48:01.825672 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 48/65] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:01.844184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e698260e-1183-482e-8e81-ab58fea49262 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.877438 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e698260e-1183-482e-8e81-ab58fea49262 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/152318815" status: 202 len: 0 microversion: 2.1 time: 0.037238 Oct 15 20:48:01.878536 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 49/66] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:48:01 2022] DELETE /compute/v2.1/flavors/152318815 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:48:01.887423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bddc6b4f-e41e-4cb3-8105-3c2b814c76e1 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:01.947914 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bddc6b4f-e41e-4cb3-8105-3c2b814c76e1 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.064840 Oct 15 20:48:01.949523 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 50/67] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:02.089478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:02.090057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:02.090672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:02.091246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:02.091928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:02.124722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:02.125280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:02.125724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:02.126468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:02.126926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:02.127352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:02.233129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Getting quotas for project 474c2d620e224f06b409adf776004cac. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:02.245540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Getting quotas for user 4b16bd0809f3411a91fae6c5e2f3acee and project 474c2d620e224f06b409adf776004cac. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:02.262383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:02.268575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:02.416011 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6d392ecc-cacd-4749-b0be-d84659250f99 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.593135 Oct 15 20:48:02.417168 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 18/68] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:47:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 2600 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:02.433527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-76195177-37d0-4edf-98b3-ea24d4a20dad tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1385668803", "id": 1605054527, "os-flavor-access:is_public": "True"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:02.497922 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-76195177-37d0-4edf-98b3-ea24d4a20dad tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 414 microversion: 2.1 time: 0.071681 Oct 15 20:48:02.499180 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 19/69] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:02 2022] POST /compute/v2.1/flavors => generated 414 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:02.528287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d8939db-168e-44cf-a639-dc58ea9a4ff1 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:02.529208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d8939db-168e-44cf-a639-dc58ea9a4ff1 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:02.735566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d8939db-168e-44cf-a639-dc58ea9a4ff1 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:02.764926 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d8939db-168e-44cf-a639-dc58ea9a4ff1 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1241 microversion: 2.1 time: 0.258278 Oct 15 20:48:02.765857 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 20/70] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:02 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1241 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:02.781666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-038070ae-ecbc-4955-8251-553aa71f1323 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:02.865789 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-038070ae-ecbc-4955-8251-553aa71f1323 tempest-FlavorsAdminTestJSON-1554255855 tempest-FlavorsAdminTestJSON-1554255855-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5395 microversion: 2.1 time: 0.096600 Oct 15 20:48:02.870036 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 21/71] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:02 2022] GET /compute/v2.1/flavors/detail => generated 5395 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:02.891965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc7f3fba-48b2-4e0b-8c62-a63b3934e1d2 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:02.896730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd01f39f-e9f0-4a43-a15e-61ebb5e37867 tempest-AZAdminNegativeTestJSON-1230971732 tempest-AZAdminNegativeTestJSON-1230971732-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:02.898425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fd01f39f-e9f0-4a43-a15e-61ebb5e37867 tempest-AZAdminNegativeTestJSON-1230971732 tempest-AZAdminNegativeTestJSON-1230971732-project] Policy check for os_compute_api:os-availability-zone:detail failed with credentials {'is_admin': False, 'user_id': '7ec562be10db4c2fb57476fda5a18867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2da218e9269f428fa9a89771f41aa244', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd01f39f-e9f0-4a43-a15e-61ebb5e37867 tempest-AZAdminNegativeTestJSON-1230971732 tempest-AZAdminNegativeTestJSON-1230971732-project] Request method failure captured: Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-availability-zone/detail HTTP/1.1 Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx-Nq8df8aJ3QnzVUDYNYEUpz2fRcXSlFBcYkIxsGmUeb70YohBYA0O-QY4lQucCkgo1Z7TYTkwl69TfZv1efAAItcXFeJJn2L-Sxw6yybr0c8F7xNLPsrbm6RrAbcMV5U0FWmjUcDd2C84obBD27mEzKdDhGrcl2VKm_ydbw2sQJs Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2da218e9269f428fa9a89771f41aa244 Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AZAdminNegativeTestJSON-1230971732 Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:02.902038 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2da218e9269f428fa9a89771f41aa244"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2da218e9269f428fa9a89771f41aa244"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2da218e9269f428fa9a89771f41aa244"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2da218e9269f428fa9a89771f41aa244"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AZAdminNegativeTestJSON-1230971732 Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2da218e9269f428fa9a89771f41aa244 Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AZAdminNegativeTestJSON-1230971732 Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AZAdminNegativeTestJSON-1230971732-project Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 7ec562be10db4c2fb57476fda5a18867 Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AZAdminNegativeTestJSON-1230971732-project Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Oct 15 20:48:02.903121 ncdv-master-1 devstack@n-api.service[138688]: action_args: {} Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/availability_zone.py", line 117, in detail Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(az_policies.POLICY_ROOT % 'detail', target={}) Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:02.903817 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Oct 15 20:48:02.904758 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:02.906376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd01f39f-e9f0-4a43-a15e-61ebb5e37867 tempest-AZAdminNegativeTestJSON-1230971732 tempest-AZAdminNegativeTestJSON-1230971732-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:02.907538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd01f39f-e9f0-4a43-a15e-61ebb5e37867 tempest-AZAdminNegativeTestJSON-1230971732 tempest-AZAdminNegativeTestJSON-1230971732-project] 10.222.0.16 "GET /compute/v2.1/os-availability-zone/detail" status: 403 len: 123 microversion: 2.1 time: 0.956618 Oct 15 20:48:02.908299 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 51/72] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 20:48:01 2022] GET /compute/v2.1/os-availability-zone/detail => generated 123 bytes in 958 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:02.947753 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc7f3fba-48b2-4e0b-8c62-a63b3934e1d2 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1605054527" status: 202 len: 0 microversion: 2.1 time: 0.062369 Oct 15 20:48:02.956684 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 22/73] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:48:02 2022] DELETE /compute/v2.1/flavors/1605054527 => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:48:02.957699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ad58de8-cce0-46ca-afb6-a9855ef9ed14 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:02.987303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ad58de8-cce0-46ca-afb6-a9855ef9ed14 tempest-FlavorsAdminTestJSON-1577489455 tempest-FlavorsAdminTestJSON-1577489455-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.033731 Oct 15 20:48:02.988089 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 52/74] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:02 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:03.799641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52384ad0-345e-45b5-ad7f-329d2231ad47 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:03.800398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52384ad0-345e-45b5-ad7f-329d2231ad47 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:03.898304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-52384ad0-345e-45b5-ad7f-329d2231ad47 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:03.905990 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52384ad0-345e-45b5-ad7f-329d2231ad47 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1241 microversion: 2.1 time: 0.115018 Oct 15 20:48:03.906576 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 23/75] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:03 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1241 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:04.936747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:04.938948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:04.980132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:04.981080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:04.983040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:04.984033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:04.986457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:04.988248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:06.615880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:06.797971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:06.829284 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6508f23e-f6e1-4a2c-aadb-617ce116f69a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1289 microversion: 2.1 time: 1.896686 Oct 15 20:48:06.849625 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 53/76] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:04 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1289 bytes in 1922 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:07.856384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:07.857677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:07.871297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:07.873569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:07.875328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:07.880789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:07.881629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:07.883642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:09.264483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:09.289855 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4fde1ddf-94d2-4514-bf5c-9b3ee3cb82ae tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 1.437613 Oct 15 20:48:09.298481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 24/77] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:07 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 1448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:09.609938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e6f8337-2818-45f0-830d-ce05fa8333d1 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] Calling method '.version_select at 0x7fe75492a8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:09.621864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=5,nova_api:DELETE=15 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:09.652033 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e6f8337-2818-45f0-830d-ce05fa8333d1 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 303 microversion: 2.1 time: 0.921968 Oct 15 20:48:09.661755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 54/78] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 20:48:08 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 303 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:09.675097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5255575a-781e-48aa-b3dd-bc2f169ac7a6 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] Action: 'update', calling method: .version_select at 0x7fe75538fc10>, body: {"quota_class_set": {"cores": 120, "fixed_ips": 99, "floating_ips": 99, "injected_file_content_bytes": 10340, "injected_file_path_bytes": 355, "injected_files": 105, "instances": 110, "key_pairs": 200, "metadata_items": 228, "ram": 51300, "security_group_rules": 99, "security_groups": 99}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:10.118392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=71,nova_api:DELETE=6,nova_api:INSERT=11,nova_api:UPDATE=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:10.222883 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5255575a-781e-48aa-b3dd-bc2f169ac7a6 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 290 microversion: 2.1 time: 0.552619 Oct 15 20:48:10.226863 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 25/79] 10.222.0.16 () {60 vars in 1206 bytes} [Sat Oct 15 20:48:09 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 290 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:10.237755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0daa92d0-099d-441a-acdf-be5fbe26eac5 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] Calling method '.version_select at 0x7fe75492a4c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:10.267517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0daa92d0-099d-441a-acdf-be5fbe26eac5 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 307 microversion: 2.1 time: 0.033732 Oct 15 20:48:10.277841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 55/80] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 20:48:10 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 307 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:10.289937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a4006c4-98e3-4684-b768-62bfed4577b8 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] Action: 'update', calling method: .version_select at 0x7fe755ea2af0>, body: {"quota_class_set": {"cores": 20, "fixed_ips": -1, "floating_ips": -1, "injected_file_content_bytes": 10240, "injected_file_path_bytes": 255, "injected_files": 5, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "security_group_rules": -1, "security_groups": -1}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:10.318491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:10.319515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:10.332902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:10.338797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:10.340424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:10.343348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:10.344128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:10.345351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:10.508320 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a4006c4-98e3-4684-b768-62bfed4577b8 tempest-QuotaClassesAdminTestJSON-107366912 tempest-QuotaClassesAdminTestJSON-107366912-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 286 microversion: 2.1 time: 0.222633 Oct 15 20:48:10.511163 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 26/81] 10.222.0.16 () {60 vars in 1206 bytes} [Sat Oct 15 20:48:10 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 286 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:11.228962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:11.240694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.927146 Oct 15 20:48:11.271035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 56/82] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:10 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 957 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:11.811955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:11.814048 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f14f17a-55e9-4e7a-8ef2-447197261002 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002742 Oct 15 20:48:11.818898 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 57/83] 10.222.0.16 () {56 vars in 886 bytes} [Sat Oct 15 20:48:11 2022] GET /compute/ => generated 377 bytes in 9 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Oct 15 20:48:12.015315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersAdminTestJSON-server-550274556", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:12.155755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:12.201400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:12.277637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:12.278375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:12.300865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:12.301483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:12.301948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:12.309721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:12.311484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:12.312234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:13.020516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:13.033768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3dcc8cb3-0f8a-4aa7-9a9a-2722c9e3e43a tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.761747 Oct 15 20:48:13.043609 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 58/84] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:12 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:14.201114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:14.202380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:14.202992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:14.203561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:14.204107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:14.234904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.236480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.238406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.240180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.243416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.252283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.343078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Getting quotas for project c126854c21954928af16db403a0b8919. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:14.371381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Getting quotas for user c5a178b6a74140d4910f77d811febc08 and project c126854c21954928af16db403a0b8919. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:14.373778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d00bcb49-6cd6-4773-ac83-506cfc487514", "tag": "2f9db696-0c68-4064-bfff-ec8a1ea639d8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:14.401024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.403561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.404386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.434787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:14.436986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:14.495054 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Creating event network-changed:2f9db696-0c68-4064-bfff-ec8a1ea639d8 for instance d00bcb49-6cd6-4773-ac83-506cfc487514 on nch1-master-1 Oct 15 20:48:14.497870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 20:48:14.498818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d8d6283-3f1e-4065-8d13-24334038178f req-14981bfc-8b1a-4cb2-bd97-05108c7cc392 service nova] Acquired lock "compute-rpcapi-router" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 20:48:14.503238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.505974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.507173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.509768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.515458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.518014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.541582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.compute.rpcapi [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 15 20:48:14.551580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] Releasing lock "compute-rpcapi-router" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 20:48:14.619058 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-447ac60e-aa45-4f23-915c-b04fdcc2d09c None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.669122 Oct 15 20:48:14.625910 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 59/85] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:14.646683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:14.653186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:14.661549 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-05b2d620-fb24-48e6-b545-1d60f5c4995e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.275252 Oct 15 20:48:14.669382 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 27/86] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 3290 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:14.676565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.679034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.681648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.692243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:14.697098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:14.697962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:14.699037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d14e32f8-c133-4826-b887-8371ef05ab2e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:14.699695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d14e32f8-c133-4826-b887-8371ef05ab2e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:15.045276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:15.057591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c0a0883-bcb8-4823-8ba1-ed2f8a9f2e65 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.427679 Oct 15 20:48:15.071873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 60/87] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:14 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:15.332338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d14e32f8-c133-4826-b887-8371ef05ab2e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:15.346502 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d14e32f8-c133-4826-b887-8371ef05ab2e tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1219 microversion: 2.1 time: 0.665615 Oct 15 20:48:15.362788 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 28/88] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:14 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1219 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:15.470449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe7548d2c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:15.472609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.473260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.475061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.560708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.563978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.564446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.607621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.613791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.614718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.645944 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4929c42c-282d-4c79-ba4b-687d8da70aad tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.500866 Oct 15 20:48:15.648983 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 61/89] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:15 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:15.669598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe75492a4c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:15.671751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.672554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.673189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.699300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.700093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.700778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.721546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.722473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.723075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.731545 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57b25e88-c4e9-4faa-82cc-19eeedc71f9a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1504 microversion: 2.1 time: 0.072876 Oct 15 20:48:15.735492 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 62/90] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 20:48:15 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1504 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:15.758879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe75492a430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:15.774893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.775712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.777133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.876330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.876826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.883252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.955532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:15.955994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:15.956434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:15.987933 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2da393a-bddd-40b9-bfd9-c12567e8c93f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.239332 Oct 15 20:48:15.995713 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 63/91] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:15 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:16.011430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe754871820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:16.027989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.031552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.032336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:16.105553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.113890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.115207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:16.125062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-76a5dac3-37df-4854-819d-0890801b688f tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 758 microversion: 2.1 time: 0.127781 Oct 15 20:48:16.137869 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 64/92] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:15 2022] GET /compute/v2.1/os-hypervisors/1 => generated 758 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:16.163707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:16.164821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:16.183797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.190776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.191554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:16.193709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.197692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.199961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:16.643245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35890db5-046f-4be4-9c5d-a7a76b853e6f tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessTestJSON-flavor-2052232083", "id": 2019547973, "os-flavor-access:is_public": "False"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:16.686367 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35890db5-046f-4be4-9c5d-a7a76b853e6f tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 431 microversion: 2.1 time: 1.284063 Oct 15 20:48:16.692650 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 29/93] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:15 2022] POST /compute/v2.1/flavors => generated 431 bytes in 1291 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:16.707495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d5d31e4-09eb-49c6-aebb-7eedd582c058 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe754921f70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:16.709392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d5d31e4-09eb-49c6-aebb-7eedd582c058 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.729655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d5d31e4-09eb-49c6-aebb-7eedd582c058 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.730196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d5d31e4-09eb-49c6-aebb-7eedd582c058 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:16.796720 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d5d31e4-09eb-49c6-aebb-7eedd582c058 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/statistics" status: 200 len: 263 microversion: 2.1 time: 0.102851 Oct 15 20:48:16.806322 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 30/94] 10.222.0.16 () {58 vars in 1179 bytes} [Sat Oct 15 20:48:16 2022] GET /compute/v2.1/os-hypervisors/statistics => generated 263 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:16.834244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:16.837376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:16.863315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.864537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.865244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:16.867999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:16.868818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:16.869634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.139120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:17.149106 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9b5af2a3-9dde-420b-94ee-483ea0a6b361 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1223 microversion: 2.1 time: 0.339716 Oct 15 20:48:17.169063 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 31/95] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:16 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1223 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:17.208650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-01c85def-d3e7-437c-97a5-48c898de4559 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "788a7292d90d4a069e17d14bfcfcdb1d"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:17.229937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:17.254304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea9f9931-b30d-4ca0-8465-7798f1484568 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 1.114527 Oct 15 20:48:17.259646 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 65/96] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:16 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 1121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:17.267740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe7548869d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:17.278228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.279022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.279695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.326277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.327026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.327610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.356482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.357207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.360830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.374832 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e779cff-a70b-4b4a-b3ed-f05de6cb397a tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.113796 Oct 15 20:48:17.379972 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 66/97] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:17 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:17.395379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe754886a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:17.397493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.400501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.402480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.471514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.473260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.478241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.508493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10a659c8-3b08-4272-959d-c9ece43e4290 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 758 microversion: 2.1 time: 0.119957 Oct 15 20:48:17.515118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 67/98] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:17 2022] GET /compute/v2.1/os-hypervisors/1 => generated 758 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:17.531713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe754886160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:17.536589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.537625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.541369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.581578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.582445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.583109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.599566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8025f2e9-ced5-4634-b8c5-242e1741022c tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/2" status: 200 len: 758 microversion: 2.1 time: 0.071713 Oct 15 20:48:17.608728 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 68/99] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:17 2022] GET /compute/v2.1/os-hypervisors/2 => generated 758 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:17.636733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Calling method '.version_select at 0x7fe754886430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:17.644882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.655010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.658043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.703794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.718258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.723754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.772384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:17.773341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:17.774539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:17.807002 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-01c85def-d3e7-437c-97a5-48c898de4559 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2019547973/action" status: 200 len: 97 microversion: 2.1 time: 0.643777 Oct 15 20:48:17.815570 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 32/100] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:48:17 2022] POST /compute/v2.1/flavors/2019547973/action => generated 97 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:17.846324 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5700288f-ebcd-457e-aa64-da4c1fb535d7 tempest-HypervisorAdminTestJSON-356493465 tempest-HypervisorAdminTestJSON-356493465-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1/uptime" status: 200 len: 175 microversion: 2.1 time: 0.223999 Oct 15 20:48:17.854796 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 69/101] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Oct 15 20:48:17 2022] GET /compute/v2.1/os-hypervisors/1/uptime => generated 175 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:18.144080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-562abb99-b95d-4e5c-9033-139c08729d96 tempest-FlavorsAccessTestJSON-48009540 tempest-FlavorsAccessTestJSON-48009540-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:18.184510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:18.185641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:18.248417 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-562abb99-b95d-4e5c-9033-139c08729d96 tempest-FlavorsAccessTestJSON-48009540 tempest-FlavorsAccessTestJSON-48009540-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5412 microversion: 2.1 time: 0.430560 Oct 15 20:48:18.249310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:18.249820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:18.250271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:18.251140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:18.251673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:18.265884 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 33/102] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:17 2022] GET /compute/v2.1/flavors/detail => generated 5412 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:18.276597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:18.280552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6c4aa269-1669-4616-9af5-dc359c17902b tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "788a7292d90d4a069e17d14bfcfcdb1d"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:18.748997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:18.770630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b98c888d-5e62-4e38-8d81-e695f62d1783 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1277 microversion: 2.1 time: 0.594106 Oct 15 20:48:18.791115 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 70/103] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:18 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1277 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:18.798446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:18.799342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:18.821190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:18.821905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:18.822637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:18.823856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:18.824519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:18.825284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:18.873790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:19.407321 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6c4aa269-1669-4616-9af5-dc359c17902b tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2019547973/action" status: 200 len: 21 microversion: 2.1 time: 1.132886 Oct 15 20:48:19.415165 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 34/104] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:48:18 2022] POST /compute/v2.1/flavors/2019547973/action => generated 21 bytes in 1148 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:19.668046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:19.676941 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fda2da9d-2e4f-4436-9bba-04522c1be216 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.884421 Oct 15 20:48:19.693513 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 71/105] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:18 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 901 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:19.699577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d23e69e7-547f-45b8-bda3-a2fad8b7e068 tempest-FlavorsAccessTestJSON-48009540 tempest-FlavorsAccessTestJSON-48009540-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:19.723830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d23e69e7-547f-45b8-bda3-a2fad8b7e068 tempest-FlavorsAccessTestJSON-48009540 tempest-FlavorsAccessTestJSON-48009540-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.028096 Oct 15 20:48:19.725751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 72/106] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:19 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:19.752812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cfe92d2c-be25-43e0-9f10-1d288c2b95d3 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:19.790596 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cfe92d2c-be25-43e0-9f10-1d288c2b95d3 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2019547973" status: 202 len: 0 microversion: 2.1 time: 0.050196 Oct 15 20:48:19.791471 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 73/107] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:48:19 2022] DELETE /compute/v2.1/flavors/2019547973 => generated 0 bytes in 52 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:48:19.801152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:19.806175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:19.817498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:19.820783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:19.824838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:19.826024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:19.826650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:19.827263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:20.064034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:20.074830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9aaf9a54-28f2-4d11-a8cc-ca84900ad330 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1277 microversion: 2.1 time: 0.279388 Oct 15 20:48:20.089666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 74/108] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:19 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1277 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:20.106151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bede24f9-d89d-4a5d-8225-33e85d9ad84e tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:20.156227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bede24f9-d89d-4a5d-8225-33e85d9ad84e tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.064994 Oct 15 20:48:20.157099 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 75/109] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:20 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:20.184744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7bd55816-7b57-43d2-a98a-47950bfe660d tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessTestJSON-flavor-539686711", "id": 691377931, "os-flavor-access:is_public": "False"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:20.223457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7bd55816-7b57-43d2-a98a-47950bfe660d tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 427 microversion: 2.1 time: 0.036369 Oct 15 20:48:20.225020 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 76/110] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:20 2022] POST /compute/v2.1/flavors => generated 427 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:20.248043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10339e95-affe-41e2-b897-a3938b184ed4 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:20.263932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:20.288246 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10339e95-affe-41e2-b897-a3938b184ed4 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/691377931/os-flavor-access" status: 200 len: 21 microversion: 2.1 time: 0.053651 Oct 15 20:48:20.291601 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 77/111] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 20:48:20 2022] GET /compute/v2.1/flavors/691377931/os-flavor-access => generated 21 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:48:20.303889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed945ba4-bf07-4d1d-ac98-ddb9d8f50f45 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:20.360834 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed945ba4-bf07-4d1d-ac98-ddb9d8f50f45 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/691377931" status: 202 len: 0 microversion: 2.1 time: 0.065022 Oct 15 20:48:20.362248 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 78/112] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:48:20 2022] DELETE /compute/v2.1/flavors/691377931 => generated 0 bytes in 67 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:48:20.374213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ecf949c-5eb5-4faf-91a8-5a82b0279cb5 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:20.433797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ecf949c-5eb5-4faf-91a8-5a82b0279cb5 tempest-FlavorsAccessTestJSON-374384751 tempest-FlavorsAccessTestJSON-374384751-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.066168 Oct 15 20:48:20.436854 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 79/113] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:20 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:20.579198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb03efc4-d467-45ba-9635-c0d6041f4267 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:20.601780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb03efc4-d467-45ba-9635-c0d6041f4267 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 1.180585 Oct 15 20:48:20.602455 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 35/114] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:48:19 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 1186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:20.703116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:20.703754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:20.731825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:20.732344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:20.732769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:20.734609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:20.735581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:20.736396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:21.134036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:21.149132 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cc655091-78a9-4160-9100-aef9756cb736 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.450904 Oct 15 20:48:21.170885 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 36/115] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:20 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:21.181884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:21.182966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:21.192504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:21.193408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:21.194175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:21.195221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:21.195971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:21.196684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:21.518581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:21.533890 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8a71040c-4e30-4be0-9b4d-489305459191 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.360883 Oct 15 20:48:21.536900 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 37/116] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:21 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:21.650574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e2dcb490-b53d-4018-96fb-16108928b134 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "vcpus": 1, "disk": 1, "name": "tempest-flavor_with_ephemeral_1-30065082", "id": 1723496106, "OS-FLV-EXT-DATA:ephemeral": 1, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:21.691631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e2dcb490-b53d-4018-96fb-16108928b134 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 423 microversion: 2.1 time: 1.078532 Oct 15 20:48:21.694415 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 80/117] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:20 2022] POST /compute/v2.1/flavors => generated 423 bytes in 1082 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:21.706240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-439ffb7e-dcd8-4013-a38e-75ac81f40169 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:21.718206 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-439ffb7e-dcd8-4013-a38e-75ac81f40169 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.016957 Oct 15 20:48:21.724235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 38/118] 10.222.0.16 () {58 vars in 1182 bytes} [Sat Oct 15 20:48:21 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:21.735030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5216ba43-b3d3-419f-ae73-75574c2877cb tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "vcpus": 1, "disk": 1, "name": "tempest-flavor_with_ephemeral_0-580176546", "id": 668428536, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:21.768093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5216ba43-b3d3-419f-ae73-75574c2877cb tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 421 microversion: 2.1 time: 0.037395 Oct 15 20:48:21.773763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 81/119] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:21 2022] POST /compute/v2.1/flavors => generated 421 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:21.785814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ecf422b0-5d2a-428f-8d1c-29a37d2c5d6b tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:21.798215 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ecf422b0-5d2a-428f-8d1c-29a37d2c5d6b tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.016401 Oct 15 20:48:21.812567 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 39/120] 10.222.0.16 () {58 vars in 1182 bytes} [Sat Oct 15 20:48:21 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:21.824295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21bb97c1-1cdb-4a4e-8a36-bd3ccf1f80c8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Action: 'create', calling method: .version_select at 0x7fe7547f4670>, body: {"keypair": {"name": "tempest-keypair-1690905189", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL7lBVZFeQXwi/+iH/Maa+wuQ7rmAWHFOe5VtJjUbdvDZpwlTEVZlRnnRRGyE7FKzoFSSjXLZCBTs4zqlNc9GdlEZKzwCqqT43t6k37Cgm/pq+pSsQSfZHB2p87B7oNUoA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:21.904732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-21bb97c1-1cdb-4a4e-8a36-bd3ccf1f80c8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Getting quotas for project f856d315b3b84bdcac94b6d300ef8b2b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:21.923963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-21bb97c1-1cdb-4a4e-8a36-bd3ccf1f80c8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Getting quotas for user 14bd717b05ce4e40a1cd54c2db8474a6 and project f856d315b3b84bdcac94b6d300ef8b2b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:22.017358 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21bb97c1-1cdb-4a4e-8a36-bd3ccf1f80c8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.197003 Oct 15 20:48:22.027070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 82/121] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:48:21 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:22.187239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:22.188396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:22.208142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:22.211233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:22.212778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:22.215405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:22.216117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:22.217915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:22.537322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:22.548990 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-846822df-fc10-44cc-a3ad-bdb915c903b9 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.368234 Oct 15 20:48:22.572111 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 40/122] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:22 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:22.572284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:22.575513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:22.584099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:22.584827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:22.595852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:22.606688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:22.607973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:22.609669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:23.582024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:23.583209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:23.594813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:23.601456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:23.602200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:23.607695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:23.608463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:23.611254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:24.050187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:24.068669 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31b48efc-3672-4432-bc31-c8dbf32d354b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.492939 Oct 15 20:48:24.074119 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 41/123] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:23 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:24.517831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:24.527739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-821d88b4-e94e-4fd3-8c16-1e40d68b2109 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 1.969994 Oct 15 20:48:24.530378 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 83/124] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:22 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 1972 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:24.823384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:25.044409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:25.341617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=70,nova_cell0:UPDATE=14 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:25.610265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-201941575", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:25.663913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-567f64f6-1761-48c2-ba79-0fd17a4e742c tempest-InstanceUsageAuditLogTestJSON-1344408156 tempest-InstanceUsageAuditLogTestJSON-1344408156-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:25.762648 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-567f64f6-1761-48c2-ba79-0fd17a4e742c tempest-InstanceUsageAuditLogTestJSON-1344408156 tempest-InstanceUsageAuditLogTestJSON-1344408156-project-admin] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log/2022-10-15 20:48:25" status: 200 len: 318 microversion: 2.1 time: 0.725913 Oct 15 20:48:25.763289 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 84/125] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 20:48:25 2022] GET /compute/v2.1/os-instance_usage_audit_log/2022-10-15%2020%3A48%3A25 => generated 318 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:25.806054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:25.806899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:25.810127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:25.837448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:25.842370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:25.846216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:25.851482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:25.861879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:25.862714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:26.304765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:26.326607 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a4e20f9-eafb-43c2-8957-89433ebabf22 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.556816 Oct 15 20:48:26.328604 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 85/126] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:25 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:26.345820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:26.346830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:26.366471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:26.368847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:26.371213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:26.373985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:26.377232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:26.377821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:26.735143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:26.736032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:26.736555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:26.736930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:26.737389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:26.767973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:26.768666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:26.769142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:26.770225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:26.773672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:26.774554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:26.785380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=57 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:26.813987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Getting quotas for project 1f0b466076164a309233726db31b0efd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:26.825603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Getting quotas for user c84b11d2d54b43cbac87115e227ef50e and project 1f0b466076164a309233726db31b0efd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:26.850933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:26.859179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:26.861628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:26.863114 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5457d82-b37e-4c3a-89b5-76739661199d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.532053 Oct 15 20:48:26.866600 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 86/127] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:26 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:26.888465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f36e74b9-7b91-4c6b-a7d0-d69d10bcd0b6 tempest-InstanceUsageAuditLogTestJSON-1344408156 tempest-InstanceUsageAuditLogTestJSON-1344408156-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:26.909590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=8,nova_api:UPDATE=12,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:26.932071 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f36e74b9-7b91-4c6b-a7d0-d69d10bcd0b6 tempest-InstanceUsageAuditLogTestJSON-1344408156 tempest-InstanceUsageAuditLogTestJSON-1344408156-project-admin] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log" status: 200 len: 319 microversion: 2.1 time: 0.058575 Oct 15 20:48:26.948652 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 87/128] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 20:48:26 2022] GET /compute/v2.1/os-instance_usage_audit_log => generated 319 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:26.972076 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7a8b59eb-0608-4869-af62-a90298afe547 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.154291 Oct 15 20:48:26.975385 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 42/129] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 2158 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:27.324381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7b850691-98ca-4413-a12c-6baa20ff5da9 req-da4e9cd1-d6a7-44d9-a0cb-fdbaf1a32eaa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d6bfd383-07fb-4c4e-9d98-00a96ea7274d", "tag": "86dae641-1ad4-48b9-9898-d71d3c88a8c4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:27.338713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=90,nova_api:DELETE=6,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:27.357684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:27.358701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:27.373559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:27.377715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:27.378481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:27.379471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:27.380562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:27.382415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:27.389389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7b850691-98ca-4413-a12c-6baa20ff5da9 req-da4e9cd1-d6a7-44d9-a0cb-fdbaf1a32eaa service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:27.389860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7b850691-98ca-4413-a12c-6baa20ff5da9 req-da4e9cd1-d6a7-44d9-a0cb-fdbaf1a32eaa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:27.398649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7b850691-98ca-4413-a12c-6baa20ff5da9 req-da4e9cd1-d6a7-44d9-a0cb-fdbaf1a32eaa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:27.456253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7b850691-98ca-4413-a12c-6baa20ff5da9 req-da4e9cd1-d6a7-44d9-a0cb-fdbaf1a32eaa service nova] Creating event network-changed:86dae641-1ad4-48b9-9898-d71d3c88a8c4 for instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d on nch2-master-1 Oct 15 20:48:27.483768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7b850691-98ca-4413-a12c-6baa20ff5da9 req-da4e9cd1-d6a7-44d9-a0cb-fdbaf1a32eaa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.532089 Oct 15 20:48:27.485194 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 88/130] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:27.571978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-656362120", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:27.715326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:27.769862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:27.782325 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-50600945-3c1f-4f7c-84e3-dbbd69959bc0 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.428795 Oct 15 20:48:27.785160 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 43/131] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:27 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:27.846511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersWithSpecificFlavorTestJSON-server-1837146460", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "668428536", "adminPass": "***", "networks": [{"uuid": "44be626a-f6e0-4bbc-b8f7-82aacab3bfed"}], "security_groups": [{"name": "tempest-securitygroup--1012426555"}], "key_name": "tempest-keypair-1690905189", "user_data": "}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:28.313889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] validate_networks() for [('44be626a-f6e0-4bbc-b8f7-82aacab3bfed', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:29.030366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:29.031573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:29.032298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:29.032891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:29.033526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:29.050956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:29.052219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:29.055030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:29.056069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:29.056714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:29.058619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:29.123962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Getting quotas for project 1f0b466076164a309233726db31b0efd. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:29.139568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Getting quotas for user c84b11d2d54b43cbac87115e227ef50e and project 1f0b466076164a309233726db31b0efd. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:29.171166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:29.175747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:29.333476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d6062eb-5e4f-4a18-9b21-ac809973fea1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.754380 Oct 15 20:48:29.335167 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 89/132] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 1771 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:29.344598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:29.345490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:29.363556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:29.364334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:29.364967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:29.366468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:29.373062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:29.373701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:29.685128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:29.700101 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad1115eb-076d-45f3-851a-4d36ba6cd96c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.363545 Oct 15 20:48:29.700775 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 90/133] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:29 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:29.706517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:29.707138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:29.721510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:29.722066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:29.722525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:29.723274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:29.735622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:29.736253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:29.999789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:30.015942 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71bc645f-7a98-474f-8193-e7c715d03843 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.313674 Oct 15 20:48:30.016845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 91/134] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:29 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:30.047832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-226684f8-addb-4d08-900d-2db31df31387 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:30.049610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-226684f8-addb-4d08-900d-2db31df31387 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:30.227684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-226684f8-addb-4d08-900d-2db31df31387 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:30.261871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-226684f8-addb-4d08-900d-2db31df31387 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1213 microversion: 2.1 time: 0.243203 Oct 15 20:48:30.279042 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 92/135] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:30 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1213 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:30.325973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:30.326551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:30.326958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:30.327610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:30.328191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:30.351830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:30.352727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:30.353605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:30.354673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:30.355651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:30.356608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:30.378664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Getting quotas for project f856d315b3b84bdcac94b6d300ef8b2b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:30.387511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Getting quotas for user 14bd717b05ce4e40a1cd54c2db8474a6 and project f856d315b3b84bdcac94b6d300ef8b2b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:30.406365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:30.409143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:30.472373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8f73c38c-8f5f-4163-a7a8-d923c2fadcfe tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.633064 Oct 15 20:48:30.473205 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 44/136] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:27 2022] POST /compute/v2.1/servers => generated 402 bytes in 2635 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:30.581861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-af24dae5-c236-46cd-8a8a-0eeffd8bb0b7 req-172e1d9e-bb16-41f9-8c77-37e3ef78ecf6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d00bcb49-6cd6-4773-ac83-506cfc487514", "name": "network-vif-plugged", "status": "completed", "tag": "2f9db696-0c68-4064-bfff-ec8a1ea639d8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:30.607013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-af24dae5-c236-46cd-8a8a-0eeffd8bb0b7 req-172e1d9e-bb16-41f9-8c77-37e3ef78ecf6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:30.607671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-af24dae5-c236-46cd-8a8a-0eeffd8bb0b7 req-172e1d9e-bb16-41f9-8c77-37e3ef78ecf6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:30.608081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-af24dae5-c236-46cd-8a8a-0eeffd8bb0b7 req-172e1d9e-bb16-41f9-8c77-37e3ef78ecf6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:30.630259 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-af24dae5-c236-46cd-8a8a-0eeffd8bb0b7 req-172e1d9e-bb16-41f9-8c77-37e3ef78ecf6 service nova] Creating event network-vif-plugged:2f9db696-0c68-4064-bfff-ec8a1ea639d8 for instance d00bcb49-6cd6-4773-ac83-506cfc487514 on nch1-master-1 Oct 15 20:48:30.636217 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-af24dae5-c236-46cd-8a8a-0eeffd8bb0b7 req-172e1d9e-bb16-41f9-8c77-37e3ef78ecf6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.355405 Oct 15 20:48:30.637434 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 93/137] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:30.648897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a804c7a7-b08e-4b8f-a73c-ca3b2605f944 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:30.648897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a804c7a7-b08e-4b8f-a73c-ca3b2605f944 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:30.723265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a804c7a7-b08e-4b8f-a73c-ca3b2605f944 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:30.734191 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a804c7a7-b08e-4b8f-a73c-ca3b2605f944 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1263 microversion: 2.1 time: 0.094956 Oct 15 20:48:30.735250 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 94/138] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:30 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1263 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:30.741957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:30.743910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:30.757509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:30.760703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:30.761842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:30.762833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:30.763549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:30.764068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.024036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:31.034518 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ee67fdf-51a0-4623-9066-c671b22b94df tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.297677 Oct 15 20:48:31.039584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 95/139] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:30 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:31.045920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.046851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:31.058315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.059544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.060419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.061814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.063107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.064118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.212907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Calling method '.version_select at 0x7fe754889670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.214971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.215928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.216348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.246356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.247173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.247951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.273554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.275703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.276350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.288793 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d1aa1321-9b56-4531-908b-bca2df512711 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.812862 Oct 15 20:48:31.289965 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 45/140] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:30 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 816 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:31.295664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.296522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:31.303268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.304296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.304953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.306062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.306810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.307431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.399346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:31.417742 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ede1a8d8-694a-4f5b-88c9-2b142de74658 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1294 microversion: 2.1 time: 0.125289 Oct 15 20:48:31.432719 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 46/141] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1294 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:31.496566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:31.505190 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7096e7b1-0bb6-46de-a739-e60c341c13ec tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.464236 Oct 15 20:48:31.522901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 96/142] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:31.730201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-15117cd2-4fba-4e8e-8453-2793ddcf92d8 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Calling method '.version_select at 0x7fe754848940>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.732178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-15117cd2-4fba-4e8e-8453-2793ddcf92d8 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Policy check for os_compute_api:os-hypervisors:search failed with credentials {'is_admin': False, 'user_id': 'f9e0957b26c14313867167480a60d849', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3fcb4e2a06884be29a06f7910ff2635a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-15117cd2-4fba-4e8e-8453-2793ddcf92d8 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Request method failure captured: Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors/nch1-master-1/search HTTP/1.1 Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxybvFJ_yRpUqSQz_w-YKk4SMndZVm-Wsg0euBf5ymJM2FxIYYnGgTuUPWdZO03IXvjsrmQcnES4ypKs8CQM7bLhH-icgM1V9AkdzZxI8Qs4ppz0ecrdYjh8o2gkmlP8DkJ6Uw7fMLJUbNjcs3rJ9CsB-x7eEQzE5bt68eaRiAJJyd0 Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 3fcb4e2a06884be29a06f7910ff2635a Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-432492807 Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:31.736146 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-432492807 Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 3fcb4e2a06884be29a06f7910ff2635a Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-432492807 Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-432492807-project Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: f9e0957b26c14313867167480a60d849 Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-432492807-project Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Oct 15 20:48:31.737301 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'nch1-master-1'} Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 439, in search Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'search', target={}) Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:31.737911 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Oct 15 20:48:31.738870 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.739761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-15117cd2-4fba-4e8e-8453-2793ddcf92d8 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:31.739761 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-15117cd2-4fba-4e8e-8453-2793ddcf92d8 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nch1-master-1/search" status: 403 len: 117 microversion: 2.1 time: 0.315888 Oct 15 20:48:31.739761 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 47/143] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/search => generated 117 bytes in 318 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:31.749485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Calling method '.version_select at 0x7fe754777f70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.751606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.752231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.752708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.767503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.768291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:31.776098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] HTTP exception thrown: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Request method failure captured: Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search HTTP/1.1 Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxye1C913nZIi26d-nIkHsBr6xppZFNMiC3VujmJxNusAzZA1w4Y1DCuhhHWruaGB2TRNn1pvcXKugmejoW_R96znGMPRYJWI-FhyCBHi4uyTZMj8awt5ghf-0YAUJW8LH88vuvzfqd6FdSErMXOv_NXcMLbmhCPJlH1a6vGM6DaK8w Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 341f31852cf343caa3a4e6abf2b6a8c2 Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-1497673718 Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:31.777961 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-1497673718 Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 341f31852cf343caa3a4e6abf2b6a8c2 Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-1497673718 Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f10c851fb47541728f7b044fbcff158a Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: exception: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Oct 15 20:48:31.779067 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'nonexistent_hypervisor_name'} Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 443, in search Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi compute_nodes = self._get_compute_nodes_by_name_pattern(context, id) Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 139, in _get_compute_nodes_by_name_pattern Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.780095 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Returning 404 to user: No hypervisor matching 'nonexistent_hypervisor_name' could be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:31.781126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e8e544d5-f8c8-4728-adaf-5ff60008bce3 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search" status: 404 len: 114 microversion: 2.1 time: 0.033852 Oct 15 20:48:31.785555 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 97/144] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search => generated 114 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:31.785865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.786282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.786697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.787680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.788165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.788667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.795150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Calling method '.version_select at 0x7fe7546ed160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.798119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.799077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.799685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.820509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.821091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.821680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.834794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.835647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.836289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.846313 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3909df2-ac20-454c-b851-39acdb946b21 tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.059623 Oct 15 20:48:31.847356 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 98/145] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:31.873676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e136625e-590c-4e36-841e-d340ffbfbf97 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Calling method '.version_select at 0x7fe7547cb9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.875859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e136625e-590c-4e36-841e-d340ffbfbf97 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Policy check for os_compute_api:os-hypervisors:servers failed with credentials {'is_admin': False, 'user_id': 'f9e0957b26c14313867167480a60d849', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3fcb4e2a06884be29a06f7910ff2635a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e136625e-590c-4e36-841e-d340ffbfbf97 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Request method failure captured: Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-hypervisors/1/servers HTTP/1.1 Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxybvFJ_yRpUqSQz_w-YKk4SMndZVm-Wsg0euBf5ymJM2FxIYYnGgTuUPWdZO03IXvjsrmQcnES4ypKs8CQM7bLhH-icgM1V9AkdzZxI8Qs4ppz0ecrdYjh8o2gkmlP8DkJ6Uw7fMLJUbNjcs3rJ9CsB-x7eEQzE5bt68eaRiAJJyd0 Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 3fcb4e2a06884be29a06f7910ff2635a Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-432492807 Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:31.877785 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3fcb4e2a06884be29a06f7910ff2635a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-432492807 Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 3fcb4e2a06884be29a06f7910ff2635a Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-432492807 Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-432492807-project Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f9e0957b26c14313867167480a60d849 Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-432492807-project Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Oct 15 20:48:31.878855 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1'} Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 478, in servers Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'servers', target={}) Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:31.879446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Oct 15 20:48:31.880345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.882443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e136625e-590c-4e36-841e-d340ffbfbf97 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:31.882443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e136625e-590c-4e36-841e-d340ffbfbf97 tempest-HypervisorAdminNegativeUnderV252Test-432492807 tempest-HypervisorAdminNegativeUnderV252Test-432492807-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1/servers" status: 403 len: 118 microversion: 2.1 time: 0.018687 Oct 15 20:48:31.882443 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 99/146] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/os-hypervisors/1/servers => generated 118 bytes in 20 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:31.888587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:31.900257 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81d06089-0d1b-4045-b1ac-037721fb07d6 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1336 microversion: 2.1 time: 0.135966 Oct 15 20:48:31.901307 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 48/147] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1336 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:31.906300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Calling method '.version_select at 0x7fe754848790>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:31.908441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:31.908936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:31.909535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:31.933411 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] HTTP exception thrown: No hypervisor matching '609491c5-a974-40b3-b620-0b85bb80a8d3' could be found. Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Request method failure captured: Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors/609491c5-a974-40b3-b620-0b85bb80a8d3/servers HTTP/1.1 Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxye1C913nZIi26d-nIkHsBr6xppZFNMiC3VujmJxNusAzZA1w4Y1DCuhhHWruaGB2TRNn1pvcXKugmejoW_R96znGMPRYJWI-FhyCBHi4uyTZMj8awt5ghf-0YAUJW8LH88vuvzfqd6FdSErMXOv_NXcMLbmhCPJlH1a6vGM6DaK8w Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 341f31852cf343caa3a4e6abf2b6a8c2 Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-1497673718 Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:31.935220 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/341f31852cf343caa3a4e6abf2b6a8c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-1497673718 Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 341f31852cf343caa3a4e6abf2b6a8c2 Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-1497673718 Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: f10c851fb47541728f7b044fbcff158a Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: exception: No hypervisor matching '609491c5-a974-40b3-b620-0b85bb80a8d3' could be found. Oct 15 20:48:31.936578 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '609491c5-a974-40b3-b620-0b85bb80a8d3'} Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 482, in servers Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi compute_nodes = self._get_compute_nodes_by_name_pattern(context, id) Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 139, in _get_compute_nodes_by_name_pattern Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: No hypervisor matching '609491c5-a974-40b3-b620-0b85bb80a8d3' could be found. Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.937342 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:31.938582 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:31.938582 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:31.938582 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:31.938582 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:31.938582 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: No hypervisor matching '609491c5-a974-40b3-b620-0b85bb80a8d3' could be found. Oct 15 20:48:31.938582 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:31.941027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] Returning 404 to user: No hypervisor matching '609491c5-a974-40b3-b620-0b85bb80a8d3' could be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:31.942102 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aaa60d8d-dd8b-4ed8-9de9-234b44a49f3b tempest-HypervisorAdminNegativeUnderV252Test-1497673718 tempest-HypervisorAdminNegativeUnderV252Test-1497673718-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/609491c5-a974-40b3-b620-0b85bb80a8d3/servers" status: 404 len: 123 microversion: 2.1 time: 0.039772 Oct 15 20:48:31.947822 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 49/148] 10.222.0.16 () {58 vars in 1281 bytes} [Sat Oct 15 20:48:31 2022] GET /compute/v2.1/os-hypervisors/609491c5-a974-40b3-b620-0b85bb80a8d3/servers => generated 123 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:32.036561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0a9590a-de56-4bb3-843c-7630d6e57969 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_flavor-502887376", "ram": 512, "vcpus": 1, "disk": 10, "id": 296951257, "swap": 1024, "rxtx_factor": 1, "OS-FLV-EXT-DATA:ephemeral": 10}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.063784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.064390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:32.074596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.075149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.076251 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0a9590a-de56-4bb3-843c-7630d6e57969 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 412 microversion: 2.1 time: 0.193734 Oct 15 20:48:32.077142 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 100/149] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:31 2022] POST /compute/v2.1/flavors => generated 412 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:32.085229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.085229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.085229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.085229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.251993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40df09b7-3fe9-4fcf-9648-06ffa42e5c5d tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.265068 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-40df09b7-3fe9-4fcf-9648-06ffa42e5c5d tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] HTTP exception thrown: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40df09b7-3fe9-4fcf-9648-06ffa42e5c5d tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Request method failure captured: Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/flavors/296951257/os-extra_specs/hw:cpu_thread_policy HTTP/1.1 Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyeY74SX109cu6lEWiw5q5YE0B6lGH21IEQfl7z8AntKV2rCLeqWeaD3S_bcHx7YzynIJ_JnOgp6LExRf-f-_rUeCRNJH_q65_92shZtgIVDp7dgT-ebXlX35AZx0il1wueI_Kp_nS0VF8vPQ3OVCyP4ccECFe-RGWkyJS6a5_vP34 Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:32.267069 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3a5dc54f7703403fa7e19332d5ff81e8 Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:32.268498 ncdv-master-1 devstack@n-api.service[138688]: exception: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'flavor_id': '296951257', 'id': 'hw:cpu_thread_policy'} Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 115, in show Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return {id: flavor.extra_specs[id]} Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi KeyError: 'hw:cpu_thread_policy' Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 120, in show Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.269499 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.270765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.273857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40df09b7-3fe9-4fcf-9648-06ffa42e5c5d tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Returning 404 to user: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.276759 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40df09b7-3fe9-4fcf-9648-06ffa42e5c5d tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] 10.222.0.16 "GET /compute/v2.1/flavors/296951257/os-extra_specs/hw:cpu_thread_policy" status: 404 len: 112 microversion: 2.1 time: 0.189584 Oct 15 20:48:32.282193 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 101/150] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:48:32 2022] GET /compute/v2.1/flavors/296951257/os-extra_specs/hw:cpu_thread_policy => generated 112 bytes in 192 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:32.292961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3bf3c19-9d21-4f04-88f1-4ab6a021acd8 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.308897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3bf3c19-9d21-4f04-88f1-4ab6a021acd8 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Policy check for os_compute_api:os-flavor-extra-specs:create failed with credentials {'is_admin': False, 'user_id': '3a5dc54f7703403fa7e19332d5ff81e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '180fa0d40e2546448720f9c60a0d581f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3bf3c19-9d21-4f04-88f1-4ab6a021acd8 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Request method failure captured: Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/flavors/296951257/os-extra_specs HTTP/1.1 Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 66 Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyeY74SX109cu6lEWiw5q5YE0B6lGH21IEQfl7z8AntKV2rCLeqWeaD3S_bcHx7YzynIJ_JnOgp6LExRf-f-_rUeCRNJH_q65_92shZtgIVDp7dgT-ebXlX35AZx0il1wueI_Kp_nS0VF8vPQ3OVCyP4ccECFe-RGWkyJS6a5_vP34 Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:32.311704 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3a5dc54f7703403fa7e19332d5ff81e8 Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} Oct 15 20:48:32.312957 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'flavor_id': '296951257', 'body': {'extra_specs': {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'}}} Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 73, in create Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'create', target={}) Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:32.313593 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Oct 15 20:48:32.314497 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.315487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:32.317909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3bf3c19-9d21-4f04-88f1-4ab6a021acd8 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.319207 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3bf3c19-9d21-4f04-88f1-4ab6a021acd8 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] 10.222.0.16 "POST /compute/v2.1/flavors/296951257/os-extra_specs" status: 403 len: 124 microversion: 2.1 time: 0.033003 Oct 15 20:48:32.323268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 102/151] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Oct 15 20:48:32 2022] POST /compute/v2.1/flavors/296951257/os-extra_specs => generated 124 bytes in 37 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:32.328498 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aebe72eb-0619-4564-8567-d57da480977c tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.268608 Oct 15 20:48:32.329439 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 50/152] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:32 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:32.332473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9233c82-76a2-41f9-9cf2-ab72b22ee35f tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.377460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9233c82-76a2-41f9-9cf2-ab72b22ee35f tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/296951257/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.050266 Oct 15 20:48:32.378070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 103/153] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Oct 15 20:48:32 2022] POST /compute/v2.1/flavors/296951257/os-extra_specs => generated 66 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:32.391336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40a23958-e48e-4090-bfa9-13656e0bf665 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.392952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-40a23958-e48e-4090-bfa9-13656e0bf665 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Policy check for os_compute_api:os-flavor-extra-specs:delete failed with credentials {'is_admin': False, 'user_id': '3a5dc54f7703403fa7e19332d5ff81e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '180fa0d40e2546448720f9c60a0d581f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40a23958-e48e-4090-bfa9-13656e0bf665 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Request method failure captured: Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes HTTP/1.1 Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyeY74SX109cu6lEWiw5q5YE0B6lGH21IEQfl7z8AntKV2rCLeqWeaD3S_bcHx7YzynIJ_JnOgp6LExRf-f-_rUeCRNJH_q65_92shZtgIVDp7dgT-ebXlX35AZx0il1wueI_Kp_nS0VF8vPQ3OVCyP4ccECFe-RGWkyJS6a5_vP34 Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:32.402295 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 3a5dc54f7703403fa7e19332d5ff81e8 Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:32.404082 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'flavor_id': '296951257', 'id': 'hw:numa_nodes'} Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 129, in delete Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'delete', target={}) Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:32.404833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Oct 15 20:48:32.406417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.408309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40a23958-e48e-4090-bfa9-13656e0bf665 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.409388 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-40a23958-e48e-4090-bfa9-13656e0bf665 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] 10.222.0.16 "DELETE /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes" status: 403 len: 124 microversion: 2.1 time: 0.021571 Oct 15 20:48:32.410676 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 51/154] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 20:48:32 2022] DELETE /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes => generated 124 bytes in 24 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:32.421566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc2f4570-e45c-4bb8-8029-6e0daf7b3e73 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.449507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc2f4570-e45c-4bb8-8029-6e0daf7b3e73 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/296951257/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.032330 Oct 15 20:48:32.450242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 104/155] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Oct 15 20:48:32 2022] POST /compute/v2.1/flavors/296951257/os-extra_specs => generated 66 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:32.450607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.451331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:32.461909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa9bfdd4-a2cc-440e-a4a6-bca56c7dd0cb tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Action: 'update', calling method: >, body: {"hw:numa_nodes": "1"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.466291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa9bfdd4-a2cc-440e-a4a6-bca56c7dd0cb tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Policy check for os_compute_api:os-flavor-extra-specs:update failed with credentials {'is_admin': False, 'user_id': '3a5dc54f7703403fa7e19332d5ff81e8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '180fa0d40e2546448720f9c60a0d581f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa9bfdd4-a2cc-440e-a4a6-bca56c7dd0cb tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Request method failure captured: Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes HTTP/1.1 Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 22 Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyeY74SX109cu6lEWiw5q5YE0B6lGH21IEQfl7z8AntKV2rCLeqWeaD3S_bcHx7YzynIJ_JnOgp6LExRf-f-_rUeCRNJH_q65_92shZtgIVDp7dgT-ebXlX35AZx0il1wueI_Kp_nS0VF8vPQ3OVCyP4ccECFe-RGWkyJS6a5_vP34 Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:32.468377 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/180fa0d40e2546448720f9c60a0d581f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 180fa0d40e2546448720f9c60a0d581f Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3a5dc54f7703403fa7e19332d5ff81e8 Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: {"hw:numa_nodes": "1"} Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:32.469512 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'flavor_id': '296951257', 'id': 'hw:numa_nodes', 'body': {'hw:numa_nodes': '1'}} Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 91, in update Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'update', target={}) Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:32.470223 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Oct 15 20:48:32.471124 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.472033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.473228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.473343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.474006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.475284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.476849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa9bfdd4-a2cc-440e-a4a6-bca56c7dd0cb tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.479959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa9bfdd4-a2cc-440e-a4a6-bca56c7dd0cb tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380 tempest-FlavorsExtraSpecsNegativeTestJSON-1924161380-project] 10.222.0.16 "PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes" status: 403 len: 124 microversion: 2.1 time: 0.022954 Oct 15 20:48:32.481732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.486630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 105/156] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:48:32 2022] PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes => generated 124 bytes in 30 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:48:32.494445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13df76e4-b2f3-4d48-b743-0a4710c332ad tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.504183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-13df76e4-b2f3-4d48-b743-0a4710c332ad tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] HTTP exception thrown: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13df76e4-b2f3-4d48-b743-0a4710c332ad tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Request method failure captured: Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/flavors/296951257/os-extra_specs/hw:cpu_thread_policy HTTP/1.1 Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyfQXH_iUG94-caAFjwyGVYoIJxSE3M5b6x5LnvUeY02rsPNVVgu06eK6Otg87ZSfbA3N2T88bsJFV5BCp6_k7ef-QadM_2ylc0pxUlZeMpqvJS2EMKzDz68ucyiWS_yGZVrvHceX8Fkszm_cC-D1lbBD1PtDsYD0f1EKg8RAlq4RY Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: eb4fbfc6501a4ff4bb849f86cd3fbc04 Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:32.506450 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: eb4fbfc6501a4ff4bb849f86cd3fbc04 Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2c6909b25e674917abe5692130bb05fc Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:32.507840 ncdv-master-1 devstack@n-api.service[138688]: exception: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'flavor_id': '296951257', 'id': 'hw:cpu_thread_policy'} Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 132, in delete Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi del flavor.extra_specs[id] Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi KeyError: 'hw:cpu_thread_policy' Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 141, in delete Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.508483 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. Oct 15 20:48:32.509622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.512298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13df76e4-b2f3-4d48-b743-0a4710c332ad tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Returning 404 to user: Flavor 296951257 has no extra specs with key hw:cpu_thread_policy. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.515159 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13df76e4-b2f3-4d48-b743-0a4710c332ad tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/296951257/os-extra_specs/hw:cpu_thread_policy" status: 404 len: 112 microversion: 2.1 time: 0.026987 Oct 15 20:48:32.521127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 106/157] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 20:48:32 2022] DELETE /compute/v2.1/flavors/296951257/os-extra_specs/hw:cpu_thread_policy => generated 112 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:32.527221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71732acc-275f-4d38-9f63-ccc29638dffd tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Action: 'update', calling method: >, body: {"hw:cpu_policy": "shared"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.532117 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-71732acc-275f-4d38-9f63-ccc29638dffd tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] HTTP exception thrown: Request body and URI mismatch Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71732acc-275f-4d38-9f63-ccc29638dffd tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Request method failure captured: Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes HTTP/1.1 Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 27 Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyfQXH_iUG94-caAFjwyGVYoIJxSE3M5b6x5LnvUeY02rsPNVVgu06eK6Otg87ZSfbA3N2T88bsJFV5BCp6_k7ef-QadM_2ylc0pxUlZeMpqvJS2EMKzDz68ucyiWS_yGZVrvHceX8Fkszm_cC-D1lbBD1PtDsYD0f1EKg8RAlq4RY Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: eb4fbfc6501a4ff4bb849f86cd3fbc04 Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:32.534076 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: eb4fbfc6501a4ff4bb849f86cd3fbc04 Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2c6909b25e674917abe5692130bb05fc Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: {"hw:cpu_policy": "shared"} Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:32.535120 ncdv-master-1 devstack@n-api.service[138688]: exception: Request body and URI mismatch Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'flavor_id': '296951257', 'id': 'hw:numa_nodes', 'body': {'hw:cpu_policy': 'shared'}} Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 96, in update Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=expl) Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Request body and URI mismatch Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.535750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.536704 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:32.536704 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:32.536704 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Request body and URI mismatch Oct 15 20:48:32.536704 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.537892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71732acc-275f-4d38-9f63-ccc29638dffd tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Returning 400 to user: Request body and URI mismatch {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.539545 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71732acc-275f-4d38-9f63-ccc29638dffd tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes" status: 400 len: 73 microversion: 2.1 time: 0.016394 Oct 15 20:48:32.540324 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 107/158] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:48:32 2022] PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes => generated 73 bytes in 18 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:48:32.549971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.550787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:32.558096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.558799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.559426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.560310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.560924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.561530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.602302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:32.611152 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3554a2d4-64fe-4bf1-a6b2-a0c38a01367c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1292 microversion: 2.1 time: 0.168171 Oct 15 20:48:32.612586 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 52/159] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:32 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1292 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:32.633833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8344426f-f0cc-42b8-850f-ac0bd50db4ce tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Action: 'update', calling method: >, body: {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8344426f-f0cc-42b8-850f-ac0bd50db4ce tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Request method failure captured: Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes HTTP/1.1 Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 49 Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyfQXH_iUG94-caAFjwyGVYoIJxSE3M5b6x5LnvUeY02rsPNVVgu06eK6Otg87ZSfbA3N2T88bsJFV5BCp6_k7ef-QadM_2ylc0pxUlZeMpqvJS2EMKzDz68ucyiWS_yGZVrvHceX8Fkszm_cC-D1lbBD1PtDsYD0f1EKg8RAlq4RY Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: eb4fbfc6501a4ff4bb849f86cd3fbc04 Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:32.651306 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/eb4fbfc6501a4ff4bb849f86cd3fbc04"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: eb4fbfc6501a4ff4bb849f86cd3fbc04 Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 2c6909b25e674917abe5692130bb05fc Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"} Oct 15 20:48:32.652438 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: exception: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'flavor_id': '296951257', 'id': 'hw:numa_nodes', 'body': {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'}} Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'maxProperties' in schema: Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'maxProperties': 1, Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'minProperties': 1, Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'type': ['string', Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'number']}}, Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance: Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'hw:cpu_policy': 'shared', 'hw:numa_nodes': '1'} Oct 15 20:48:32.653076 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.654806 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8344426f-f0cc-42b8-850f-ac0bd50db4ce tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Returning 400 to user: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:32.655796 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8344426f-f0cc-42b8-850f-ac0bd50db4ce tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes" status: 400 len: 117 microversion: 2.1 time: 0.026076 Oct 15 20:48:32.665441 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 53/160] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:48:32 2022] PUT /compute/v2.1/flavors/296951257/os-extra_specs/hw:numa_nodes => generated 117 bytes in 52 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:48:32.670404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bc452be1-03ef-4f41-bc64-a8a31c1c8e6b tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.712963 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bc452be1-03ef-4f41-bc64-a8a31c1c8e6b tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/296951257" status: 202 len: 0 microversion: 2.1 time: 0.046288 Oct 15 20:48:32.717341 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 54/161] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:48:32 2022] DELETE /compute/v2.1/flavors/296951257 => generated 0 bytes in 51 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:48:32.723014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d14a38f-59e5-430e-bddb-d648ece487bb tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.749664 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d14a38f-59e5-430e-bddb-d648ece487bb tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609 tempest-FlavorsExtraSpecsNegativeTestJSON-1295190609-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5815 microversion: 2.1 time: 0.030862 Oct 15 20:48:32.765703 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 55/162] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:32 2022] GET /compute/v2.1/flavors/detail => generated 5815 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:32.928206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:32.929092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:32.939148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.941313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.941801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:32.944993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:32.946050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:32.946810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:33.137386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:33.147854 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17623565-bee0-4f3c-ad64-15dc077af4de tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1354 microversion: 2.1 time: 0.225652 Oct 15 20:48:33.149620 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 56/163] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:32 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1354 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:33.164498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:33.183353 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6c73678-e68d-476b-a0bf-ce10431b0191 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1337 microversion: 2.1 time: 0.639247 Oct 15 20:48:33.186776 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 108/164] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:32 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1337 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:33.355930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:33.356764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:33.371297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:33.374745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:33.376127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:33.377484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:33.379749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:33.380233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:33.640483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:33.641694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:33.671961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:33.675049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:33.678273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:33.680365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:33.682110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:33.683021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:33.867461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:33.876213 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-041f99ad-8e93-4211-817d-d4f0c3f24020 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.527024 Oct 15 20:48:33.879263 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 57/165] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:33 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:33.998886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:34.009789 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52ab95aa-ef39-4252-8344-afc32ee311cf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1334 microversion: 2.1 time: 0.373948 Oct 15 20:48:34.012182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 109/166] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:33 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1334 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:34.172055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:34.173056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:34.180758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.181446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.182033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.182842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.183405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.184042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.212409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:34.213578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:34.221462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.222233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.222763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.223675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.225286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.226303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.311194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:34.319883 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-90f21e33-26fc-44a6-9dab-af2bf1fa9e41 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1342 microversion: 2.1 time: 0.151906 Oct 15 20:48:34.321172 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 58/167] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:34 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1342 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:34.725808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:34.736429 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73e87525-8596-4522-98da-e27b9ba07816 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1487 microversion: 2.1 time: 0.526497 Oct 15 20:48:34.737073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 110/168] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:34 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1487 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:34.763997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:34.764616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:34.778987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.779619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.780071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.780790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.781288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.781717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.791044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Action: 'update', calling method: .version_select at 0x7fe754783dc0>, body: {"quota_set": {"ram": "5120"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:34.904433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.905553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.906133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.907024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:34.907816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:34.908417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:34.985065 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2eb1cc35-1d88-42d0-9630-fb373fb477b8 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104" status: 200 len: 148 microversion: 2.57 time: 0.424378 Oct 15 20:48:34.994024 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 59/169] 10.222.0.16 () {62 vars in 1303 bytes} [Sat Oct 15 20:48:34 2022] PUT /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104 => generated 148 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:35.000617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:35.000913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:35.009086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.010234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.010688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:35.011962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.013306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.014157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:35.357372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:35.370140 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70e21caa-ac21-43fd-826e-061a3d4917b6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.374943 Oct 15 20:48:35.370735 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 60/170] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:34 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:35.377274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Calling method '.version_select at 0x7fe75478d790>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:35.667759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.669013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.669762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:35.670723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.671892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.672699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:35.714835 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b1869059-d6a4-4980-b99f-426d72d83044 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104/detail" status: 200 len: 463 microversion: 2.57 time: 0.340434 Oct 15 20:48:35.715716 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 61/171] 10.222.0.16 () {60 vars in 1304 bytes} [Sat Oct 15 20:48:35 2022] GET /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104/detail => generated 463 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:35.725914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:35.728273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:35.745727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.748370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.749092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:35.751798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.753625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.756286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:35.944262 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf0d9c35-65c0-4ad0-a154-f7bbb2cbd3c4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1638 microversion: 2.1 time: 1.182851 Oct 15 20:48:35.946297 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 111/172] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:34 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1638 bytes in 1187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:35.959486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d7d932df-c6bb-4a85-abc0-2e9dc1edae9b req-14e28526-d837-4098-89cc-758f19a76b8a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "63f6e4c4-f541-4646-955c-0d10645390ee", "tag": "36b4370d-6280-4bfa-ae65-44586b1902e8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:35.989639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d7d932df-c6bb-4a85-abc0-2e9dc1edae9b req-14e28526-d837-4098-89cc-758f19a76b8a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:35.990169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d7d932df-c6bb-4a85-abc0-2e9dc1edae9b req-14e28526-d837-4098-89cc-758f19a76b8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:35.990612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d7d932df-c6bb-4a85-abc0-2e9dc1edae9b req-14e28526-d837-4098-89cc-758f19a76b8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.028932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d7d932df-c6bb-4a85-abc0-2e9dc1edae9b req-14e28526-d837-4098-89cc-758f19a76b8a service nova] Creating event network-changed:36b4370d-6280-4bfa-ae65-44586b1902e8 for instance 63f6e4c4-f541-4646-955c-0d10645390ee on nch1-master-1 Oct 15 20:48:36.046236 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d7d932df-c6bb-4a85-abc0-2e9dc1edae9b req-14e28526-d837-4098-89cc-758f19a76b8a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097060 Oct 15 20:48:36.055557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 112/173] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:36.064512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:36.066127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:36.076567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.078317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.079873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.087036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.088213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.090197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.158209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:36.168111 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c91478dc-6f72-47d0-9f3d-37358d41e54d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1334 microversion: 2.1 time: 0.451107 Oct 15 20:48:36.169814 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 62/174] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:35 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1334 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:36.176717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:36.179109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:36.195677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.196306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.196900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.197686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.199150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.199727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.239302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:48:36.384751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "compute-rpcapi-router" {{(pid=138689) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 20:48:36.385574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076fc6fd-f639-4c2c-9fb4-db56d15a29fa tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquired lock "compute-rpcapi-router" {{(pid=138689) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 20:48:36.387083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.388077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.390377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.392794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.394943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.396248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.417448 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.rpcapi [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 15 20:48:36.418360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] Releasing lock "compute-rpcapi-router" {{(pid=138689) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 20:48:36.469232 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e58f5b92-49c1-4db1-aec2-c1775d4e047c None None] 10.222.0.16 "DELETE /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 204 len: 0 microversion: 2.1 time: 0.295469 Oct 15 20:48:36.470730 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 63/175] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:48:36 2022] DELETE /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:48:36.477753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Action: 'update', calling method: .version_select at 0x7fe7547a5c10>, body: {"quota_set": {"ram": "2048"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:36.611812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:36.647069 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-372a21b9-0d00-4da3-8f7a-f155514d39ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1410 microversion: 2.1 time: 0.592386 Oct 15 20:48:36.648385 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 113/176] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:36 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1410 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:36.669644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-aa3dc7e6-cafa-4e25-8cfe-297ad052373e req-106b2835-2216-480d-a557-1d0bfa4fdff9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56f00c4c-5642-40b7-b0a7-62c3ec8b3b56", "tag": "1ed285bf-73a2-45c3-8c63-b59f9d0797cc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:36.687601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aa3dc7e6-cafa-4e25-8cfe-297ad052373e req-106b2835-2216-480d-a557-1d0bfa4fdff9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.688672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aa3dc7e6-cafa-4e25-8cfe-297ad052373e req-106b2835-2216-480d-a557-1d0bfa4fdff9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.689830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aa3dc7e6-cafa-4e25-8cfe-297ad052373e req-106b2835-2216-480d-a557-1d0bfa4fdff9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.705421 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-aa3dc7e6-cafa-4e25-8cfe-297ad052373e req-106b2835-2216-480d-a557-1d0bfa4fdff9 service nova] Creating event network-changed:1ed285bf-73a2-45c3-8c63-b59f9d0797cc for instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 on nch1-master-1 Oct 15 20:48:36.713532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-aa3dc7e6-cafa-4e25-8cfe-297ad052373e req-106b2835-2216-480d-a557-1d0bfa4fdff9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060639 Oct 15 20:48:36.713532 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 114/177] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:36.719772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:36.720559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:36.729291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.730050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.730658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.732057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:36.732687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:36.733315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:36.739300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:36.977574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:36.997501 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02b3e6bf-2d8a-4111-89af-2cffd361af1b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.281947 Oct 15 20:48:37.009532 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 115/178] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:36 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:37.014554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.015149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.015604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.016350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.020413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.020564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:37.020927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.021645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:37.050078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.050603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.051048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.051770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.052296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.052875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.151795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.152893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.153522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.154548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.155175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.155758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.229806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:INSERT=6,nova_api:SELECT=90,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:37.251414 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b3443eba-c953-499a-8b40-400e18a2b26d tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104?user_id=3505aa8d89534b48a9f50907e1a6bbe6" status: 200 len: 148 microversion: 2.57 time: 0.779253 Oct 15 20:48:37.252888 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 64/179] 10.222.0.16 () {62 vars in 1384 bytes} [Sat Oct 15 20:48:36 2022] PUT /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104?user_id=3505aa8d89534b48a9f50907e1a6bbe6 => generated 148 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:37.264694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:37.266438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:37.279891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.281177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.281915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.283259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.283731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.284382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.292462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:37.422024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:37.433368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-28ba302a-aaa3-452d-9817-6b0e51b780ce tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1493 microversion: 2.1 time: 0.422454 Oct 15 20:48:37.437571 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 116/180] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:37 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1493 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:37.450632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed12907c-c4c7-481c-a31d-a927fec62345 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] Calling method '.version_select at 0x7fe754619c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:37.572388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:37.585680 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7064453b-b86d-46fb-836c-36815d2b3dce tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1334 microversion: 2.1 time: 0.331121 Oct 15 20:48:37.586686 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 65/181] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:37 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1334 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:37.688247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:37.688880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:37.704587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.714566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.715027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.715787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:37.716377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:37.725603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:37.914036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed12907c-c4c7-481c-a31d-a927fec62345 tempest-QuotasAdminTestV257-925871111 tempest-QuotasAdminTestV257-925871111-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104?user_id=3505aa8d89534b48a9f50907e1a6bbe6" status: 200 len: 190 microversion: 2.57 time: 0.475023 Oct 15 20:48:37.915196 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 117/182] 10.222.0.16 () {60 vars in 1364 bytes} [Sat Oct 15 20:48:37 2022] GET /compute/v2.1/os-quota-sets/3ec5392df39b4786b3e8819ec8cf6104?user_id=3505aa8d89534b48a9f50907e1a6bbe6 => generated 190 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:37.930828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:37.945482 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d0567917-526c-4abe-a72f-469d2f689cc4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1410 microversion: 2.1 time: 0.261429 Oct 15 20:48:37.949551 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 66/183] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:37 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1410 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:38.028229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:38.030125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:38.042919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.043764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.044992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.046181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.046953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.047651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.321379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:38.331680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e8374da2-ed0e-4568-96ab-d5da1f35e883 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.310640 Oct 15 20:48:38.337225 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 118/184] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:38 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:38.462517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:38.463693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:38.469709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.470494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.471142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.472171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.472885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.473585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.557865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fd083ae7-fd08-4b4d-8d26-d6ba15d9c911 req-3de61019-ad46-4baa-b0de-5d98a561778b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d1ddd46-47f6-4948-a897-3b963780991e", "tag": "7d14c08a-67ff-4f01-a95d-0c19d04975e3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:38.579708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fd083ae7-fd08-4b4d-8d26-d6ba15d9c911 req-3de61019-ad46-4baa-b0de-5d98a561778b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.580546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fd083ae7-fd08-4b4d-8d26-d6ba15d9c911 req-3de61019-ad46-4baa-b0de-5d98a561778b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.581492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fd083ae7-fd08-4b4d-8d26-d6ba15d9c911 req-3de61019-ad46-4baa-b0de-5d98a561778b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.600919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fd083ae7-fd08-4b4d-8d26-d6ba15d9c911 req-3de61019-ad46-4baa-b0de-5d98a561778b service nova] Creating event network-changed:7d14c08a-67ff-4f01-a95d-0c19d04975e3 for instance 5d1ddd46-47f6-4948-a897-3b963780991e on nch2-master-1 Oct 15 20:48:38.606294 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fd083ae7-fd08-4b4d-8d26-d6ba15d9c911 req-3de61019-ad46-4baa-b0de-5d98a561778b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052305 Oct 15 20:48:38.607954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 119/185] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:38.613350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:38.614343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:38.631372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.633654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.634392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.635738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:38.637741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:38.638871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:38.769671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:38.783979 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d1f4479e-d9cf-4138-9b7f-2928f0f9dd8f tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1493 microversion: 2.1 time: 0.326003 Oct 15 20:48:38.790727 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 67/186] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:38 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1493 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:39.092887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:39.168366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:39.177658 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f5bf7cf-cd62-4a51-b2a6-168caeb660a2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1334 microversion: 2.1 time: 0.568230 Oct 15 20:48:39.178632 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 120/187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:38 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1334 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:39.183738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.184627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:39.190466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=6,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:39.192346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.193236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.193890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.194832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.195628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.196253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.286574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-484b69bf-0996-4114-b279-c207acb08c95 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Action: 'create', calling method: .version_select at 0x7fe754620700>, body: {"keypair": {"name": "tempest-keypair-1188790566", "type": "ssh", "user_id": "2d907d81af2a4c5486fc464320bb84e4", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP+eyhCMVr4EMfzt6xNyTpDdDfLRdCgleSqYL1+1swWdsZM57KQnbSI1naUHsf+olHd2UrPuJfW5cdiBLR+2vjzlCR8A9SLbbDpcvUsLxAA7TRCIra7UipnFPkjJAN2oaw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:39.315407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-484b69bf-0996-4114-b279-c207acb08c95 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Getting quotas for project 0325b17a072841fbb0c06c45040e912e. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:39.330069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-484b69bf-0996-4114-b279-c207acb08c95 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Getting quotas for user 2fe64fcd24244811b4e6506f1b212bff and project 0325b17a072841fbb0c06c45040e912e. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:39.369263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-484b69bf-0996-4114-b279-c207acb08c95 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.10 time: 0.403127 Oct 15 20:48:39.377477 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 68/188] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 20:48:38 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 412 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 20:48:39.461205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe754889670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.466286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.466840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.467425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.468329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.469476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.470004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.503172 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81d2dc0e-87df-433d-a5cc-d70b53e90f6a tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 400 microversion: 2.1 time: 0.124154 Oct 15 20:48:39.504458 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 69/189] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/os-hosts => generated 400 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:39.513136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.513986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:39.522087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.522954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.523545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.524744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.525767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.526354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.607175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:39.615739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-94f70632-c3f4-456b-9545-b0b9d9919a89 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1410 microversion: 2.1 time: 0.435879 Oct 15 20:48:39.616588 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 121/190] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1410 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:39.633429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3755ad93-d5a5-4c39-bce8-1af8314e01d4 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Action: 'create', calling method: .version_select at 0x7fe7546ed430>, body: {"keypair": {"name": "tempest-keypair-110465310", "type": "ssh", "user_id": "2d907d81af2a4c5486fc464320bb84e4", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBE1bLPzhGTJxhw4sxs033paFw0M/IyoRYl5z7GJsO72Kk48TF2nZtvXiNg50E0i996n0s18eQeshzno34851L5ydQaqcWhelgwYyx9dodVtBraauAEwGVwYkmcC+MP+pQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:39.653541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3755ad93-d5a5-4c39-bce8-1af8314e01d4 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Getting quotas for project 0325b17a072841fbb0c06c45040e912e. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:39.660035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3755ad93-d5a5-4c39-bce8-1af8314e01d4 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Getting quotas for user 2fe64fcd24244811b4e6506f1b212bff and project 0325b17a072841fbb0c06c45040e912e. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:39.687161 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3755ad93-d5a5-4c39-bce8-1af8314e01d4 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.10 time: 0.069378 Oct 15 20:48:39.688011 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 122/191] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 20:48:39 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 71 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 20:48:39.699808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-83e65ce0-f275-4477-9aea-b037c082e93b req-01b149b0-5b56-405d-af8e-a95d8ca505bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d6bfd383-07fb-4c4e-9d98-00a96ea7274d", "name": "network-vif-plugged", "status": "completed", "tag": "86dae641-1ad4-48b9-9898-d71d3c88a8c4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:39.711360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-83e65ce0-f275-4477-9aea-b037c082e93b req-01b149b0-5b56-405d-af8e-a95d8ca505bd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.712032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-83e65ce0-f275-4477-9aea-b037c082e93b req-01b149b0-5b56-405d-af8e-a95d8ca505bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.712772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-83e65ce0-f275-4477-9aea-b037c082e93b req-01b149b0-5b56-405d-af8e-a95d8ca505bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.726768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-83e65ce0-f275-4477-9aea-b037c082e93b req-01b149b0-5b56-405d-af8e-a95d8ca505bd service nova] Creating event network-vif-plugged:86dae641-1ad4-48b9-9898-d71d3c88a8c4 for instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d on nch2-master-1 Oct 15 20:48:39.732908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-83e65ce0-f275-4477-9aea-b037c082e93b req-01b149b0-5b56-405d-af8e-a95d8ca505bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036951 Oct 15 20:48:39.734465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 123/192] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:39.741848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe7546d5a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.749813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.750615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.751294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.752790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.753516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.754162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.781139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:39.790632 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5bedee7-682e-4a81-ad8e-a1190a920edb tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts?zone=" status: 200 len: 400 microversion: 2.1 time: 0.054447 Oct 15 20:48:39.791495 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 124/193] 10.222.0.16 () {58 vars in 1139 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/os-hosts?zone= => generated 400 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:39.797331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2ea5398-b065-4a01-a322-c4c7e71b6ca5 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Calling method '.version_select at 0x7fe7546ed160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.799316 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-34bf9e62-3b37-477f-869e-14aadc449b8d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.293136 Oct 15 20:48:39.799910 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 70/194] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:39.804923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe754620ee0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.807239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2ea5398-b065-4a01-a322-c4c7e71b6ca5 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-1188790566?user_id=2d907d81af2a4c5486fc464320bb84e4" status: 200 len: 512 microversion: 2.10 time: 0.014686 Oct 15 20:48:39.808075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 125/195] 10.222.0.16 () {60 vars in 1340 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1188790566?user_id=2d907d81af2a4c5486fc464320bb84e4 => generated 512 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:39.817017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.817017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.817017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.817017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.821230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.821976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.822398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.823225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:39.832074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.832882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.834524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.835360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:39.835904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:39.836412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:39.848758 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4992a84e-3351-4e60-9ca1-1093935225ef tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts?zone=xxx" status: 200 len: 13 microversion: 2.1 time: 0.047507 Oct 15 20:48:39.850814 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 71/196] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/os-hosts?zone=xxx => generated 13 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:39.861111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa4877ea-1608-46fb-bcd8-234c25c1e75b tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Action: 'create', calling method: .version_select at 0x7fe754631670>, body: {"keypair": {"name": "tempest-KeyPairsV210TestJSON-keypair-1251083499", "type": "ssh", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFENRuaFUebZ1nLx2PS/LZGMtmaxGB0SC6pTSKOzs5AFe/91KzWVVb780ONHaUvvN0gOpDzZIj4BOQ7QSfPEq/iL30ra+yMOEv8/DFXzYjofazHJE81ZZc5UsizYqUSEhw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:39.883849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-aa4877ea-1608-46fb-bcd8-234c25c1e75b tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Getting quotas for project 0325b17a072841fbb0c06c45040e912e. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:39.900915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-aa4877ea-1608-46fb-bcd8-234c25c1e75b tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Getting quotas for user 2fe64fcd24244811b4e6506f1b212bff and project 0325b17a072841fbb0c06c45040e912e. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:39.934499 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa4877ea-1608-46fb-bcd8-234c25c1e75b tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 422 microversion: 2.10 time: 0.082271 Oct 15 20:48:39.936070 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 72/197] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 20:48:39 2022] POST /compute/v2.1/os-keypairs => generated 422 bytes in 84 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 20:48:39.953702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-321cc727-90bb-4033-b623-7599973c7de1 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Calling method '.version_select at 0x7fe754616040>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:39.966964 ncdv-master-1 devstack@n-api.service[138689]: WARNING oslo_db.sqlalchemy.utils [None req-321cc727-90bb-4033-b623-7599973c7de1 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Unique keys not in sort_keys. The sorting order may be unstable. Oct 15 20:48:39.975463 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-321cc727-90bb-4033-b623-7599973c7de1 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "GET /compute/v2.1/os-keypairs?user_id=2d907d81af2a4c5486fc464320bb84e4" status: 200 len: 725 microversion: 2.10 time: 0.032791 Oct 15 20:48:39.976083 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 73/198] 10.222.0.16 () {60 vars in 1259 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/os-keypairs?user_id=2d907d81af2a4c5486fc464320bb84e4 => generated 725 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:39.990718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2f3fa86b-2284-4a67-a392-b55e4e34973e tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Calling method '.version_select at 0x7fe7547a5e50>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:40.018843 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2f3fa86b-2284-4a67-a392-b55e4e34973e tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV210TestJSON-keypair-1251083499" status: 204 len: 0 microversion: 2.10 time: 0.034578 Oct 15 20:48:40.022838 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 74/199] 10.222.0.16 () {60 vars in 1325 bytes} [Sat Oct 15 20:48:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV210TestJSON-keypair-1251083499 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:48:40.028699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2ff14c03-1c23-4f6c-8c11-b5da3f773835 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Calling method '.version_select at 0x7fe75478db80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:40.059971 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2ff14c03-1c23-4f6c-8c11-b5da3f773835 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-110465310?user_id=2d907d81af2a4c5486fc464320bb84e4" status: 204 len: 0 microversion: 2.10 time: 0.035545 Oct 15 20:48:40.064471 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 75/200] 10.222.0.16 () {60 vars in 1340 bytes} [Sat Oct 15 20:48:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-110465310?user_id=2d907d81af2a4c5486fc464320bb84e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:48:40.072026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f2b35eb-e5be-4e7c-9af0-cb53d4b09e92 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] Calling method '.version_select at 0x7fe754631430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:40.103908 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f2b35eb-e5be-4e7c-9af0-cb53d4b09e92 tempest-KeyPairsV210TestJSON-318947502 tempest-KeyPairsV210TestJSON-318947502-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1188790566?user_id=2d907d81af2a4c5486fc464320bb84e4" status: 204 len: 0 microversion: 2.10 time: 0.037829 Oct 15 20:48:40.105217 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 76/201] 10.222.0.16 () {60 vars in 1343 bytes} [Sat Oct 15 20:48:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1188790566?user_id=2d907d81af2a4c5486fc464320bb84e4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 15 20:48:40.208375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:40.210491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:40.226909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:40.227782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:40.229101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:40.230948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:40.231927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:40.242790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:40.529489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:40.543926 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-643e5f7e-cb4e-4c6a-a7d0-b2aa1570e270 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1493 microversion: 2.1 time: 0.725390 Oct 15 20:48:40.560152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 126/202] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:39 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1493 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:40.643246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:40.644179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:40.654865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:40.656359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:40.658562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:40.659518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:40.660148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:40.664477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:40.793542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:40.804851 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-39253311-28fe-4d4b-8547-71a1272ed263 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1334 microversion: 2.1 time: 0.603384 Oct 15 20:48:40.807137 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 77/203] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:40 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1334 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:40.834165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:40.837398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:40.846389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:40.847242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:40.847882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:40.849426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:40.850116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:40.850788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:41.034929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:41.047252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fa3b95a2-e2cf-4d22-8801-5f25489742e5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1410 microversion: 2.1 time: 0.406532 Oct 15 20:48:41.048389 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 127/204] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:40 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1410 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:41.060347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f0a9f530-6cb8-4572-8a1a-57b3f3b141dd req-3e53b97c-d28f-4a22-b464-fb1e53fface2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d00bcb49-6cd6-4773-ac83-506cfc487514", "name": "network-vif-deleted", "tag": "2f9db696-0c68-4064-bfff-ec8a1ea639d8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:41.080547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f0a9f530-6cb8-4572-8a1a-57b3f3b141dd req-3e53b97c-d28f-4a22-b464-fb1e53fface2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:41.083425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f0a9f530-6cb8-4572-8a1a-57b3f3b141dd req-3e53b97c-d28f-4a22-b464-fb1e53fface2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:41.084089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f0a9f530-6cb8-4572-8a1a-57b3f3b141dd req-3e53b97c-d28f-4a22-b464-fb1e53fface2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:41.120258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f0a9f530-6cb8-4572-8a1a-57b3f3b141dd req-3e53b97c-d28f-4a22-b464-fb1e53fface2 service nova] Creating event network-vif-deleted:2f9db696-0c68-4064-bfff-ec8a1ea639d8 for instance d00bcb49-6cd6-4773-ac83-506cfc487514 on nch1-master-1 Oct 15 20:48:41.127075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f0a9f530-6cb8-4572-8a1a-57b3f3b141dd req-3e53b97c-d28f-4a22-b464-fb1e53fface2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075624 Oct 15 20:48:41.127693 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 128/205] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:41.205444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:41.216449 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8931e22d-3e96-4c91-9b7f-1f54cf461517 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.390323 Oct 15 20:48:41.218308 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 78/206] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:40 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:41.568993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:41.569727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:41.581503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:41.581996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:41.582699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:41.585425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:41.587803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:41.588615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:41.629474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:41.630829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:41.631308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:41.631746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lazy-loading 'fault' on Instance uuid d00bcb49-6cd6-4773-ac83-506cfc487514 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:48:41.690640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b16bd0809f3411a91fae6c5e2f3acee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '474c2d620e224f06b409adf776004cac', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:41.698557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30bae0c4-7c67-42aa-a2ea-9b3b0f9b72dd tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 200 len: 1327 microversion: 2.1 time: 0.134891 Oct 15 20:48:41.699120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 129/207] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:41 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 1327 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:41.832319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:41.833705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:41.840251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:41.840744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:41.841237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:41.841964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:41.842406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:41.843122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.023027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:42.044312 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cd2ea86a-f2fb-4691-8daf-fea4f75884fa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1334 microversion: 2.1 time: 0.216476 Oct 15 20:48:42.049091 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 79/208] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:41 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1334 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:42.075402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:42.076324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:42.085309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.086228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.086943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.087865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.088520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.089180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.245490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:42.246425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:42.251443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:42.259016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.260120 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69eb6c93-1654-44ed-96dc-75d0de15bdb8 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1410 microversion: 2.1 time: 0.192284 Oct 15 20:48:42.261203 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 130/209] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:42 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1410 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:42.261654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.262308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.263105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.264136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.265487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.475483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:42.486532 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-10749538-4ec1-40b9-8bdf-729145fa1ef6 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.246714 Oct 15 20:48:42.506318 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 80/210] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:42 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:42.722483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:42.723105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:42.736312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.736826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.737319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.738603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.739129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.740996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.770054 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] HTTP exception thrown: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Request method failure captured: Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 HTTP/1.1 Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx6n7Ar_VhiRPEzhw3IQFnKIzWN2MkvMJ178i3KGZHfD0lS4v8sDoo6yfpaiQwrFquIb-7Lr-XVGz7rxx8oGZT5N1Abp4Z5SN_dxfXbWmYeUjBWAkZ63qdFZRberO0tsstTVpnejoYlmarA6ku-dZTaLbVoJjI66wQA_C7XYU3qkh0 Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 474c2d620e224f06b409adf776004cac Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:42.776224 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/474c2d620e224f06b409adf776004cac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_474c2d620e224f06b409adf776004cac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/474c2d620e224f06b409adf776004cac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/474c2d620e224f06b409adf776004cac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 474c2d620e224f06b409adf776004cac Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminTestJSON-2070443666-project Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4b16bd0809f3411a91fae6c5e2f3acee Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminTestJSON-2070443666-project Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:42.777462 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd00bcb49-6cd6-4773-ac83-506cfc487514'} Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:42.778201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:42.779259 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:42.781835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Returning 404 to user: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:42.782969 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0119f9f-9e70-413f-ab27-b027c5cd272b tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 404 len: 111 microversion: 2.1 time: 0.064582 Oct 15 20:48:42.784260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 131/211] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:42 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:42.793758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b4a87fab-c8f7-45bf-b9ec-f26d578bbe5b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:42.805614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4a87fab-c8f7-45bf-b9ec-f26d578bbe5b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.806225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4a87fab-c8f7-45bf-b9ec-f26d578bbe5b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.806787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4a87fab-c8f7-45bf-b9ec-f26d578bbe5b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.904787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8bfd31dc-3130-4095-9ef8-d02daa4c29d2 req-9943d114-4c4f-4164-ac48-3d8d222ac4bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "63f6e4c4-f541-4646-955c-0d10645390ee", "name": "network-vif-plugged", "status": "completed", "tag": "36b4370d-6280-4bfa-ae65-44586b1902e8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:42.925718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8bfd31dc-3130-4095-9ef8-d02daa4c29d2 req-9943d114-4c4f-4164-ac48-3d8d222ac4bd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:42.926222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8bfd31dc-3130-4095-9ef8-d02daa4c29d2 req-9943d114-4c4f-4164-ac48-3d8d222ac4bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:42.926658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8bfd31dc-3130-4095-9ef8-d02daa4c29d2 req-9943d114-4c4f-4164-ac48-3d8d222ac4bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:42.953378 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8bfd31dc-3130-4095-9ef8-d02daa4c29d2 req-9943d114-4c4f-4164-ac48-3d8d222ac4bd service nova] Creating event network-vif-plugged:36b4370d-6280-4bfa-ae65-44586b1902e8 for instance 63f6e4c4-f541-4646-955c-0d10645390ee on nch1-master-1 Oct 15 20:48:42.981610 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8bfd31dc-3130-4095-9ef8-d02daa4c29d2 req-9943d114-4c4f-4164-ac48-3d8d222ac4bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080766 Oct 15 20:48:42.982265 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 132/212] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:43.001746 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b4a87fab-c8f7-45bf-b9ec-f26d578bbe5b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/2/action" status: 200 len: 297 microversion: 2.1 time: 0.212400 Oct 15 20:48:43.015879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37630d4f-156d-42cd-a6a1-bc27b4120820 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.018724 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 81/213] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:42 2022] POST /compute/v2.1/os-aggregates/2/action => generated 297 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:43.044836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37630d4f-156d-42cd-a6a1-bc27b4120820 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/2" status: 200 len: 0 microversion: 2.1 time: 0.033046 Oct 15 20:48:43.050354 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 133/214] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:43 2022] DELETE /compute/v2.1/os-aggregates/2 => generated 0 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:43.062637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3321e536-1c97-40ee-9046-43fc1ebbbf54 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1471100572"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:43.068002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-3321e536-1c97-40ee-9046-43fc1ebbbf54 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 90de10b0-711d-49ca-ae20-9ee6d2ef83e0 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:43.078273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.079427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:43.096404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.096918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.099586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.101571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.102575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.103787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.109097 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3321e536-1c97-40ee-9046-43fc1ebbbf54 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.050128 Oct 15 20:48:43.118214 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 82/215] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:43 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:43.123714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-11f6a879-7684-434f-8911-61692495fa05 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:43.139053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11f6a879-7684-434f-8911-61692495fa05 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.139738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11f6a879-7684-434f-8911-61692495fa05 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.140730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11f6a879-7684-434f-8911-61692495fa05 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.332507 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-11f6a879-7684-434f-8911-61692495fa05 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/3/action" status: 200 len: 242 microversion: 2.1 time: 0.212824 Oct 15 20:48:43.333648 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 83/216] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:43 2022] POST /compute/v2.1/os-aggregates/3/action => generated 242 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:43.340003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.343170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:43.359331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.360157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.360814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.361610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.362986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.365054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.547804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:43.559629 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36c11726-6cd3-4f05-ab6e-8ceabece47a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1313 microversion: 2.1 time: 0.484495 Oct 15 20:48:43.561429 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 134/217] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1313 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:43.588583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb3eb56b-bbc0-4d39-a16a-efd3b83fa10b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.588854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:43.596835 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87c8197c-04ed-4cca-ae26-8346ff9bae72 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1389 microversion: 2.1 time: 0.261761 Oct 15 20:48:43.607201 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb3eb56b-bbc0-4d39-a16a-efd3b83fa10b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/3" status: 200 len: 242 microversion: 2.1 time: 0.044419 Oct 15 20:48:43.607939 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 135/218] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/os-aggregates/3 => generated 242 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:43.613516 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 84/219] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1389 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:43.620631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.621483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:43.628981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.629631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.630372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.631100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.631546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.632008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.805062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:43.814109 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-647cc10f-84ef-4dd0-9ce4-5a04f44e90c2 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1319 microversion: 2.1 time: 0.199375 Oct 15 20:48:43.814676 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 85/220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1319 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:43.826317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e66c01b9-f532-49f1-8988-9f6c67fbd6c0 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:43.839284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.843352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.843863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.844290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.845017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.845486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.845905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.854082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e66c01b9-f532-49f1-8988-9f6c67fbd6c0 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.854594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e66c01b9-f532-49f1-8988-9f6c67fbd6c0 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.855165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e66c01b9-f532-49f1-8988-9f6c67fbd6c0 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.874806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fdb5a69b-5bfa-4710-b113-49374f632db3 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.265586 Oct 15 20:48:43.875390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 136/221] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/os-services => generated 947 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:43.887749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.891424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.891911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.892346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.893088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.893784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.894210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.918466 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f1178c7-6e01-464d-91b7-da56ba5f9f48 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.034123 Oct 15 20:48:43.919061 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 137/222] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/os-services?xxx=nova-compute => generated 947 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:43.933123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.936545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.937059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.937539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.938280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.938742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.939277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.966782 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c0ed036-9f7a-429d-9118-04b166717377 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.037189 Oct 15 20:48:43.967710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 138/223] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/os-services => generated 947 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:43.980981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:43.984649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.985173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.985614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.986347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:43.986807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:43.987239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:43.997717 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e66c01b9-f532-49f1-8988-9f6c67fbd6c0 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/3/action" status: 200 len: 227 microversion: 2.1 time: 0.175166 Oct 15 20:48:43.998802 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 86/224] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:43 2022] POST /compute/v2.1/os-aggregates/3/action => generated 227 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:44.009938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f12a9eba-79ad-48ba-9022-f5e2391f5880 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.015650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=4,nova_api:DELETE=5 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:44.022151 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8126fc1e-14cc-40c8-ab2c-0919ad7c152a tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx" status: 200 len: 16 microversion: 2.1 time: 0.042458 Oct 15 20:48:44.022151 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 139/225] 10.222.0.16 () {58 vars in 1196 bytes} [Sat Oct 15 20:48:43 2022] GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx => generated 16 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:48:44.032669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.036993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.037573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.038025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.038869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.039460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.039943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.047221 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f12a9eba-79ad-48ba-9022-f5e2391f5880 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/3" status: 200 len: 0 microversion: 2.1 time: 0.040304 Oct 15 20:48:44.049214 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 87/226] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:44 2022] DELETE /compute/v2.1/os-aggregates/3 => generated 0 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Oct 15 20:48:44.064168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b7e7bd2c-a993-4eea-b9f1-4cd64afe26ca tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1064010526"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:44.071217 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b967162d-0902-4eb0-8516-90c8ae63d096 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.042101 Oct 15 20:48:44.072146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 140/227] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/os-services => generated 947 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:44.081148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-b7e7bd2c-a993-4eea-b9f1-4cd64afe26ca tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 110f5733-d26a-4669-831a-e1a8369538f0 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:44.088586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.097069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.097720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.099828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.101471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.102080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.102607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.129914 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b7e7bd2c-a993-4eea-b9f1-4cd64afe26ca tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.072723 Oct 15 20:48:44.131788 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 88/228] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:44 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:44.153007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-43cdec82-712c-47b9-86e6-9b5b5ea90d00 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:44.173665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3be73aaf-6fac-4ebe-8e18-3c8bbb4eacb8 tempest-ServicesAdminNegativeTestJSON-1723721346 tempest-ServicesAdminNegativeTestJSON-1723721346-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler" status: 200 len: 16 microversion: 2.1 time: 0.092888 Oct 15 20:48:44.177558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 141/229] 10.222.0.16 () {58 vars in 1198 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler => generated 16 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:44.183788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43cdec82-712c-47b9-86e6-9b5b5ea90d00 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.184335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43cdec82-712c-47b9-86e6-9b5b5ea90d00 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.184783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43cdec82-712c-47b9-86e6-9b5b5ea90d00 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.350401 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-43cdec82-712c-47b9-86e6-9b5b5ea90d00 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/4/action" status: 200 len: 242 microversion: 2.1 time: 0.208801 Oct 15 20:48:44.351535 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 89/230] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:44 2022] POST /compute/v2.1/os-aggregates/4/action => generated 242 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:44.363704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e26a38e7-f210-46a0-af4c-9daf7ec671ac tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.379498 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e26a38e7-f210-46a0-af4c-9daf7ec671ac tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 245 microversion: 2.1 time: 0.020106 Oct 15 20:48:44.381284 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 90/231] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/os-aggregates => generated 245 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:44.393139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8fdbf423-654e-49a6-b544-056715ec3048 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:44.404279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fdbf423-654e-49a6-b544-056715ec3048 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.405025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fdbf423-654e-49a6-b544-056715ec3048 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.405652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fdbf423-654e-49a6-b544-056715ec3048 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.473216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9edbe5f-c22f-49b7-b008-abb50c3a1f83 tempest-ServicesAdminNegativeTestJSON-252590966 tempest-ServicesAdminNegativeTestJSON-252590966-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.477226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c9edbe5f-c22f-49b7-b008-abb50c3a1f83 tempest-ServicesAdminNegativeTestJSON-252590966 tempest-ServicesAdminNegativeTestJSON-252590966-project] Policy check for os_compute_api:os-services:list failed with credentials {'is_admin': False, 'user_id': '9ef1f3504aac48f6a241c0d546271d06', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a2d555ea5162478db2ba424f8351b2a3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9edbe5f-c22f-49b7-b008-abb50c3a1f83 tempest-ServicesAdminNegativeTestJSON-252590966 tempest-ServicesAdminNegativeTestJSON-252590966-project] Request method failure captured: Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-services HTTP/1.1 Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyqQFEa8UIhI6qlAxmDSoLWlfNMfg5AycfTi37Y2O7-GLzHB7pH-FoigYBg_PTzeJs2hgeGFUarlJPSG_mzfMAiBHLmz6kWI9taoeeUF8w3a3brSJRcvpOqKVb9nWnlBy18NpiN8W7Co51B7wYY3Tv261cY9pAvG2fe05jcN14ZS0Q Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: a2d555ea5162478db2ba424f8351b2a3 Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServicesAdminNegativeTestJSON-252590966 Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:44.486652 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a2d555ea5162478db2ba424f8351b2a3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a2d555ea5162478db2ba424f8351b2a3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a2d555ea5162478db2ba424f8351b2a3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a2d555ea5162478db2ba424f8351b2a3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServicesAdminNegativeTestJSON-252590966 Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: a2d555ea5162478db2ba424f8351b2a3 Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServicesAdminNegativeTestJSON-252590966 Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServicesAdminNegativeTestJSON-252590966-project Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 9ef1f3504aac48f6a241c0d546271d06 Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServicesAdminNegativeTestJSON-252590966-project Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-services:list to be performed. Oct 15 20:48:44.487797 ncdv-master-1 devstack@n-api.service[138688]: action_args: {} Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 384, in index Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(services_policies.BASE_POLICY_NAME % 'list', target={}) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:44.488431 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-services:list to be performed. Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-services:list to be performed. Oct 15 20:48:44.489989 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:44.491173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9edbe5f-c22f-49b7-b008-abb50c3a1f83 tempest-ServicesAdminNegativeTestJSON-252590966 tempest-ServicesAdminNegativeTestJSON-252590966-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-services:list to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:44.491173 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9edbe5f-c22f-49b7-b008-abb50c3a1f83 tempest-ServicesAdminNegativeTestJSON-252590966 tempest-ServicesAdminNegativeTestJSON-252590966-project] 10.222.0.16 "GET /compute/v2.1/os-services" status: 403 len: 112 microversion: 2.1 time: 0.302839 Oct 15 20:48:44.491173 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 142/232] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/os-services => generated 112 bytes in 304 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:44.556590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8fdbf423-654e-49a6-b544-056715ec3048 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/4/action" status: 200 len: 227 microversion: 2.1 time: 0.166285 Oct 15 20:48:44.557548 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 91/233] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:44 2022] POST /compute/v2.1/os-aggregates/4/action => generated 227 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:44.569533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-674d8a69-8f00-4e98-ae6d-4f26694651c6 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.593621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.596861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:44.611357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.614479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.615048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.615978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.616277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.617001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.632793 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-674d8a69-8f00-4e98-ae6d-4f26694651c6 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/4" status: 200 len: 0 microversion: 2.1 time: 0.062530 Oct 15 20:48:44.633810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 143/234] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:44 2022] DELETE /compute/v2.1/os-aggregates/4 => generated 0 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:44.646226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.646892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:44.660203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.660708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.661272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.661975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.662448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.662871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.806946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:44.816265 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f2c1a61c-2790-4fea-932b-46c07ccc9f6d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1462 microversion: 2.1 time: 0.233093 Oct 15 20:48:44.821075 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 92/235] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1462 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:44.822812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5217ce4e-faf1-4dce-bf58-55a467465091 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1190824292"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:44.826615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-5217ce4e-faf1-4dce-bf58-55a467465091 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 567a4451-fa8e-4596-8cb8-582f4c282159 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:44.845432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:44.856441 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5217ce4e-faf1-4dce-bf58-55a467465091 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.036903 Oct 15 20:48:44.859461 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9210c665-708b-4cf5-a179-d49cb24df79d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1389 microversion: 2.1 time: 0.221960 Oct 15 20:48:44.860450 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 144/236] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1389 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:44.861927 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 93/237] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:44 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:44.868918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:44.869662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:44.876563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.877110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.877623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:44.878550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:44.879118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:44.879594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.040675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:45.049599 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5c67a4b4-12b4-4f39-8e55-ee89b462ec22 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1468 microversion: 2.1 time: 0.186265 Oct 15 20:48:45.050769 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 94/238] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:44 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1468 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:45.055794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:45.056478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:45.064391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.065388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.066017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.066941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.067718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.068381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.090064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-588069589", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:45.146778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:45.298773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:45.303187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:45.309468 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8bb13915-8ae3-4ed7-90cb-0ac4ae78d271 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1313 microversion: 2.1 time: 0.257364 Oct 15 20:48:45.310039 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 95/239] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:45 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1313 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:45.314890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c8e4e839-1e02-4fa6-9ef9-7def7a82bdb8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:45.325492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8e4e839-1e02-4fa6-9ef9-7def7a82bdb8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.325969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8e4e839-1e02-4fa6-9ef9-7def7a82bdb8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.326573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8e4e839-1e02-4fa6-9ef9-7def7a82bdb8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.368638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:45.369081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:45.369559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:45.369976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:45.370389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:45.389212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.389611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.390230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.391024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.391555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.392081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.424157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:45.432206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for user e32a1c53b4be44188b07c768405961ed and project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:45.448596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:45.451569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:45.511700 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6b0c24e-7b64-4013-ab8a-e9ce87710a9e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.650127 Oct 15 20:48:45.512266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c8e4e839-1e02-4fa6-9ef9-7def7a82bdb8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/5/action" status: 200 len: 242 microversion: 2.1 time: 0.201037 Oct 15 20:48:45.512861 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 96/240] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:45 2022] POST /compute/v2.1/os-aggregates/5/action => generated 242 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:45.518571 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 145/241] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 657 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:45.523045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c217eb9a-a43d-40fd-993d-af8517f0b303 req-def95a8f-d235-41e5-a610-bd0c6fe0fa87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56f00c4c-5642-40b7-b0a7-62c3ec8b3b56", "name": "network-vif-plugged", "status": "completed", "tag": "1ed285bf-73a2-45c3-8c63-b59f9d0797cc"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:45.560012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c217eb9a-a43d-40fd-993d-af8517f0b303 req-def95a8f-d235-41e5-a610-bd0c6fe0fa87 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.560500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c217eb9a-a43d-40fd-993d-af8517f0b303 req-def95a8f-d235-41e5-a610-bd0c6fe0fa87 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.561882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c217eb9a-a43d-40fd-993d-af8517f0b303 req-def95a8f-d235-41e5-a610-bd0c6fe0fa87 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.600741 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c217eb9a-a43d-40fd-993d-af8517f0b303 req-def95a8f-d235-41e5-a610-bd0c6fe0fa87 service nova] Creating event network-vif-plugged:1ed285bf-73a2-45c3-8c63-b59f9d0797cc for instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 on nch1-master-1 Oct 15 20:48:45.624034 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c217eb9a-a43d-40fd-993d-af8517f0b303 req-def95a8f-d235-41e5-a610-bd0c6fe0fa87 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107987 Oct 15 20:48:45.626355 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 97/242] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:45.639274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8e21270a-bf95-4d12-902c-be20f764b97f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:45.641182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8e21270a-bf95-4d12-902c-be20f764b97f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:45.765074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8e21270a-bf95-4d12-902c-be20f764b97f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:45.767471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:45.768098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:45.773448 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8e21270a-bf95-4d12-902c-be20f764b97f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1212 microversion: 2.1 time: 0.143968 Oct 15 20:48:45.774267 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 98/243] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:45 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1212 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:45.778161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.778690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-53df1d0a-7f5c-4e2c-a6c8-8e5f01dc5283 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:45.779004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.779610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.780623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.781398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.781981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.790227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53df1d0a-7f5c-4e2c-a6c8-8e5f01dc5283 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.790916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53df1d0a-7f5c-4e2c-a6c8-8e5f01dc5283 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.791350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53df1d0a-7f5c-4e2c-a6c8-8e5f01dc5283 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.835805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:48:45.905430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:45.940639 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-53df1d0a-7f5c-4e2c-a6c8-8e5f01dc5283 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/5/action" status: 200 len: 227 microversion: 2.1 time: 0.165385 Oct 15 20:48:45.941431 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 99/244] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:45 2022] POST /compute/v2.1/os-aggregates/5/action => generated 227 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:45.947879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:45.948758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:45.959714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.961347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.962445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:45.963412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:45.963982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:45.964666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.006365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=4,nova_api:DELETE=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:46.041643 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd5aafd0-c1cd-4945-bf39-66010bc7e084 tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] 10.222.0.16 "DELETE /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 204 len: 0 microversion: 2.1 time: 0.521546 Oct 15 20:48:46.042912 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 146/245] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:48:45 2022] DELETE /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 0 bytes in 524 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:48:46.048813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-049fe36c-4078-4628-a07a-1ad9b757e15f tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.074887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-049fe36c-4078-4628-a07a-1ad9b757e15f tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/5" status: 200 len: 0 microversion: 2.1 time: 0.030540 Oct 15 20:48:46.082379 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 147/246] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:46 2022] DELETE /compute/v2.1/os-aggregates/5 => generated 0 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:46.089162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.090082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:46.097069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.097609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.098055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.099380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.100105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.100557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.156739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:46.165001 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5a5c9ea-5377-420f-a656-a102de3fb9f7 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1389 microversion: 2.1 time: 0.221342 Oct 15 20:48:46.165623 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 100/247] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:45 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1389 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:46.176917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3df20d28-ebe8-4ce6-8c61-6349e9bddbd8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-2129103911"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.182441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-3df20d28-ebe8-4ce6-8c61-6349e9bddbd8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 219860d0-3a37-453f-89a8-8a222285e1cf for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:46.213271 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3df20d28-ebe8-4ce6-8c61-6349e9bddbd8 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.046164 Oct 15 20:48:46.214093 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 101/248] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.219404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3503f731-9864-4c22-821f-7d84ab69a459 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-670235245"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.223020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-3503f731-9864-4c22-821f-7d84ab69a459 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid be7c35ca-ca10-4a6e-b55b-1357fdafc081 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:46.250029 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3503f731-9864-4c22-821f-7d84ab69a459 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.034561 Oct 15 20:48:46.250742 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 102/249] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.258094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0b4036d-bf36-435e-9f55-f48ec1ad40ff tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.283194 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0b4036d-bf36-435e-9f55-f48ec1ad40ff tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/6" status: 200 len: 0 microversion: 2.1 time: 0.031111 Oct 15 20:48:46.284141 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 103/250] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:46 2022] DELETE /compute/v2.1/os-aggregates/6 => generated 0 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:46.298076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-984db395-12ca-4f73-8e07-46ab8bed42f3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.316482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-984db395-12ca-4f73-8e07-46ab8bed42f3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.316972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-984db395-12ca-4f73-8e07-46ab8bed42f3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.317986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-984db395-12ca-4f73-8e07-46ab8bed42f3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.331138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:46.340615 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b25e1257-b859-421e-8a01-fee258478b69 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1474 microversion: 2.1 time: 0.255701 Oct 15 20:48:46.341465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 148/251] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1474 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:46.345987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b36533c0-591a-4017-abe7-243fb2d6c9e2 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.361586 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b36533c0-591a-4017-abe7-243fb2d6c9e2 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] HTTP exception thrown: Aggregate 6 could not be found. Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b36533c0-591a-4017-abe7-243fb2d6c9e2 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Request method failure captured: Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-aggregates/6 HTTP/1.1 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx8G5ZkbKDAMV0S3Gc_ODd9Rm8zpeJsh5OfwsLGBxDdbTtXVdhgGPE2SEYYjEd_8KsAAJgEBH6dpAzzjaPiFT-xzE6xUlMHOhZWz8X9qHq47zg_P7-oXcNn3PVEjLYTdbXY5X5zpYyDXxFxTf5amy1AN5pR4fois2yE3yQVYXq3Cq8 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:46.366515 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3e09319070164e118879498437350f7f Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: exception: Aggregate 6 could not be found. Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6'} Oct 15 20:48:46.367904 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6464, in get_aggregate Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 6 could not be found. Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.368734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 6 could not be found. Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 6 could not be found. Oct 15 20:48:46.370010 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.371118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b36533c0-591a-4017-abe7-243fb2d6c9e2 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Returning 404 to user: Aggregate 6 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:46.371118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b36533c0-591a-4017-abe7-243fb2d6c9e2 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/6" status: 404 len: 77 microversion: 2.1 time: 0.025387 Oct 15 20:48:46.371118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 149/252] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/os-aggregates/6 => generated 77 bytes in 26 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:46.373197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.373934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:46.382830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.384385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.385109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.385956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.386421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.386849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.452096 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-984db395-12ca-4f73-8e07-46ab8bed42f3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/7/action" status: 200 len: 241 microversion: 2.1 time: 0.166080 Oct 15 20:48:46.452749 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 104/253] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates/7/action => generated 241 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.457355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-66c7f17c-faa6-41e8-ab11-a86f036d5005 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.480728 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-66c7f17c-faa6-41e8-ab11-a86f036d5005 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] HTTP exception thrown: Aggregate 6 could not be found. Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-66c7f17c-faa6-41e8-ab11-a86f036d5005 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Request method failure captured: Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-aggregates/6 HTTP/1.1 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8G5ZkbKDAMV0S3Gc_ODd9Rm8zpeJsh5OfwsLGBxDdbTtXVdhgGPE2SEYYjEd_8KsAAJgEBH6dpAzzjaPiFT-xzE6xUlMHOhZWz8X9qHq47zg_P7-oXcNn3PVEjLYTdbXY5X5zpYyDXxFxTf5amy1AN5pR4fois2yE3yQVYXq3Cq8 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:46.489463 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 3e09319070164e118879498437350f7f Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: exception: Aggregate 6 could not be found. Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '6'} Oct 15 20:48:46.490665 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6517, in delete_aggregate Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:46.491351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 6 could not be found. Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 6 could not be found. Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.492390 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.493464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.493464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:46.493464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:46.493464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 6 could not be found. Oct 15 20:48:46.493464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.494896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-66c7f17c-faa6-41e8-ab11-a86f036d5005 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Returning 404 to user: Aggregate 6 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:46.499577 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-66c7f17c-faa6-41e8-ab11-a86f036d5005 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/6" status: 404 len: 77 microversion: 2.1 time: 0.042620 Oct 15 20:48:46.500226 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 105/254] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:46 2022] DELETE /compute/v2.1/os-aggregates/6 => generated 77 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:46.508061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.519876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.520903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.521926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.543448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:46.546455 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] HTTP exception thrown: Aggregate 7 already has host nch1-master-1. Oct 15 20:48:46.551718 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ccccdba-30a7-4b75-bf55-c27846cb270f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1313 microversion: 2.1 time: 0.181823 Oct 15 20:48:46.552517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 150/255] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1313 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:46.557815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc671f96-dfca-4628-9a5e-1733b819cd7b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"availability_zone": "tempest-test_az-442380226", "name": "tempest-test_aggregate-515748717"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates/7/action HTTP/1.1 Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 39 Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:46.563338 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: {"add_host": {"host": "nch1-master-1"}} Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:46.564451 ncdv-master-1 devstack@n-api.service[138689]: exception: Aggregate 7 already has host nch1-master-1. Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '7', 'body': {'add_host': {'host': 'nch1-master-1'}}} Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = self._query(query) Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi conn.query(q) Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result.read() Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:48:46.565155 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry 'nch1-master-1-7' for key 'aggregate_hosts.uniq_aggregate_hosts0host0aggregate_id'") Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 73, in _host_add_to_db Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi host_ref.save(context.session) Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi session.flush() Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._flush(objects) Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:48:46.566220 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi compat.raise_( Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi flush_context.execute() Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi rec.execute(self) Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:48:46.567464 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = self._query(query) Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi conn.query(q) Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:48:46.568651 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result.read() Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'nch1-master-1-7' for key 'aggregate_hosts.uniq_aggregate_hosts0host0aggregate_id'") Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO aggregate_hosts (created_at, updated_at, host, aggregate_id) VALUES (%(created_at)s, %(updated_at)s, %(host)s, %(aggregate_id)s)] Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 10, 15, 20, 48, 46, 538728), 'updated_at': None, 'host': 'nch1-master-1', 'aggregate_id': 7}] Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.569765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 180, in _add_host Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate = self.api.add_host_to_aggregate(context, id, host) Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6641, in add_host_to_aggregate Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate.add_host(host_name) Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 395, in add_host Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _host_add_to_db(self._context, self.id, host) Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 76, in _host_add_to_db Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.AggregateHostExists(host=host, Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.AggregateHostExists: Aggregate 7 already has host nch1-master-1. Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.570825 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 187, in _add_host Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPConflict(explanation=e.format_message()) Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Aggregate 7 already has host nch1-master-1. Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:46.572058 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:46.573357 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 7 already has host nch1-master-1. Oct 15 20:48:46.573357 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.573357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 409 to user: Aggregate 7 already has host nch1-master-1. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:46.573357 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0582c27e-22cb-44fb-b638-f2cd6b343262 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/7/action" status: 409 len: 95 microversion: 2.1 time: 0.064425 Oct 15 20:48:46.573357 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 106/256] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates/7/action => generated 95 bytes in 67 msecs (HTTP/1.1 409) 9 headers in 377 bytes (1 switches on core 0) Oct 15 20:48:46.579230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ba896879-62f4-4450-a9ef-5109328a29df tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.591270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-bc671f96-dfca-4628-9a5e-1733b819cd7b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 00b3b361-0a84-4900-9c9b-775c75da21e1 for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:46.591797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba896879-62f4-4450-a9ef-5109328a29df tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.592289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba896879-62f4-4450-a9ef-5109328a29df tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.592726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba896879-62f4-4450-a9ef-5109328a29df tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.628213 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc671f96-dfca-4628-9a5e-1733b819cd7b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 220 microversion: 2.1 time: 0.074183 Oct 15 20:48:46.628911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 151/257] 10.222.0.16 () {60 vars in 1165 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates => generated 220 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.641398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-00c16aed-0a53-4dc5-8bb9-6b91c81de6db tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.668886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-00c16aed-0a53-4dc5-8bb9-6b91c81de6db tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/8" status: 200 len: 0 microversion: 2.1 time: 0.031582 Oct 15 20:48:46.669651 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 152/258] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:46 2022] DELETE /compute/v2.1/os-aggregates/8 => generated 0 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:46.678006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfe23ebf-0370-4e86-9b4a-6ac61e4b60a1 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.685845 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-bfe23ebf-0370-4e86-9b4a-6ac61e4b60a1 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] HTTP exception thrown: Aggregate 8 could not be found. Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfe23ebf-0370-4e86-9b4a-6ac61e4b60a1 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Request method failure captured: Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-aggregates/8 HTTP/1.1 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx8G5ZkbKDAMV0S3Gc_ODd9Rm8zpeJsh5OfwsLGBxDdbTtXVdhgGPE2SEYYjEd_8KsAAJgEBH6dpAzzjaPiFT-xzE6xUlMHOhZWz8X9qHq47zg_P7-oXcNn3PVEjLYTdbXY5X5zpYyDXxFxTf5amy1AN5pR4fois2yE3yQVYXq3Cq8 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:46.687440 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3e09319070164e118879498437350f7f Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: exception: Aggregate 8 could not be found. Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8'} Oct 15 20:48:46.688800 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6464, in get_aggregate Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 8 could not be found. Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.689689 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 8 could not be found. Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 8 could not be found. Oct 15 20:48:46.691506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.694321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfe23ebf-0370-4e86-9b4a-6ac61e4b60a1 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Returning 404 to user: Aggregate 8 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:46.695188 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bfe23ebf-0370-4e86-9b4a-6ac61e4b60a1 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/8" status: 404 len: 77 microversion: 2.1 time: 0.021729 Oct 15 20:48:46.695847 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 153/259] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/os-aggregates/8 => generated 77 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:46.703574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0431622-ff13-46c0-a6ee-9acf7fd56b37 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.716106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=85,nova_api:DELETE=7,nova_api:INSERT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:46.718159 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e0431622-ff13-46c0-a6ee-9acf7fd56b37 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] HTTP exception thrown: Aggregate 8 could not be found. Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0431622-ff13-46c0-a6ee-9acf7fd56b37 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Request method failure captured: Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-aggregates/8 HTTP/1.1 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx8G5ZkbKDAMV0S3Gc_ODd9Rm8zpeJsh5OfwsLGBxDdbTtXVdhgGPE2SEYYjEd_8KsAAJgEBH6dpAzzjaPiFT-xzE6xUlMHOhZWz8X9qHq47zg_P7-oXcNn3PVEjLYTdbXY5X5zpYyDXxFxTf5amy1AN5pR4fois2yE3yQVYXq3Cq8 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:46.720304 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/80a8775b9bd74eed81b89c8eb16ed258"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 80a8775b9bd74eed81b89c8eb16ed258 Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2082531489 Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3e09319070164e118879498437350f7f Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminTestJSON-2082531489-project-admin Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: exception: Aggregate 8 could not be found. Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8'} Oct 15 20:48:46.721758 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6517, in delete_aggregate Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:46.722581 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 8 could not be found. Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 8 could not be found. Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:46.724543 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 8 could not be found. Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0431622-ff13-46c0-a6ee-9acf7fd56b37 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Returning 404 to user: Aggregate 8 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:46.726001 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0431622-ff13-46c0-a6ee-9acf7fd56b37 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/8" status: 404 len: 77 microversion: 2.1 time: 0.022030 Oct 15 20:48:46.732604 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 154/260] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:46 2022] DELETE /compute/v2.1/os-aggregates/8 => generated 77 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:46.733580 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ba896879-62f4-4450-a9ef-5109328a29df tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/7/action" status: 200 len: 226 microversion: 2.1 time: 0.158425 Oct 15 20:48:46.734334 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 107/261] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates/7/action => generated 226 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:46.737810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f1fd560-85cd-4a22-8fde-aa393909751c tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1629853148"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.742634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-1f1fd560-85cd-4a22-8fde-aa393909751c tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid ed5a199f-3d64-43ef-9e20-29c6d9c5e879 for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:46.746542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f05bd8e-f570-49ab-9910-f3f5d908d15a tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.766907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f1fd560-85cd-4a22-8fde-aa393909751c tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.032934 Oct 15 20:48:46.774315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 155/262] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.776206 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6f05bd8e-f570-49ab-9910-f3f5d908d15a tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/7" status: 200 len: 0 microversion: 2.1 time: 0.033405 Oct 15 20:48:46.776965 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 108/263] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:46 2022] DELETE /compute/v2.1/os-aggregates/7 => generated 0 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:46.778927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-844524c3-4630-47a6-b699-eae1d7375aa6 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.787705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea318aad-2291-4f5d-94b2-052aad7c94db tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-666420513"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.787891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-844524c3-4630-47a6-b699-eae1d7375aa6 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/9" status: 200 len: 227 microversion: 2.1 time: 0.012277 Oct 15 20:48:46.789090 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 156/264] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/os-aggregates/9 => generated 227 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.791110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-ea318aad-2291-4f5d-94b2-052aad7c94db tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid 76342901-6785-4371-8c63-539910a098e6 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:46.799901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.800510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:46.807628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.813159 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea318aad-2291-4f5d-94b2-052aad7c94db tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.030524 Oct 15 20:48:46.814140 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 109/265] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:46.819865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-675b6b86-bb29-417f-931b-b5f205961f2b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"key": "value"}}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:46.821403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.822617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.826059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.826558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.827051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.851412 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-675b6b86-bb29-417f-931b-b5f205961f2b tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/9/action" status: 200 len: 265 microversion: 2.1 time: 0.036211 Oct 15 20:48:46.852519 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 110/266] 10.222.0.16 () {60 vars in 1191 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates/9/action => generated 265 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.862191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe7544d1af0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.866809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.867510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.869529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.870637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.871345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.871940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.900544 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ca9d3c3d-744e-4b35-8173-922374bebbcc tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.046785 Oct 15 20:48:46.901168 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 111/267] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.947590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:46.965318 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c343f4c-96e0-4054-be6a-a99896c7a658 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1216 microversion: 2.1 time: 0.169414 Oct 15 20:48:46.966110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 157/268] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1216 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:46.972524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ecc1bfe-4139-4dce-9a97-945f39ba8ff4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.985355 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ecc1bfe-4139-4dce-9a97-945f39ba8ff4 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/9" status: 200 len: 241 microversion: 2.1 time: 0.017637 Oct 15 20:48:46.986267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 158/269] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/os-aggregates/9 => generated 241 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:46.992100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe7544d3550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:46.996374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.996868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.997442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:46.998180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:46.998635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:46.999547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.035239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-967c2123-45cd-4dc1-9206-758396039193 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts?zone=internal" status: 200 len: 239 microversion: 2.1 time: 0.047760 Oct 15 20:48:47.036617 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 159/270] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:46 2022] GET /compute/v2.1/os-hosts?zone=internal => generated 239 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.041324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a096cb95-d1eb-48a6-9274-35a7fc76e657 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.070797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a096cb95-d1eb-48a6-9274-35a7fc76e657 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/9" status: 200 len: 0 microversion: 2.1 time: 0.032807 Oct 15 20:48:47.073268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 160/271] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:47 2022] DELETE /compute/v2.1/os-aggregates/9 => generated 0 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:47.082964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe7544d39d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.087638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.088274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.089870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.090979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.091563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.092190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.122110 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6fde70ac-c70f-425e-9d83-6eb3535aea55 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.045295 Oct 15 20:48:47.122710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 161/272] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.133483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bbb7fc61-601f-43b3-80bc-cd93c75adae7 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-785817782", "availability_zone": "tempest-test_az-355783680"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:47.137620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-bbb7fc61-601f-43b3-80bc-cd93c75adae7 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 83b3dace-0af1-4b46-9d4d-f1529d89f083 for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:47.166587 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bbb7fc61-601f-43b3-80bc-cd93c75adae7 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 221 microversion: 2.1 time: 0.036807 Oct 15 20:48:47.167833 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 162/273] 10.222.0.16 () {60 vars in 1165 bytes} [Sat Oct 15 20:48:47 2022] POST /compute/v2.1/os-aggregates => generated 221 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.172902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe7544adee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.182876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.183723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.184482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.211975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.213574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.215010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.251874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f3859d7-f8c8-41fd-a57e-a7c7533fab52 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nch1-master-1" status: 200 len: 598 microversion: 2.1 time: 0.081892 Oct 15 20:48:47.252648 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 163/274] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/os-hosts/nch1-master-1 => generated 598 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.258061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-835d6041-db98-4e1a-83a9-63d3b6e1e495 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-785817782_new", "availability_zone": "tempest-test_az-355783680_new"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:47.292626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-835d6041-db98-4e1a-83a9-63d3b6e1e495 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.293563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-835d6041-db98-4e1a-83a9-63d3b6e1e495 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.294819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-835d6041-db98-4e1a-83a9-63d3b6e1e495 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.329076 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-835d6041-db98-4e1a-83a9-63d3b6e1e495 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-aggregates/11" status: 200 len: 334 microversion: 2.1 time: 0.075283 Oct 15 20:48:47.329810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 164/275] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Oct 15 20:48:47 2022] PUT /compute/v2.1/os-aggregates/11 => generated 334 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.335196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.336257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:47.343615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.344104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.344541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.345285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.345752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.346177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.346613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a817b683-9acf-4052-91ad-2c442ec4a7a9 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:47.351726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a817b683-9acf-4052-91ad-2c442ec4a7a9 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Policy check for os_compute_api:os-aggregates:add_host failed with credentials {'is_admin': False, 'user_id': 'b13dec0172eb4a7b9c28d89456d2a6cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41d79b5940a246eabe32fc5c5309a3d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a817b683-9acf-4052-91ad-2c442ec4a7a9 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Request method failure captured: Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates/10/action HTTP/1.1 Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 39 Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx6FClK4YztMu1QRwJjB-jSkKQhKEXhbefsmXEwWZ5W5dbc7ks4mEykosCpWwsCXXwkdOJTquiqDSqQAustuh5J8hwVGdXi0Ooa88kCvR9lwHBcO2xjLZjGZ5dPFlz-VYUgk9WbZisjEymizc_C5NFCGaG7p5Y2vMdxfpIdtDySidw Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:47.353551 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: b13dec0172eb4a7b9c28d89456d2a6cb Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: {"add_host": {"host": "nch1-master-1"}} Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:47.355031 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '10', 'body': {'add_host': {'host': 'nch1-master-1'}}} Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 172, in _add_host Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'add_host', target={}) Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:47.355927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Oct 15 20:48:47.357406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:47.358965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a817b683-9acf-4052-91ad-2c442ec4a7a9 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:47.361635 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a817b683-9acf-4052-91ad-2c442ec4a7a9 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] 10.222.0.16 "POST /compute/v2.1/os-aggregates/10/action" status: 403 len: 118 microversion: 2.1 time: 0.459221 Oct 15 20:48:47.362316 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 112/276] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:48:46 2022] POST /compute/v2.1/os-aggregates/10/action => generated 118 bytes in 460 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:47.367776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Calling method '.version_select at 0x7fe7544d1550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.375583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.376864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.377486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.397872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.398374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.398819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.442699 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23960b3c-3458-4336-8990-ccf2f93104d4 tempest-HostsAdminTestJSON-121254418 tempest-HostsAdminTestJSON-121254418-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nch2-master-1" status: 200 len: 729 microversion: 2.1 time: 0.078505 Oct 15 20:48:47.448918 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 113/277] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/os-hosts/nch2-master-1 => generated 729 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.465408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31eb1687-f891-4c35-a99a-b45f8542f431 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.476299 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31eb1687-f891-4c35-a99a-b45f8542f431 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 551 microversion: 2.1 time: 0.025512 Oct 15 20:48:47.477126 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 114/278] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/os-aggregates => generated 551 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.482128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.482928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:47.493000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.493727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.494463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.495238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.495709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.496144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.588787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:47.598222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15fcb7b6-1405-4422-ba59-c4e52bd758db tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1587 microversion: 2.1 time: 0.267320 Oct 15 20:48:47.600195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 165/279] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1587 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:47.627760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-788ed910-eac1-4d60-9254-56f70c7d183d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.662112 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-788ed910-eac1-4d60-9254-56f70c7d183d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/10" status: 200 len: 0 microversion: 2.1 time: 0.048716 Oct 15 20:48:47.662852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 166/280] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:47 2022] DELETE /compute/v2.1/os-aggregates/10 => generated 0 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:47.668790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3deaf5d-91b3-4432-99a8-004256562943 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.707594 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3deaf5d-91b3-4432-99a8-004256562943 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/11" status: 200 len: 0 microversion: 2.1 time: 0.043317 Oct 15 20:48:47.714013 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 167/281] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:47 2022] DELETE /compute/v2.1/os-aggregates/11 => generated 0 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:47.719786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.720513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:47.730853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.730853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.730853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.731674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.732279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.732930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.747393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:47.756152 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-595205c4-cefb-4d65-8a0d-e61544fe8e03 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1474 microversion: 2.1 time: 0.277819 Oct 15 20:48:47.756738 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 115/282] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1474 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:47.766424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:47.766919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:47.782888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.783389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.783991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.784578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.786862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.787298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.966945 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e5cd5e65-4025-4ee7-b271-89f10173a716 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.209029 Oct 15 20:48:47.968355 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 116/283] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56/os-interface => generated 284 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:47.976347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a4ba11a6-dd64-4851-8f01-1e70ed47ee50 req-3c3df39b-c3b5-4a72-83ac-ecf881c175f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d1ddd46-47f6-4948-a897-3b963780991e", "name": "network-vif-plugged", "status": "completed", "tag": "7d14c08a-67ff-4f01-a95d-0c19d04975e3"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:47.989958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:47.997331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a4ba11a6-dd64-4851-8f01-1e70ed47ee50 req-3c3df39b-c3b5-4a72-83ac-ecf881c175f1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:47.998204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a4ba11a6-dd64-4851-8f01-1e70ed47ee50 req-3c3df39b-c3b5-4a72-83ac-ecf881c175f1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:47.998864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a4ba11a6-dd64-4851-8f01-1e70ed47ee50 req-3c3df39b-c3b5-4a72-83ac-ecf881c175f1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:47.999258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d76538b-ddab-4551-9ab2-e1896d2dd7de tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1313 microversion: 2.1 time: 0.284011 Oct 15 20:48:48.003073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 168/284] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:47 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1313 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:48.011342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c53ec7e3-ffd0-4362-9d61-05c4532ec52d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1091697904"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.016535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-c53ec7e3-ffd0-4362-9d61-05c4532ec52d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid b6e9bff2-0d42-4574-a367-4a93ed577589 for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:48.031520 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a4ba11a6-dd64-4851-8f01-1e70ed47ee50 req-3c3df39b-c3b5-4a72-83ac-ecf881c175f1 service nova] Creating event network-vif-plugged:7d14c08a-67ff-4f01-a95d-0c19d04975e3 for instance 5d1ddd46-47f6-4948-a897-3b963780991e on nch2-master-1 Oct 15 20:48:48.038178 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a4ba11a6-dd64-4851-8f01-1e70ed47ee50 req-3c3df39b-c3b5-4a72-83ac-ecf881c175f1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068397 Oct 15 20:48:48.038896 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 117/285] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:48.044516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f08a7f46-4f3c-48bd-bf49-37f33dbba661 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-67405678"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.053913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-f08a7f46-4f3c-48bd-bf49-37f33dbba661 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Generated uuid 750cf177-56de-471c-910d-b43b4ec3a52a for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:48.060371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=3,nova_api:DELETE=10,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:48.076351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c53ec7e3-ffd0-4362-9d61-05c4532ec52d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.071732 Oct 15 20:48:48.078352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 169/286] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:48.087714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.088907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:48.104813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.105918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.106646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.108256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.108885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.109507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.135981 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f08a7f46-4f3c-48bd-bf49-37f33dbba661 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.092405 Oct 15 20:48:48.138221 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 118/287] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:48.146050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "tempest-nonexist_host-1828333424"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.166430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.167588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.168248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.178647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.179759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.180654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.204669 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] HTTP exception thrown: Compute host tempest-nonexist_host-1828333424 could not be found. Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates/12/action HTTP/1.1 Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 58 Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:48.207140 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: {"add_host": {"host": "tempest-nonexist_host-1828333424"}} Oct 15 20:48:48.208817 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: exception: Compute host tempest-nonexist_host-1828333424 could not be found. Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '12', 'body': {'add_host': {'host': 'tempest-nonexist_host-1828333424'}}} Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5999, in _get_service_in_cell_by_host Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'tempest-nonexist_host-1828333424' is not mapped to any cell Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.209724 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6005, in _get_service_in_cell_by_host Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi service = _find_service_in_cell(context, service_host=host_name) Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 180, in _add_host Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate = self.api.add_host_to_aggregate(context, id, host) Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6621, in add_host_to_aggregate Oct 15 20:48:48.211314 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi service = _get_service_in_cell_by_host(context, host_name) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6007, in _get_service_in_cell_by_host Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host_name) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host tempest-nonexist_host-1828333424 could not be found. Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 184, in _add_host Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Compute host tempest-nonexist_host-1828333424 could not be found. Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.212587 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Compute host tempest-nonexist_host-1828333424 could not be found. Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 404 to user: Compute host tempest-nonexist_host-1828333424 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4bf12b9f-ebe7-41e0-84f4-b238a3e1aead tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/12/action" status: 404 len: 111 microversion: 2.1 time: 0.068861 Oct 15 20:48:48.214151 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 119/288] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates/12/action => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:48.225307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fdc97e7c-88ba-4f19-a803-6ee2f41d4368 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.234983 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fdc97e7c-88ba-4f19-a803-6ee2f41d4368 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 444 microversion: 2.1 time: 0.021121 Oct 15 20:48:48.235559 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 120/289] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:48:48 2022] GET /compute/v2.1/os-aggregates => generated 444 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:48.241363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8fcf19d5-456b-42ee-9a03-a5b38e85cbc1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.272845 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8fcf19d5-456b-42ee-9a03-a5b38e85cbc1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/12" status: 200 len: 0 microversion: 2.1 time: 0.035990 Oct 15 20:48:48.277847 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 121/290] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/os-aggregates/12 => generated 0 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:48.293470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f3be8c6-47f4-4c85-bbfb-11b61288b7d7 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.332697 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f3be8c6-47f4-4c85-bbfb-11b61288b7d7 tempest-AggregatesAdminTestJSON-2082531489 tempest-AggregatesAdminTestJSON-2082531489-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/13" status: 200 len: 0 microversion: 2.1 time: 0.057170 Oct 15 20:48:48.334081 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 122/291] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/os-aggregates/13 => generated 0 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:48.341963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-baf6b2ba-57a6-436f-9f15-f01fced5c1f8 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-baf6b2ba-57a6-436f-9f15-f01fced5c1f8 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 283 Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:48.345969 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.347673 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: {"aggregate": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'body': {'aggregate': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:48:48.348277 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['aggregate']['properties']['name']: Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance['aggregate']: Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.349164 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.350375 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.351439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:48:48.351439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:48:48.351439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 20:48:48.351439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.351439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-baf6b2ba-57a6-436f-9f15-f01fced5c1f8 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.351859 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-baf6b2ba-57a6-436f-9f15-f01fced5c1f8 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 400 len: 619 microversion: 2.1 time: 0.011822 Oct 15 20:48:48.354865 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 123/292] 10.222.0.16 () {60 vars in 1165 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 619 bytes in 20 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:48:48.359879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.360499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:48.368431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.369254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.369844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.370716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.371340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.371768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.388872 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] HTTP exception thrown: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Request method failure captured: Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 HTTP/1.1 Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx6n7Ar_VhiRPEzhw3IQFnKIzWN2MkvMJ178i3KGZHfD0lS4v8sDoo6yfpaiQwrFquIb-7Lr-XVGz7rxx8oGZT5N1Abp4Z5SN_dxfXbWmYeUjBWAkZ63qdFZRberO0tsstTVpnejoYlmarA6ku-dZTaLbVoJjI66wQA_C7XYU3qkh0 Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 474c2d620e224f06b409adf776004cac Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:48.391408 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/474c2d620e224f06b409adf776004cac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_474c2d620e224f06b409adf776004cac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/474c2d620e224f06b409adf776004cac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/474c2d620e224f06b409adf776004cac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 474c2d620e224f06b409adf776004cac Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminTestJSON-2070443666-project Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4b16bd0809f3411a91fae6c5e2f3acee Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminTestJSON-2070443666-project Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.392808 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'd00bcb49-6cd6-4773-ac83-506cfc487514'} Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.393592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.394856 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.397626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Returning 404 to user: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.398656 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b40cf817-c1dc-4927-a6ba-00143e8b7b84 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 404 len: 111 microversion: 2.1 time: 0.042675 Oct 15 20:48:48.399395 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 124/293] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:48.404811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c03ea34-cdcb-4cf1-9178-23c603b1dfd3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": ""}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c03ea34-cdcb-4cf1-9178-23c603b1dfd3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 27 Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:48.408706 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: {"aggregate": {"name": ""}} Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.410169 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid input for field/attribute name. Value: . '' is too short Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'body': {'aggregate': {'name': ''}}} Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['aggregate']['properties']['name']: Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance['aggregate']: Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi '' Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.411041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Oct 15 20:48:48.412512 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.414605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c03ea34-cdcb-4cf1-9178-23c603b1dfd3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.415448 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c03ea34-cdcb-4cf1-9178-23c603b1dfd3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 400 len: 108 microversion: 2.1 time: 0.014767 Oct 15 20:48:48.416177 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 125/294] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 108 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:48:48.425563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.426458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] [instance: d00bcb49-6cd6-4773-ac83-506cfc487514] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:48.433790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.434278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.434772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.435447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.435903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.436329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.450176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:48.452208 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] HTTP exception thrown: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Request method failure captured: Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 HTTP/1.1 Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx6n7Ar_VhiRPEzhw3IQFnKIzWN2MkvMJ178i3KGZHfD0lS4v8sDoo6yfpaiQwrFquIb-7Lr-XVGz7rxx8oGZT5N1Abp4Z5SN_dxfXbWmYeUjBWAkZ63qdFZRberO0tsstTVpnejoYlmarA6ku-dZTaLbVoJjI66wQA_C7XYU3qkh0 Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 474c2d620e224f06b409adf776004cac Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:48.453965 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/474c2d620e224f06b409adf776004cac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_474c2d620e224f06b409adf776004cac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/474c2d620e224f06b409adf776004cac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/474c2d620e224f06b409adf776004cac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 474c2d620e224f06b409adf776004cac Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-2070443666 Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminTestJSON-2070443666-project Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4b16bd0809f3411a91fae6c5e2f3acee Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminTestJSON-2070443666-project Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.455497 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'd00bcb49-6cd6-4773-ac83-506cfc487514'} Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.456258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. Oct 15 20:48:48.457514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.459919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] Returning 404 to user: Instance d00bcb49-6cd6-4773-ac83-506cfc487514 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.461599 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-519570c4-5c28-408f-9684-844303e9b729 tempest-AggregatesAdminTestJSON-2070443666 tempest-AggregatesAdminTestJSON-2070443666-project] 10.222.0.16 "GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514" status: 404 len: 111 microversion: 2.1 time: 0.040003 Oct 15 20:48:48.462682 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 126/295] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:48 2022] GET /compute/v2.1/servers/d00bcb49-6cd6-4773-ac83-506cfc487514 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:48.465577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba79ac0d-70ba-4af5-bf47-ff52525726ae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1282 microversion: 2.1 time: 0.385911 Oct 15 20:48:48.466428 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 170/296] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:48 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1282 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:48.468629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a287d0e-f945-4af8-9d7d-595d8b7f602b tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-732625068"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.472099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a287d0e-f945-4af8-9d7d-595d8b7f602b tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Policy check for os_compute_api:os-aggregates:create failed with credentials {'is_admin': False, 'user_id': 'b13dec0172eb4a7b9c28d89456d2a6cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41d79b5940a246eabe32fc5c5309a3d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a287d0e-f945-4af8-9d7d-595d8b7f602b tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Request method failure captured: Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 59 Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx6FClK4YztMu1QRwJjB-jSkKQhKEXhbefsmXEwWZ5W5dbc7ks4mEykosCpWwsCXXwkdOJTquiqDSqQAustuh5J8hwVGdXi0Ooa88kCvR9lwHBcO2xjLZjGZ5dPFlz-VYUgk9WbZisjEymizc_C5NFCGaG7p5Y2vMdxfpIdtDySidw Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:48.473987 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: b13dec0172eb4a7b9c28d89456d2a6cb Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: {"aggregate": {"name": "tempest-test_aggregate-732625068"}} Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.475119 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'body': {'aggregate': {'name': 'tempest-test_aggregate-732625068'}}} Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 70, in create Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'create', target={}) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:48.475795 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:48.476766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Oct 15 20:48:48.477826 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.478560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a287d0e-f945-4af8-9d7d-595d8b7f602b tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.479539 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a287d0e-f945-4af8-9d7d-595d8b7f602b tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 403 len: 116 microversion: 2.1 time: 0.015638 Oct 15 20:48:48.484977 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 127/297] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 116 bytes in 21 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:48.491189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30c29f8b-177a-4233-9b53-c92ae1068270 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1012187050"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.495049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-30c29f8b-177a-4233-9b53-c92ae1068270 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid f543a722-d8fa-4fd2-a216-d73558b5082d for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:48.536680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30c29f8b-177a-4233-9b53-c92ae1068270 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.050560 Oct 15 20:48:48.544051 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 171/298] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:48.550465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d24ce008-0880-423d-8c9c-c4697d583b0f tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1012187050"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.556414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-d24ce008-0880-423d-8c9c-c4697d583b0f tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid 3e9abd8a-e874-4f13-ac0b-462f625a6d38 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:48.568922 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-d24ce008-0880-423d-8c9c-c4697d583b0f tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] HTTP exception thrown: Aggregate tempest-test_aggregate-1012187050 already exists. Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d24ce008-0880-423d-8c9c-c4697d583b0f tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 60 Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:48.571095 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: {"aggregate": {"name": "tempest-test_aggregate-1012187050"}} Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.572570 ncdv-master-1 devstack@n-api.service[138689]: exception: Aggregate tempest-test_aggregate-1012187050 already exists. Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'body': {'aggregate': {'name': 'tempest-test_aggregate-1012187050'}}} Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 78, in create Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate = self.api.create_aggregate(context, name, avail_zone) Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6458, in create_aggregate Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate.create() Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 310, in create Oct 15 20:48:48.573987 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_create_in_db(self._context, updates, Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 175, in _aggregate_create_in_db Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.AggregateNameExists(aggregate_name=values['name']) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNameExists: Aggregate tempest-test_aggregate-1012187050 already exists. Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:48.575686 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 80, in create Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPConflict(explanation=e.format_message()) Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Aggregate tempest-test_aggregate-1012187050 already exists. Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate tempest-test_aggregate-1012187050 already exists. Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d24ce008-0880-423d-8c9c-c4697d583b0f tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 409 to user: Aggregate tempest-test_aggregate-1012187050 already exists. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.579086 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d24ce008-0880-423d-8c9c-c4697d583b0f tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 409 len: 111 microversion: 2.1 time: 0.027352 Oct 15 20:48:48.580629 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 128/299] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 111 bytes in 34 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:48.587459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-661de038-0239-47d8-aaaf-8c42cf1fb358 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.621387 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-661de038-0239-47d8-aaaf-8c42cf1fb358 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/14" status: 200 len: 0 microversion: 2.1 time: 0.038977 Oct 15 20:48:48.637356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef78badd-d227-43be-9643-31c03c8f21c3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-513915342"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.643338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-ef78badd-d227-43be-9643-31c03c8f21c3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid b155941a-3f50-4b97-96c0-695c3a91a63d for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:48.644812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 172/300] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/os-aggregates/14 => generated 0 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:48.703875 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef78badd-d227-43be-9643-31c03c8f21c3 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.071298 Oct 15 20:48:48.720000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-752ab1e5-b53e-4c53-89f0-a01033eb6789 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.721655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-752ab1e5-b53e-4c53-89f0-a01033eb6789 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Policy check for os_compute_api:os-aggregates:delete failed with credentials {'is_admin': False, 'user_id': 'b13dec0172eb4a7b9c28d89456d2a6cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41d79b5940a246eabe32fc5c5309a3d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-752ab1e5-b53e-4c53-89f0-a01033eb6789 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Request method failure captured: Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-aggregates/15 HTTP/1.1 Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx6FClK4YztMu1QRwJjB-jSkKQhKEXhbefsmXEwWZ5W5dbc7ks4mEykosCpWwsCXXwkdOJTquiqDSqQAustuh5J8hwVGdXi0Ooa88kCvR9lwHBcO2xjLZjGZ5dPFlz-VYUgk9WbZisjEymizc_C5NFCGaG7p5Y2vMdxfpIdtDySidw Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:48.724679 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: b13dec0172eb4a7b9c28d89456d2a6cb Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Oct 15 20:48:48.726066 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '15'} Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 147, in delete Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'delete', target={}) Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:48.726731 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.727684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-752ab1e5-b53e-4c53-89f0-a01033eb6789 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.728563 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-752ab1e5-b53e-4c53-89f0-a01033eb6789 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/15" status: 403 len: 116 microversion: 2.1 time: 0.011993 Oct 15 20:48:48.730628 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 129/301] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:48.734174 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 173/302] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/os-aggregates/15 => generated 116 bytes in 20 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:48.739900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-30c4129d-5772-4cf5-bc9e-f88da0fc9498 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.793319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.794051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:48.798731 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-30c4129d-5772-4cf5-bc9e-f88da0fc9498 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/15" status: 200 len: 0 microversion: 2.1 time: 0.066921 Oct 15 20:48:48.799536 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 130/303] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/os-aggregates/15 => generated 0 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:48.810909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4ce56313-be45-4154-979b-2fb1bcedc2c5 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.820527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.821269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.821961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.822804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.824109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.824776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.841660 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-4ce56313-be45-4154-979b-2fb1bcedc2c5 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] HTTP exception thrown: Aggregate -1 could not be found. Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4ce56313-be45-4154-979b-2fb1bcedc2c5 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-aggregates/-1 HTTP/1.1 Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:48.843539 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: exception: Aggregate -1 could not be found. Oct 15 20:48:48.844755 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '-1'} Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6517, in delete_aggregate Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Oct 15 20:48:48.845444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate -1 could not be found. Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate -1 could not be found. Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.851654 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate -1 could not be found. Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4ce56313-be45-4154-979b-2fb1bcedc2c5 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 404 to user: Aggregate -1 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4ce56313-be45-4154-979b-2fb1bcedc2c5 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/-1" status: 404 len: 78 microversion: 2.1 time: 0.038122 Oct 15 20:48:48.853488 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 131/304] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:48 2022] DELETE /compute/v2.1/os-aggregates/-1 => generated 78 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:48.861175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d4bb2b16-b8c9-4b95-b645-fe83387e29a7 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1991663235"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.864864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-d4bb2b16-b8c9-4b95-b645-fe83387e29a7 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid e98ae1c1-85e9-4f5f-8e97-9b7915b4a4ae for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:48.912545 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d4bb2b16-b8c9-4b95-b645-fe83387e29a7 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.060504 Oct 15 20:48:48.921897 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 132/305] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:48.949159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-37789fac-7a11-4623-9292-33d1207a5db1 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:48.952235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-37789fac-7a11-4623-9292-33d1207a5db1 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Policy check for os_compute_api:os-aggregates:show failed with credentials {'is_admin': False, 'user_id': 'b13dec0172eb4a7b9c28d89456d2a6cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41d79b5940a246eabe32fc5c5309a3d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-37789fac-7a11-4623-9292-33d1207a5db1 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Request method failure captured: Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-aggregates/16 HTTP/1.1 Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxx6FClK4YztMu1QRwJjB-jSkKQhKEXhbefsmXEwWZ5W5dbc7ks4mEykosCpWwsCXXwkdOJTquiqDSqQAustuh5J8hwVGdXi0Ooa88kCvR9lwHBcO2xjLZjGZ5dPFlz-VYUgk9WbZisjEymizc_C5NFCGaG7p5Y2vMdxfpIdtDySidw Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:48.954506 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: b13dec0172eb4a7b9c28d89456d2a6cb Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Oct 15 20:48:48.955707 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '16'} Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 100, in show Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'show', target={}) Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:48.956352 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:48.958086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-37789fac-7a11-4623-9292-33d1207a5db1 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:48.959873 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-37789fac-7a11-4623-9292-33d1207a5db1 tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] 10.222.0.16 "GET /compute/v2.1/os-aggregates/16" status: 403 len: 114 microversion: 2.1 time: 0.033180 Oct 15 20:48:48.959873 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 133/306] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:48 2022] GET /compute/v2.1/os-aggregates/16 => generated 114 bytes in 35 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:48.967093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0b278f8c-b0f5-426f-89fc-999e7b4e79fd req-35249c5a-0fa4-4c59-857f-dac2fbd5a0e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d6bfd383-07fb-4c4e-9d98-00a96ea7274d", "name": "network-vif-deleted", "tag": "86dae641-1ad4-48b9-9898-d71d3c88a8c4"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:48.979379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0b278f8c-b0f5-426f-89fc-999e7b4e79fd req-35249c5a-0fa4-4c59-857f-dac2fbd5a0e6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:48.980052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0b278f8c-b0f5-426f-89fc-999e7b4e79fd req-35249c5a-0fa4-4c59-857f-dac2fbd5a0e6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:48.980851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0b278f8c-b0f5-426f-89fc-999e7b4e79fd req-35249c5a-0fa4-4c59-857f-dac2fbd5a0e6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:48.993708 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0b278f8c-b0f5-426f-89fc-999e7b4e79fd req-35249c5a-0fa4-4c59-857f-dac2fbd5a0e6 service nova] Creating event network-vif-deleted:86dae641-1ad4-48b9-9898-d71d3c88a8c4 for instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d on nch2-master-1 Oct 15 20:48:49.000214 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0b278f8c-b0f5-426f-89fc-999e7b4e79fd req-35249c5a-0fa4-4c59-857f-dac2fbd5a0e6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036514 Oct 15 20:48:49.000917 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 134/307] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:49.007932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b99bae8-3a52-411a-817d-67c4e3c8db3d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.051846 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b99bae8-3a52-411a-817d-67c4e3c8db3d tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/16" status: 200 len: 0 microversion: 2.1 time: 0.046635 Oct 15 20:48:49.052001 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 135/308] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:49 2022] DELETE /compute/v2.1/os-aggregates/16 => generated 0 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:49.061289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.062255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:49.071437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.072436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.073274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.074526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.075188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.076480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.098642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:49.111539 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-485be118-e8c8-4f34-a1ec-13678f89d4f9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1432 microversion: 2.1 time: 0.332474 Oct 15 20:48:49.130090 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 174/309] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:48 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1432 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:49.135493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0e71f95-76ed-449c-8686-565093ac9e3e tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.148416 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a0e71f95-76ed-449c-8686-565093ac9e3e tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] HTTP exception thrown: Aggregate -1 could not be found. Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0e71f95-76ed-449c-8686-565093ac9e3e tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-aggregates/-1 HTTP/1.1 Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:49.159522 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: exception: Aggregate -1 could not be found. Oct 15 20:48:49.161329 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '-1'} Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6464, in get_aggregate Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate -1 could not be found. Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.161961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate -1 could not be found. Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate -1 could not be found. Oct 15 20:48:49.163359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.165798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0e71f95-76ed-449c-8686-565093ac9e3e tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 404 to user: Aggregate -1 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:49.166840 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0e71f95-76ed-449c-8686-565093ac9e3e tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/-1" status: 404 len: 78 microversion: 2.1 time: 0.035390 Oct 15 20:48:49.167612 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 175/310] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:49 2022] GET /compute/v2.1/os-aggregates/-1 => generated 78 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:49.178399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abd7b2cd-8dc9-41b8-8ee4-5538991fbd0c tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.180290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abd7b2cd-8dc9-41b8-8ee4-5538991fbd0c tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Policy check for os_compute_api:os-aggregates:index failed with credentials {'is_admin': False, 'user_id': 'b13dec0172eb4a7b9c28d89456d2a6cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41d79b5940a246eabe32fc5c5309a3d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abd7b2cd-8dc9-41b8-8ee4-5538991fbd0c tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Request method failure captured: Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-aggregates HTTP/1.1 Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx6FClK4YztMu1QRwJjB-jSkKQhKEXhbefsmXEwWZ5W5dbc7ks4mEykosCpWwsCXXwkdOJTquiqDSqQAustuh5J8hwVGdXi0Ooa88kCvR9lwHBcO2xjLZjGZ5dPFlz-VYUgk9WbZisjEymizc_C5NFCGaG7p5Y2vMdxfpIdtDySidw Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:49.182105 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: b13dec0172eb4a7b9c28d89456d2a6cb Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Oct 15 20:48:49.183351 ncdv-master-1 devstack@n-api.service[138688]: action_args: {} Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 55, in index Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'index', target={}) Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:49.183983 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Oct 15 20:48:49.184950 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.187420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abd7b2cd-8dc9-41b8-8ee4-5538991fbd0c tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:49.188490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abd7b2cd-8dc9-41b8-8ee4-5538991fbd0c tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 403 len: 115 microversion: 2.1 time: 0.014918 Oct 15 20:48:49.189298 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 176/311] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:48:49 2022] GET /compute/v2.1/os-aggregates => generated 115 bytes in 16 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:49.209713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-add320ce-7ddf-41c8-82d6-7d4cc3540399 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1587097836"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:49.214822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-add320ce-7ddf-41c8-82d6-7d4cc3540399 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid 256a6322-6e06-4cf9-9357-424d293f0ba2 for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:49.259382 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-add320ce-7ddf-41c8-82d6-7d4cc3540399 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.060667 Oct 15 20:48:49.260414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 177/312] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:49 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:49.276610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-969e9c96-079d-4084-a14c-ee10be2d82c1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:49.296761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-969e9c96-079d-4084-a14c-ee10be2d82c1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.297711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-969e9c96-079d-4084-a14c-ee10be2d82c1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.298305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-969e9c96-079d-4084-a14c-ee10be2d82c1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.460412 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-969e9c96-079d-4084-a14c-ee10be2d82c1 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/17/action" status: 200 len: 243 microversion: 2.1 time: 0.191761 Oct 15 20:48:49.465459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 178/313] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:48:49 2022] POST /compute/v2.1/os-aggregates/17/action => generated 243 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:48:49.475551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08e4dada-b9ac-4e3e-9fc8-6ea524f3d0eb tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:49.480373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-08e4dada-b9ac-4e3e-9fc8-6ea524f3d0eb tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Policy check for os_compute_api:os-aggregates:remove_host failed with credentials {'is_admin': False, 'user_id': 'b13dec0172eb4a7b9c28d89456d2a6cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '41d79b5940a246eabe32fc5c5309a3d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08e4dada-b9ac-4e3e-9fc8-6ea524f3d0eb tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Request method failure captured: Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-aggregates/17/action HTTP/1.1 Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 42 Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx6FClK4YztMu1QRwJjB-jSkKQhKEXhbefsmXEwWZ5W5dbc7ks4mEykosCpWwsCXXwkdOJTquiqDSqQAustuh5J8hwVGdXi0Ooa88kCvR9lwHBcO2xjLZjGZ5dPFlz-VYUgk9WbZisjEymizc_C5NFCGaG7p5Y2vMdxfpIdtDySidw Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:49.482207 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/41d79b5940a246eabe32fc5c5309a3d6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 41d79b5940a246eabe32fc5c5309a3d6 Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-731858378 Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: b13dec0172eb4a7b9c28d89456d2a6cb Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-731858378-project Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: {"remove_host": {"host": "nch1-master-1"}} Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:49.483422 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '17', 'body': {'remove_host': {'host': 'nch1-master-1'}}} Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 201, in _remove_host Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'remove_host', target={}) Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:49.484144 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Oct 15 20:48:49.485168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.487896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08e4dada-b9ac-4e3e-9fc8-6ea524f3d0eb tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:49.487896 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08e4dada-b9ac-4e3e-9fc8-6ea524f3d0eb tempest-AggregatesAdminNegativeTestJSON-731858378 tempest-AggregatesAdminNegativeTestJSON-731858378-project] 10.222.0.16 "POST /compute/v2.1/os-aggregates/17/action" status: 403 len: 121 microversion: 2.1 time: 0.014097 Oct 15 20:48:49.487896 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 179/314] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:48:49 2022] POST /compute/v2.1/os-aggregates/17/action => generated 121 bytes in 15 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:48:49.495072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4feae61-d4be-4be5-a2d3-11853ec7888b tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:49.507665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4feae61-d4be-4be5-a2d3-11853ec7888b tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.508159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4feae61-d4be-4be5-a2d3-11853ec7888b tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.508601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4feae61-d4be-4be5-a2d3-11853ec7888b tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.638681 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4feae61-d4be-4be5-a2d3-11853ec7888b tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/17/action" status: 200 len: 228 microversion: 2.1 time: 0.147979 Oct 15 20:48:49.639630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 180/315] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:48:49 2022] POST /compute/v2.1/os-aggregates/17/action => generated 228 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:49.650045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.651036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:49.658827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.659459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.660030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.661756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.662342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.663808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.747207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:49.756903 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c6193a1-4f5f-4481-bc64-fe04f3c27427 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1270 microversion: 2.1 time: 0.116120 Oct 15 20:48:49.757871 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 181/316] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:49 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1270 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:49.763119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-388f4671-7b47-4b20-b585-3c29a941a03a tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.788288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-388f4671-7b47-4b20-b585-3c29a941a03a tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/17" status: 200 len: 0 microversion: 2.1 time: 0.028746 Oct 15 20:48:49.789145 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 182/317] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:49 2022] DELETE /compute/v2.1/os-aggregates/17 => generated 0 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:49.802562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d9106ee3-f8b0-494e-9526-d549dc445c3c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1801166215"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:49.808521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.aggregate [None req-d9106ee3-f8b0-494e-9526-d549dc445c3c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Generated uuid 63d2cf95-e8df-42e5-a281-4524fce5bd1b for aggregate {{(pid=138688) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 20:48:49.838939 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d9106ee3-f8b0-494e-9526-d549dc445c3c tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.043937 Oct 15 20:48:49.843777 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 183/318] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 20:48:49 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:49.868545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nonexist_host"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:49.881342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:49.883319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.884124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.884618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.891957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52351b52-8d15-476d-8c33-f14d971bcab1 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1313 microversion: 2.1 time: 0.838160 Oct 15 20:48:49.893518 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 136/319] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:49 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1313 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:49.894227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:49.894708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:49.895398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:49.912182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.compute.aggregates [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Failed to remove host nonexist_host from aggregate 18. Error: Compute host nonexist_host could not be found.: nova.exception.ComputeHostNotFound: Compute host nonexist_host could not be found. Oct 15 20:48:49.912920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] HTTP exception thrown: Cannot remove host nonexist_host in aggregate 18 Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Request method failure captured: Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-aggregates/18/action HTTP/1.1 Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 42 Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxx8xO8I3VOfoynbwbNV6kVKbL07G7062D2-TypZFwKpdvLNvpOcaRCDrFj9W2Qc0oxXLPjF6JauZc5FT-LLT3fz1d1eNtUk3cDVycHYoC1chfvs6fi4xNMj8xtN9Np3q5pDlBQYxEwCDY_xbWJ4V2AVnKN6hqQxwmZ-_dxkDMFfK7c Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:49.918917 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1b4645ec7c584e549cdc3cbf9965a11d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 1b4645ec7c584e549cdc3cbf9965a11d Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179 Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a1e9c90d8c1549aa83893ed2476c3a6d Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: {"remove_host": {"host": "nonexist_host"}} Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:48:49.920188 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot remove host nonexist_host in aggregate 18 Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '18', 'body': {'remove_host': {'host': 'nonexist_host'}}} Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5999, in _get_service_in_cell_by_host Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexist_host' is not mapped to any cell Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6005, in _get_service_in_cell_by_host Oct 15 20:48:49.920903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi service = _find_service_in_cell(context, service_host=host_name) Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 209, in _remove_host Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi aggregate = self.api.remove_host_from_aggregate(context, id, host) Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6683, in remove_host_from_aggregate Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _get_service_in_cell_by_host(context, host_name) Oct 15 20:48:49.922012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6007, in _get_service_in_cell_by_host Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host_name) Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host nonexist_host could not be found. Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 217, in _remove_host Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Cannot remove host nonexist_host in aggregate 18 Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.923034 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot remove host nonexist_host in aggregate 18 Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Returning 404 to user: Cannot remove host nonexist_host in aggregate 18 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4fc5d409-841d-4fa7-a399-19b8345aa5c2 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/18/action" status: 404 len: 94 microversion: 2.1 time: 0.056949 Oct 15 20:48:49.924121 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 184/320] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:48:49 2022] POST /compute/v2.1/os-aggregates/18/action => generated 94 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:48:49.931854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bbea3fc0-df05-41a9-b3dd-19b7bb233066 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:49.981787 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bbea3fc0-df05-41a9-b3dd-19b7bb233066 tempest-AggregatesAdminNegativeTestJSON-1546140179 tempest-AggregatesAdminNegativeTestJSON-1546140179-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/18" status: 200 len: 0 microversion: 2.1 time: 0.052559 Oct 15 20:48:49.983032 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 137/321] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:48:49 2022] DELETE /compute/v2.1/os-aggregates/18 => generated 0 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:48:50.138463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:50.139078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:50.149098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.152991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.153613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:50.154389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.154885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.155322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:50.278277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.278788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.282992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:50.287078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lazy-loading 'fault' on Instance uuid d6bfd383-07fb-4c4e-9d98-00a96ea7274d {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:48:50.463325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:50.476976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fcae6a32-e16d-4ce3-942b-5ce971ac6e3f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 200 len: 1309 microversion: 2.1 time: 0.344223 Oct 15 20:48:50.481621 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 185/322] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:50 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 1309 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:50.780956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:50.782062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:50.794106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.794106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.794106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:50.794786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.795237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.795660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:50.916207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:50.917152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:50.930638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.931827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.934936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:50.938078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:50.940048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:50.943479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:51.159819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:51.179021 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-42c79095-394b-4b35-8ee7-b08d86883724 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.401747 Oct 15 20:48:51.180258 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 138/323] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:50 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:51.279197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:51.279197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:51.300923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:51.305301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:51.308689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:51.312605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:51.313309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:51.314419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:51.545814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:51.569988 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4bf7b905-8433-44f3-b63a-f889bc2c0a41 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1463 microversion: 2.1 time: 0.654625 Oct 15 20:48:51.573610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 186/324] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:50 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1463 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:51.581757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ac15aa80-cd85-4d59-9d8a-643cad6952bf req-d2b41fee-1428-4ddb-bb5f-d98c517e1ed9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "56f00c4c-5642-40b7-b0a7-62c3ec8b3b56", "tag": "1ed285bf-73a2-45c3-8c63-b59f9d0797cc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:51.596267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ac15aa80-cd85-4d59-9d8a-643cad6952bf req-d2b41fee-1428-4ddb-bb5f-d98c517e1ed9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:51.605218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ac15aa80-cd85-4d59-9d8a-643cad6952bf req-d2b41fee-1428-4ddb-bb5f-d98c517e1ed9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:51.605836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ac15aa80-cd85-4d59-9d8a-643cad6952bf req-d2b41fee-1428-4ddb-bb5f-d98c517e1ed9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:51.652252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ac15aa80-cd85-4d59-9d8a-643cad6952bf req-d2b41fee-1428-4ddb-bb5f-d98c517e1ed9 service nova] Creating event network-changed:1ed285bf-73a2-45c3-8c63-b59f9d0797cc for instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 on nch1-master-1 Oct 15 20:48:51.666370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:51.680373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8c75e890-2ed3-4a15-95f8-c45f04779c49 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 200 len: 1587 microversion: 2.1 time: 0.407131 Oct 15 20:48:51.688160 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 139/325] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:51 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 1587 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:51.706127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:51.707036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:51.715545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:51.716262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:51.717263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:51.718603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:51.719313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:51.727900 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ac15aa80-cd85-4d59-9d8a-643cad6952bf req-d2b41fee-1428-4ddb-bb5f-d98c517e1ed9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.152056 Oct 15 20:48:51.730643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 187/326] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:51.737493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:51.777950 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] HTTP exception thrown: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Request method failure captured: Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d HTTP/1.1 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyGbYOmxMZXrEpFWF7vRs2xfglVYJXAOH1UU0pRObKCbtySTA8XW1DChJ2qnp1sm_nh2knXzbJAg1paYMFrR98Dq3dhA7PvNeIbIqHnv9hqFwykMNxqqs_LUMwz3BqfTGsh1sXFVDDfvHbooX6HCHoHmfh6tryekXE1U5ir8UhNNC4 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: c126854c21954928af16db403a0b8919 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:51.782073 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c126854c21954928af16db403a0b8919"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c126854c21954928af16db403a0b8919"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: c126854c21954928af16db403a0b8919 Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c5a178b6a74140d4910f77d811febc08 Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:48:51.783572 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'd6bfd383-07fb-4c4e-9d98-00a96ea7274d'} Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:51.784372 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:48:51.785687 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:51.788153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Returning 404 to user: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:51.789725 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff7adf5c-6f02-44cb-ae80-15d2cd1ccd1d tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 404 len: 111 microversion: 2.1 time: 0.100295 Oct 15 20:48:51.791217 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 140/327] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:51 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 111 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:51.813384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-1922862821", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}, "os:scheduler_hints": {"same_host": "5d1ddd46-47f6-4948-a897-3b963780991e"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:51.913543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:52.051021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersAdminTestJSON-server-445274494", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:52.139121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:48:52.662386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:52.663934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:52.664879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:52.665644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:52.667272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:52.690264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:52.693189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:52.696199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:52.697671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:52.700644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:52.701909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:52.742904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=83,nova_api:DELETE=12,nova_api:INSERT=5 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:52.773953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Getting quotas for project c126854c21954928af16db403a0b8919. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:52.792354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Getting quotas for user c5a178b6a74140d4910f77d811febc08 and project c126854c21954928af16db403a0b8919. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:52.814361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:52.818271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:52.928853 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ec6f3da-bd0d-41df-9020-c7f27b9b49de tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.884606 Oct 15 20:48:52.932407 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 141/328] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 889 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:52.946398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:52.950300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:52.959006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:52.961193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:52.962042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:52.963237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:52.963843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:52.964532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:53.091223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:48:53.091748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:48:53.092453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:48:53.093187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:48:53.093972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:48:53.114498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=37 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:53.125781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:53.127075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:53.127956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:53.129519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:53.130316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:53.132086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:53.160272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Getting quotas for project 1f0b466076164a309233726db31b0efd. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:48:53.178940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Getting quotas for user c84b11d2d54b43cbac87115e227ef50e and project 1f0b466076164a309233726db31b0efd. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:48:53.239793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:48:53.248517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:48:53.281479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:53.294693 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bef02c82-8e91-46bc-a11b-94d802ccc7e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.354322 Oct 15 20:48:53.295273 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 142/329] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:52 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:53.322338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a5290ca8-84a8-4a33-95e1-ed7a34835adc tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:53.322968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a5290ca8-84a8-4a33-95e1-ed7a34835adc tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:53.416628 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e3aa98e-ed67-49ef-b5f1-e4bb39398fdc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.608546 Oct 15 20:48:53.427979 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 188/330] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:48:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 1621 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:48:53.447976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a36686f-48a3-4d16-9fcc-72fb81dd57e3 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:53.451519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a36686f-48a3-4d16-9fcc-72fb81dd57e3 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:53.453454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a5290ca8-84a8-4a33-95e1-ed7a34835adc tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:53.470580 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a5290ca8-84a8-4a33-95e1-ed7a34835adc tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1219 microversion: 2.1 time: 0.154647 Oct 15 20:48:53.471242 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 143/331] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:53 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1219 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:53.687716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9a36686f-48a3-4d16-9fcc-72fb81dd57e3 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:53.697264 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a36686f-48a3-4d16-9fcc-72fb81dd57e3 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1214 microversion: 2.1 time: 0.258008 Oct 15 20:48:53.701485 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 189/332] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:53 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1214 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:53.843636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-180df9b2-f3d6-4759-b905-a4e3f562df67 req-bfdb82e4-c2c0-4ced-93d2-d70f1b5ec469 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e597eee9-ea21-44d4-8f17-5726355a6114", "tag": "a1988c6d-abca-4574-9f81-2e94f32f51da"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:53.860043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-180df9b2-f3d6-4759-b905-a4e3f562df67 req-bfdb82e4-c2c0-4ced-93d2-d70f1b5ec469 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:53.862987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-180df9b2-f3d6-4759-b905-a4e3f562df67 req-bfdb82e4-c2c0-4ced-93d2-d70f1b5ec469 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:53.864240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-180df9b2-f3d6-4759-b905-a4e3f562df67 req-bfdb82e4-c2c0-4ced-93d2-d70f1b5ec469 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:53.912804 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-180df9b2-f3d6-4759-b905-a4e3f562df67 req-bfdb82e4-c2c0-4ced-93d2-d70f1b5ec469 service nova] Creating event network-changed:a1988c6d-abca-4574-9f81-2e94f32f51da for instance e597eee9-ea21-44d4-8f17-5726355a6114 on nch2-master-1 Oct 15 20:48:53.920399 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-180df9b2-f3d6-4759-b905-a4e3f562df67 req-bfdb82e4-c2c0-4ced-93d2-d70f1b5ec469 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079620 Oct 15 20:48:53.921593 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 144/333] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:48:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:54.320962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:54.322888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:54.336537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:54.337286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:54.337864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:54.339015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:54.340444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:54.346842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:54.411999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:54.509002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:54.512229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:54.538164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:54.541642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:54.543213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:54.548511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:54.550810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:54.552398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:54.766792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:54.783132 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-885d5d43-34e5-40c2-b139-552b2c99def9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1271 microversion: 2.1 time: 0.288033 Oct 15 20:48:54.785406 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 145/334] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:54 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1271 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:54.809744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:54.810429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:54.829376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:54.829876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:54.830315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:54.831031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:54.831489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:54.831914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:54.991498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:55.022986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-25e6fd2a-f358-47e0-896c-8bb48bfde942 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1210 microversion: 2.1 time: 0.231152 Oct 15 20:48:55.026490 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 146/335] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:54 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1210 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:55.216020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:55.217621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:55.218113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:55.218557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:55.219702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:55.220524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:55.221124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:55.233571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=26 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:55.253222 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3ed5ba3c-f49a-4abf-ba49-95824b0a7f31 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] 10.222.0.16 "GET /compute/v2.1/os-availability-zone" status: 200 len: 97 microversion: 2.1 time: 0.225303 Oct 15 20:48:55.254946 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 147/336] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Oct 15 20:48:55 2022] GET /compute/v2.1/os-availability-zone => generated 97 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:48:55.271254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:55.273639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:55.277878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:55.278543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:55.279557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:55.280032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:55.280473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:55.290527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:48:55.313502 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-18d380fa-1065-4f0d-94a8-fc50043c2ab0 tempest-AZAdminV2TestJSON-180068720 tempest-AZAdminV2TestJSON-180068720-project-admin] 10.222.0.16 "GET /compute/v2.1/os-availability-zone/detail" status: 200 len: 613 microversion: 2.1 time: 0.051642 Oct 15 20:48:55.314400 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 148/337] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 20:48:55 2022] GET /compute/v2.1/os-availability-zone/detail => generated 613 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:55.600128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:55.611202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8ab8f4a-589c-4b0a-ad10-1f02c6ab8846 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 1.295264 Oct 15 20:48:55.611784 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 190/338] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:54 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 1297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:55.818993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:55.819635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:55.844477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:55.845212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:55.850085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:55.853586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:55.854376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:55.856307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:56.070243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:56.070848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:56.081902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:56.083272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:56.084885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:56.086339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:56.088046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:56.089345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:56.102120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:56.111780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dcbecab0-1971-4dce-9810-c2ba802eb22f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1277 microversion: 2.1 time: 0.307267 Oct 15 20:48:56.120585 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 149/339] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:55 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1277 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:56.339129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:56.351832 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ec3dd48-9f32-4b1d-95f6-916685c2da27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1272 microversion: 2.1 time: 0.304441 Oct 15 20:48:56.355118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 191/340] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:56 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1272 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:56.634934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:56.635542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:56.642574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:56.643397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:56.644131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:56.645666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:56.646567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:56.647216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:56.820430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-991c058f-5622-408d-a709-8287ea814a96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Calling method '.version_select at 0x7fe7544d1550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:56.823001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-991c058f-5622-408d-a709-8287ea814a96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Policy check for os_compute_api:os-hypervisors:list-detail failed with credentials {'is_admin': False, 'user_id': 'f050455c8e5247cfb77c1f9044efc534', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6c560f9235945a7af176a454b799ea2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-991c058f-5622-408d-a709-8287ea814a96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Request method failure captured: Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors/detail HTTP/1.1 Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy3Ut7gNPxuev3659xGuSNXpc7Nq0k-g4tfN2te0DTaeGnqWN716PgEL4G4uwyksx0bbI6g4GAbLG6OEya-Y5xuM1k8ZdAcqPr1WArhTZ_VGtDBD95Ypva-1JVChGlYO383F6nDlyWARpJnZpGF9odFS6XwaSivxtW32hqny-dj_7Y Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:56.825127 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d6c560f9235945a7af176a454b799ea2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: f050455c8e5247cfb77c1f9044efc534 Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:48:56.826587 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 285, in detail Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._detail(req) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 289, in _detail Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'list-detail', target={}) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:56.827453 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Oct 15 20:48:56.828660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.829916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-991c058f-5622-408d-a709-8287ea814a96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:56.831461 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-991c058f-5622-408d-a709-8287ea814a96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 403 len: 122 microversion: 2.1 time: 0.246977 Oct 15 20:48:56.838184 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 150/341] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 20:48:56 2022] GET /compute/v2.1/os-hypervisors/detail => generated 122 bytes in 254 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:56.846090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9a37f188-82e0-4908-8f0d-1fb469bbaa96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Calling method '.version_select at 0x7fe754654a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:56.848884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9a37f188-82e0-4908-8f0d-1fb469bbaa96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Policy check for os_compute_api:os-hypervisors:list failed with credentials {'is_admin': False, 'user_id': 'f050455c8e5247cfb77c1f9044efc534', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6c560f9235945a7af176a454b799ea2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9a37f188-82e0-4908-8f0d-1fb469bbaa96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Request method failure captured: Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors HTTP/1.1 Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy3Ut7gNPxuev3659xGuSNXpc7Nq0k-g4tfN2te0DTaeGnqWN716PgEL4G4uwyksx0bbI6g4GAbLG6OEya-Y5xuM1k8ZdAcqPr1WArhTZ_VGtDBD95Ypva-1JVChGlYO383F6nDlyWARpJnZpGF9odFS6XwaSivxtW32hqny-dj_7Y Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:56.851154 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d6c560f9235945a7af176a454b799ea2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: f050455c8e5247cfb77c1f9044efc534 Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:48:56.852597 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 253, in index Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._index(req) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 257, in _index Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'list', target={}) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:56.853529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Oct 15 20:48:56.854680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.856753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9a37f188-82e0-4908-8f0d-1fb469bbaa96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:56.858096 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9a37f188-82e0-4908-8f0d-1fb469bbaa96 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 403 len: 115 microversion: 2.1 time: 0.018626 Oct 15 20:48:56.858882 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 151/342] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:56 2022] GET /compute/v2.1/os-hypervisors => generated 115 bytes in 20 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:56.869976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6606f20-d067-4cbc-9805-6ab2b6364b70 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Calling method '.version_select at 0x7fe7542c3550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:56.874978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6606f20-d067-4cbc-9805-6ab2b6364b70 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Policy check for os_compute_api:os-hypervisors:statistics failed with credentials {'is_admin': False, 'user_id': 'f050455c8e5247cfb77c1f9044efc534', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6c560f9235945a7af176a454b799ea2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6606f20-d067-4cbc-9805-6ab2b6364b70 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Request method failure captured: Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors/statistics HTTP/1.1 Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy3Ut7gNPxuev3659xGuSNXpc7Nq0k-g4tfN2te0DTaeGnqWN716PgEL4G4uwyksx0bbI6g4GAbLG6OEya-Y5xuM1k8ZdAcqPr1WArhTZ_VGtDBD95Ypva-1JVChGlYO383F6nDlyWARpJnZpGF9odFS6XwaSivxtW32hqny-dj_7Y Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:56.877565 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d6c560f9235945a7af176a454b799ea2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: f050455c8e5247cfb77c1f9044efc534 Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:48:56.878771 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 521, in statistics Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'statistics', target={}) Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:56.880077 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Oct 15 20:48:56.881203 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:56.883944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6606f20-d067-4cbc-9805-6ab2b6364b70 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:56.885121 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c6606f20-d067-4cbc-9805-6ab2b6364b70 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/statistics" status: 403 len: 121 microversion: 2.1 time: 0.019472 Oct 15 20:48:56.886311 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 152/343] 10.222.0.16 () {58 vars in 1179 bytes} [Sat Oct 15 20:48:56 2022] GET /compute/v2.1/os-hypervisors/statistics => generated 121 bytes in 21 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:57.114457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:57.125886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4236d4ee-e505-4db2-bd63-4839a4a92469 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.494482 Oct 15 20:48:57.141954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 192/344] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:56 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:57.149531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.150803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:57.161156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.162136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.162860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.165354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.167050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.168607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.229069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Calling method '.version_select at 0x7fe7544d1d30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.231201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.231901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.232650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.255235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.255990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.256540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.271930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.272364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.272873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.279528 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-653cd6cf-70ee-4495-84d0-41af73486866 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.388169 Oct 15 20:48:57.280492 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 153/345] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:56 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:57.293092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4dfe403c-c074-4007-a7dc-41f215e8ac99 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Calling method '.version_select at 0x7fe7544d1a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.295390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:57.297853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4dfe403c-c074-4007-a7dc-41f215e8ac99 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Policy check for os_compute_api:os-hypervisors:uptime failed with credentials {'is_admin': False, 'user_id': 'f050455c8e5247cfb77c1f9044efc534', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6c560f9235945a7af176a454b799ea2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4dfe403c-c074-4007-a7dc-41f215e8ac99 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Request method failure captured: Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors/1/uptime HTTP/1.1 Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy3Ut7gNPxuev3659xGuSNXpc7Nq0k-g4tfN2te0DTaeGnqWN716PgEL4G4uwyksx0bbI6g4GAbLG6OEya-Y5xuM1k8ZdAcqPr1WArhTZ_VGtDBD95Ypva-1JVChGlYO383F6nDlyWARpJnZpGF9odFS6XwaSivxtW32hqny-dj_7Y Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:48:57.300953 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d6c560f9235945a7af176a454b799ea2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: f050455c8e5247cfb77c1f9044efc534 Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '1'} Oct 15 20:48:57.302414 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 387, in uptime Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'uptime', target={}) Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:57.303116 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Oct 15 20:48:57.304189 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.306681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4dfe403c-c074-4007-a7dc-41f215e8ac99 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:57.307773 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4dfe403c-c074-4007-a7dc-41f215e8ac99 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1/uptime" status: 403 len: 117 microversion: 2.1 time: 0.020401 Oct 15 20:48:57.310181 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 154/346] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/os-hypervisors/1/uptime => generated 117 bytes in 22 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:57.315349 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20443831-3d19-4442-99fd-82d8a1d4a72a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1277 microversion: 2.1 time: 0.167289 Oct 15 20:48:57.315349 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 193/347] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1277 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:57.321114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09433d5f-407b-4584-a959-b94768866db3 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Calling method '.version_select at 0x7fe754256ee0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.322406 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-09433d5f-407b-4584-a959-b94768866db3 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] HTTP exception thrown: Hypervisor with ID 'a294eae2-604e-4899-b9bb-2128233f8bab' could not be found. Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09433d5f-407b-4584-a959-b94768866db3 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Request method failure captured: Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hypervisors/a294eae2-604e-4899-b9bb-2128233f8bab/uptime HTTP/1.1 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy4obt4QrKlfw1ROOpWn8iE8q6FhxuZ5Uuu-RO09OXvqOAmAbiaxBD6Rc2jv1jGUvJT7mKZIRH2eS8xG0HddpnhwCf2HwVZ56EXWN7WD21x1PScAdpmhoY17HAftyC5HzOlKsMTzKC5qgrbQ-xzw6cG9xxKr4Ot9nOb3adosYXpFW4 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 5a194c610fdf40dd849fc78fc4aa07f2 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1877169935 Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:48:57.332819 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1877169935 Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 5a194c610fdf40dd849fc78fc4aa07f2 Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1877169935 Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c123f19c891d4ad6b8844d785c2c06d3 Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: exception: Hypervisor with ID 'a294eae2-604e-4899-b9bb-2128233f8bab' could not be found. Oct 15 20:48:57.334087 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'a294eae2-604e-4899-b9bb-2128233f8bab'} Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 501, in validate_integer Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi value = int(str(value)) Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ValueError: invalid literal for int() with base 10: 'a294eae2-604e-4899-b9bb-2128233f8bab' Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 614, in validate_integer Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return strutils.validate_integer(value, name, min_value, max_value) Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 505, in validate_integer Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise ValueError(msg) Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ValueError: id must be an integer Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 313, in _validate_id Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi utils.validate_integer(hypervisor_id, 'id') Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 616, in validate_integer Oct 15 20:48:57.334873 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InvalidInput(reason=str(e)) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: id must be an integer Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 389, in uptime Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._validate_id(req, id) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 317, in _validate_id Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Hypervisor with ID 'a294eae2-604e-4899-b9bb-2128233f8bab' could not be found. Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.338215 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Hypervisor with ID 'a294eae2-604e-4899-b9bb-2128233f8bab' could not be found. Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09433d5f-407b-4584-a959-b94768866db3 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Returning 404 to user: Hypervisor with ID 'a294eae2-604e-4899-b9bb-2128233f8bab' could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-09433d5f-407b-4584-a959-b94768866db3 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/a294eae2-604e-4899-b9bb-2128233f8bab/uptime" status: 404 len: 123 microversion: 2.1 time: 0.018211 Oct 15 20:48:57.339404 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 155/348] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/os-hypervisors/a294eae2-604e-4899-b9bb-2128233f8bab/uptime => generated 123 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:48:57.350659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Calling method '.version_select at 0x7fe7544d3430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.352532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.353029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.353502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.372961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.386895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.387457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.387650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.389208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:57.402054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.403677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.404435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.405750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.406313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.407551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.408508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:57.409235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:57.410038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:57.414559 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43b0b74a-7f61-4f3c-b262-aee7ff663113 tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.069296 Oct 15 20:48:57.418122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 194/349] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:48:57.427316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d467b879-7d58-4ba1-a003-346c0b895f45 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Calling method '.version_select at 0x7fe754455790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.430569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d467b879-7d58-4ba1-a003-346c0b895f45 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Policy check for os_compute_api:os-hypervisors:show failed with credentials {'is_admin': False, 'user_id': 'f050455c8e5247cfb77c1f9044efc534', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6c560f9235945a7af176a454b799ea2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d467b879-7d58-4ba1-a003-346c0b895f45 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Request method failure captured: Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-hypervisors/1 HTTP/1.1 Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxy3Ut7gNPxuev3659xGuSNXpc7Nq0k-g4tfN2te0DTaeGnqWN716PgEL4G4uwyksx0bbI6g4GAbLG6OEya-Y5xuM1k8ZdAcqPr1WArhTZ_VGtDBD95Ypva-1JVChGlYO383F6nDlyWARpJnZpGF9odFS6XwaSivxtW32hqny-dj_7Y Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:48:57.433469 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d6c560f9235945a7af176a454b799ea2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d6c560f9235945a7af176a454b799ea2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d6c560f9235945a7af176a454b799ea2 Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227 Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f050455c8e5247cfb77c1f9044efc534 Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1010492227-project Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1'} Oct 15 20:48:57.434638 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 337, in show Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._show(req, id) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 341, in _show Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'show', target={}) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:48:57.435344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Oct 15 20:48:57.436320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.438101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d467b879-7d58-4ba1-a003-346c0b895f45 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:57.439216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d467b879-7d58-4ba1-a003-346c0b895f45 tempest-HypervisorAdminNegativeTestJSON-1010492227 tempest-HypervisorAdminNegativeTestJSON-1010492227-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 403 len: 115 microversion: 2.1 time: 0.016651 Oct 15 20:48:57.439981 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 195/350] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/os-hypervisors/1 => generated 115 bytes in 18 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:48:57.450640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12de19b7-1575-4bc9-ad0e-0ba70be8b54a tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Calling method '.version_select at 0x7fe7545924c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:57.452995 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-12de19b7-1575-4bc9-ad0e-0ba70be8b54a tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] HTTP exception thrown: Hypervisor with ID '3117b4f2-482b-4786-b8ba-7f3043f14c32' could not be found. Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12de19b7-1575-4bc9-ad0e-0ba70be8b54a tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Request method failure captured: Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-hypervisors/3117b4f2-482b-4786-b8ba-7f3043f14c32 HTTP/1.1 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxy4obt4QrKlfw1ROOpWn8iE8q6FhxuZ5Uuu-RO09OXvqOAmAbiaxBD6Rc2jv1jGUvJT7mKZIRH2eS8xG0HddpnhwCf2HwVZ56EXWN7WD21x1PScAdpmhoY17HAftyC5HzOlKsMTzKC5qgrbQ-xzw6cG9xxKr4Ot9nOb3adosYXpFW4 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 5a194c610fdf40dd849fc78fc4aa07f2 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1877169935 Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:48:57.455686 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5a194c610fdf40dd849fc78fc4aa07f2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1877169935 Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 5a194c610fdf40dd849fc78fc4aa07f2 Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1877169935 Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c123f19c891d4ad6b8844d785c2c06d3 Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: exception: Hypervisor with ID '3117b4f2-482b-4786-b8ba-7f3043f14c32' could not be found. Oct 15 20:48:57.457173 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3117b4f2-482b-4786-b8ba-7f3043f14c32'} Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 501, in validate_integer Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi value = int(str(value)) Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ValueError: invalid literal for int() with base 10: '3117b4f2-482b-4786-b8ba-7f3043f14c32' Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 614, in validate_integer Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return strutils.validate_integer(value, name, min_value, max_value) Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 505, in validate_integer Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise ValueError(msg) Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ValueError: id must be an integer Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 313, in _validate_id Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi utils.validate_integer(hypervisor_id, 'id') Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 616, in validate_integer Oct 15 20:48:57.457831 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidInput(reason=str(e)) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: id must be an integer Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 337, in show Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._show(req, id) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 343, in _show Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._validate_id(req, id) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 317, in _validate_id Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:48:57.459075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Hypervisor with ID '3117b4f2-482b-4786-b8ba-7f3043f14c32' could not be found. Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Hypervisor with ID '3117b4f2-482b-4786-b8ba-7f3043f14c32' could not be found. Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12de19b7-1575-4bc9-ad0e-0ba70be8b54a tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] Returning 404 to user: Hypervisor with ID '3117b4f2-482b-4786-b8ba-7f3043f14c32' could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:48:57.460105 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-12de19b7-1575-4bc9-ad0e-0ba70be8b54a tempest-HypervisorAdminNegativeTestJSON-1877169935 tempest-HypervisorAdminNegativeTestJSON-1877169935-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/3117b4f2-482b-4786-b8ba-7f3043f14c32" status: 404 len: 123 microversion: 2.1 time: 0.012625 Oct 15 20:48:57.464457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 196/351] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/os-hypervisors/3117b4f2-482b-4786-b8ba-7f3043f14c32 => generated 123 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:48:57.518921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:57.529112 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-094bd2c1-6cfe-4d3a-aeaa-25fe6a8cb407 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1272 microversion: 2.1 time: 0.153214 Oct 15 20:48:57.530321 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 156/352] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:57 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1272 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:58.153950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:58.155601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:58.164951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:58.166450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:58.168139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:58.169787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:58.171160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:58.172254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:58.336172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:58.337721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:58.347453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:58.351666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:58.352753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:58.353832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:58.354686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:58.356367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:58.555725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:58.570509 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f0b58a02-ec25-47e5-bb39-87e367ae8a6f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.420451 Oct 15 20:48:58.571037 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 197/353] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:58 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:58.580433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:58.581069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:58.605081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:58.614403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:58.614860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:58.615615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:58.616078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:58.616512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:59.336702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:59.345575 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ea292c7-be55-4c07-a33c-dca631fdebb9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 1.013864 Oct 15 20:48:59.346166 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 157/354] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:58 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 1015 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:59.415315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:48:59.427279 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40efb7ca-60a1-46d8-b9c3-f8b7e2fe5561 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.852859 Oct 15 20:48:59.441950 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 198/355] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:58 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 869 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:48:59.599763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:48:59.600535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:48:59.609337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:59.609993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:59.612090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:59.616366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:48:59.618129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:48:59.618798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:48:59.726124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3314c5fb-d645-454c-8814-45d0c4385686 tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_validate_metadata_-1298493026", "id": 976883759, "os-flavor-access:is_public": "True"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:59.756077 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3314c5fb-d645-454c-8814-45d0c4385686 tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 468 microversion: 2.86 time: 0.381835 Oct 15 20:48:59.757686 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 158/356] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:48:59 2022] POST /compute/v2.1/flavors => generated 468 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:48:59.771677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5584424a-e03d-4b49-bb6b-ab46eb00c4ce tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:cpu_policy": "shared", "foo:bar": "baz"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:48:59.981351 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5584424a-e03d-4b49-bb6b-ab46eb00c4ce tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/976883759/os-extra_specs" status: 200 len: 62 microversion: 2.86 time: 0.213301 Oct 15 20:48:59.987050 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 159/357] 10.222.0.16 () {62 vars in 1262 bytes} [Sat Oct 15 20:48:59 2022] POST /compute/v2.1/flavors/976883759/os-extra_specs => generated 62 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:48:59.997654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f9b7b2b-26f8-4687-a9bc-36498e928c22 tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:00.026160 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f9b7b2b-26f8-4687-a9bc-36498e928c22 tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/976883759" status: 202 len: 0 microversion: 2.86 time: 0.035866 Oct 15 20:49:00.037086 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 160/358] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Oct 15 20:48:59 2022] DELETE /compute/v2.1/flavors/976883759 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 15 20:49:00.049293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-656846a3-8c7f-403f-9dca-88c50776f37b tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:00.099158 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-656846a3-8c7f-403f-9dca-88c50776f37b tempest-FlavorMetadataValidation-1707083089 tempest-FlavorMetadataValidation-1707083089-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6679 microversion: 2.86 time: 0.057314 Oct 15 20:49:00.100820 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 161/359] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 20:49:00 2022] GET /compute/v2.1/flavors/detail => generated 6679 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:49:00.239738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:00.265991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cab5030a-a835-42f2-bc25-124feca2a596 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.668627 Oct 15 20:49:00.267727 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 199/360] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:48:59 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:00.370717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:00.371888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:00.396321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:00.398866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:00.400676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:00.401489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:00.403550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:00.403933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:00.454569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:00.455181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:00.467692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:00.468200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:00.468638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:00.470687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:00.473273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:00.474088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:00.996047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:01.004264 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4952e11d-3b63-4dcc-b950-b282faf69038 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.638801 Oct 15 20:49:01.009093 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 162/361] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:00 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:01.290941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b502b919-f66b-4fc4-9636-e7327ffd305d tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsV255TestJSON-flavor-138616038", "id": 1053681731, "os-flavor-access:is_public": "True"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:01.344596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b502b919-f66b-4fc4-9636-e7327ffd305d tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 448 microversion: 2.55 time: 0.262477 Oct 15 20:49:01.346949 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 163/362] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:49:01 2022] POST /compute/v2.1/flavors => generated 448 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:01.364798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:01.365462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:01.377309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:01.378791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:01.381171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:01.382081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:01.383068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:01.383672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:01.414109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:01.427616 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a588632f-e093-4197-88eb-4dba5ec62f30 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.973564 Oct 15 20:49:01.442224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 200/363] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:00 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 992 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:01.664826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0406f69-91b4-4f04-842c-7ad9d85d0a02 tempest-FlavorsV255TestJSON-2117545638 tempest-FlavorsV255TestJSON-2117545638-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:01.680975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0406f69-91b4-4f04-842c-7ad9d85d0a02 tempest-FlavorsV255TestJSON-2117545638 tempest-FlavorsV255TestJSON-2117545638-project] 10.222.0.16 "GET /compute/v2.1/flavors/1053681731" status: 200 len: 448 microversion: 2.55 time: 0.236989 Oct 15 20:49:01.682085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 201/364] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Oct 15 20:49:01 2022] GET /compute/v2.1/flavors/1053681731 => generated 448 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:01.684046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=7,nova_api:DELETE=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:01.688534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1bb77255-599f-4567-90c7-05debe469958 req-a24ba836-0f6a-4980-ae2a-453335e3a97c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ad32c860-892c-4d29-989d-86aebcce74bd", "tag": "17bb0738-b329-473c-b57b-c47dd0a496d4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:01.703220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1bb77255-599f-4567-90c7-05debe469958 req-a24ba836-0f6a-4980-ae2a-453335e3a97c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:01.704104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1bb77255-599f-4567-90c7-05debe469958 req-a24ba836-0f6a-4980-ae2a-453335e3a97c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:01.704839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1bb77255-599f-4567-90c7-05debe469958 req-a24ba836-0f6a-4980-ae2a-453335e3a97c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:01.724596 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1bb77255-599f-4567-90c7-05debe469958 req-a24ba836-0f6a-4980-ae2a-453335e3a97c service nova] Creating event network-changed:17bb0738-b329-473c-b57b-c47dd0a496d4 for instance ad32c860-892c-4d29-989d-86aebcce74bd on nch2-master-1 Oct 15 20:49:01.742383 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1bb77255-599f-4567-90c7-05debe469958 req-a24ba836-0f6a-4980-ae2a-453335e3a97c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058775 Oct 15 20:49:01.743993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:01.745149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 202/365] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:01.751739 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-671bccd7-f6c3-47ee-b362-8aec6e7f47b1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.402380 Oct 15 20:49:01.757473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6dedcb84-d1d0-479a-b5cf-ffa1337da292 tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] Action: 'update', calling method: , body: {"flavor": {"description": "new"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:01.757771 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 164/366] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:01 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:01.793287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6dedcb84-d1d0-479a-b5cf-ffa1337da292 tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/1053681731" status: 200 len: 449 microversion: 2.55 time: 0.043666 Oct 15 20:49:01.794507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 203/367] 10.222.0.16 () {62 vars in 1219 bytes} [Sat Oct 15 20:49:01 2022] PUT /compute/v2.1/flavors/1053681731 => generated 449 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:01.807275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-73bab9d4-5c99-4e19-8b18-0a86816675b1 tempest-FlavorsV255TestJSON-2117545638 tempest-FlavorsV255TestJSON-2117545638-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:01.845537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-73bab9d4-5c99-4e19-8b18-0a86816675b1 tempest-FlavorsV255TestJSON-2117545638 tempest-FlavorsV255TestJSON-2117545638-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6590 microversion: 2.55 time: 0.041557 Oct 15 20:49:01.847432 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 165/368] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 20:49:01 2022] GET /compute/v2.1/flavors/detail => generated 6590 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:01.868082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1e59b0fa-4317-4d9e-aea3-3585a8b3895e tempest-FlavorsV255TestJSON-2117545638 tempest-FlavorsV255TestJSON-2117545638-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:01.899730 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1e59b0fa-4317-4d9e-aea3-3585a8b3895e tempest-FlavorsV255TestJSON-2117545638 tempest-FlavorsV255TestJSON-2117545638-project] 10.222.0.16 "GET /compute/v2.1/flavors" status: 200 len: 3680 microversion: 2.55 time: 0.035421 Oct 15 20:49:01.909730 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 204/369] 10.222.0.16 () {60 vars in 1166 bytes} [Sat Oct 15 20:49:01 2022] GET /compute/v2.1/flavors => generated 3680 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:01.920964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2668cd10-2099-4a41-a7dc-1ce0e9b76c6a tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:01.949554 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2668cd10-2099-4a41-a7dc-1ce0e9b76c6a tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1053681731" status: 202 len: 0 microversion: 2.55 time: 0.033485 Oct 15 20:49:01.959741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e669293-d309-4332-bc53-a549e2d68964 tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:01.964246 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 166/370] 10.222.0.16 () {60 vars in 1202 bytes} [Sat Oct 15 20:49:01 2022] DELETE /compute/v2.1/flavors/1053681731 => generated 0 bytes in 48 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:49:02.011523 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e669293-d309-4332-bc53-a549e2d68964 tempest-FlavorsV255TestJSON-89487059 tempest-FlavorsV255TestJSON-89487059-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6151 microversion: 2.55 time: 0.049164 Oct 15 20:49:02.014211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 205/371] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 20:49:01 2022] GET /compute/v2.1/flavors/detail => generated 6151 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:02.040391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:02.041904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:02.061407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:02.062311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:02.063198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:02.064171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:02.065766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:02.066975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:02.423660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:02.442216 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5c9e89d0-3921-4305-a305-5ca95e409577 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.407662 Oct 15 20:49:02.458361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:02.459490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:02.460203 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 167/372] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:02 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:02.489797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:02.491134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:02.491589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:02.493978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:02.499977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:02.501346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:02.781392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:02.782885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:02.799405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:02.800193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:02.801694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:02.802662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:02.803377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:02.804024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:03.056705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:03.066261 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da360a22-d98e-4728-bbea-b8d6329de2b0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.289017 Oct 15 20:49:03.082797 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 168/373] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:02 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:03.147633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=24 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.247906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesNegativeTestJSON-server-484072549", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "3b9b8e16-0d9b-40c0-9997-20324bea9193"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:03.384857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] validate_networks() for [('3b9b8e16-0d9b-40c0-9997-20324bea9193', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:49:03.433574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:03.447949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-672d64f8-cc5b-4885-8e58-0feb1fbed57d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.991293 Oct 15 20:49:03.449422 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 206/374] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:02 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 996 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:03.462745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=52 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.472089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:03.474081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:03.480862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:03.493607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:03.494095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:03.494870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:03.495349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:03.495785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:03.696610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:03.750215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:03.762295 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-892546d1-5de7-49a6-8ff5-65b898a52c07 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.294647 Oct 15 20:49:03.763466 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 207/375] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:03 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:03.812188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-53e322a8-d46b-4bd7-b0b8-3dcb78300ce0 req-5f36c1c9-2e5a-464c-a7e1-77c377d0f32a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "683f8544-7601-473f-b81f-c5bf78f0c781", "tag": "c833f037-3f48-49a5-9761-1581e6331daf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:03.823114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-53e322a8-d46b-4bd7-b0b8-3dcb78300ce0 req-5f36c1c9-2e5a-464c-a7e1-77c377d0f32a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:03.823841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-53e322a8-d46b-4bd7-b0b8-3dcb78300ce0 req-5f36c1c9-2e5a-464c-a7e1-77c377d0f32a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:03.824463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-53e322a8-d46b-4bd7-b0b8-3dcb78300ce0 req-5f36c1c9-2e5a-464c-a7e1-77c377d0f32a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:03.837901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-53e322a8-d46b-4bd7-b0b8-3dcb78300ce0 req-5f36c1c9-2e5a-464c-a7e1-77c377d0f32a service nova] Creating event network-changed:c833f037-3f48-49a5-9761-1581e6331daf for instance 683f8544-7601-473f-b81f-c5bf78f0c781 on nch2-master-1 Oct 15 20:49:03.844544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-53e322a8-d46b-4bd7-b0b8-3dcb78300ce0 req-5f36c1c9-2e5a-464c-a7e1-77c377d0f32a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036108 Oct 15 20:49:03.845731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 208/376] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:04.022182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:49:04.023544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:49:04.024457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:49:04.025258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:49:04.026294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:49:04.048440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.049229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.051558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.052977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.054093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.055588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.089899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:04.090633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:04.102897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.105761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.108038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Getting quotas for project 86408ec8ca9f4e33a16bdf9a23aa9faa. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:49:04.109504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.112256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.114817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.115514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.119730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Getting quotas for user 507faa1518b34884b3e095bfae431bc2 and project 86408ec8ca9f4e33a16bdf9a23aa9faa. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:49:04.154093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:49:04.164786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:49:04.202337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=7,nova_api:SAVEPOINT=2,nova_api:RELEASE=2,nova_api:DELETE=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:04.292330 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fa14ce0b-e7dd-466b-ba12-d001fb89af34 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.202782 Oct 15 20:49:04.304015 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 169/377] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1216 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:49:04.336368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a4574b64-9902-4c7e-a453-e0b85431e016 req-e3f6f7ab-196d-4cb1-8ca8-7e945829942e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e597eee9-ea21-44d4-8f17-5726355a6114", "name": "network-vif-plugged", "status": "completed", "tag": "a1988c6d-abca-4574-9f81-2e94f32f51da"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:04.367480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a4574b64-9902-4c7e-a453-e0b85431e016 req-e3f6f7ab-196d-4cb1-8ca8-7e945829942e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.368164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a4574b64-9902-4c7e-a453-e0b85431e016 req-e3f6f7ab-196d-4cb1-8ca8-7e945829942e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.368607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a4574b64-9902-4c7e-a453-e0b85431e016 req-e3f6f7ab-196d-4cb1-8ca8-7e945829942e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.382965 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a4574b64-9902-4c7e-a453-e0b85431e016 req-e3f6f7ab-196d-4cb1-8ca8-7e945829942e service nova] Creating event network-vif-plugged:a1988c6d-abca-4574-9f81-2e94f32f51da for instance e597eee9-ea21-44d4-8f17-5726355a6114 on nch2-master-1 Oct 15 20:49:04.389645 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a4574b64-9902-4c7e-a453-e0b85431e016 req-e3f6f7ab-196d-4cb1-8ca8-7e945829942e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083004 Oct 15 20:49:04.390331 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 170/378] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:04.398271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40c69a3a-2fb8-42c1-a8bf-79b338b904e9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:04.399037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-40c69a3a-2fb8-42c1-a8bf-79b338b904e9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:04.506584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-40c69a3a-2fb8-42c1-a8bf-79b338b904e9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:04.531882 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-40c69a3a-2fb8-42c1-a8bf-79b338b904e9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1215 microversion: 2.1 time: 0.137114 Oct 15 20:49:04.533631 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 171/379] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:04 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1215 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:04.545830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:04.548073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:04.564525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.566842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.569329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.570609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.571903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.573182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.702306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:04.711775 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2132300b-b30c-4dbc-a7d8-7eb7a21f23c9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.627224 Oct 15 20:49:04.727363 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 209/380] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:04 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:04.795751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:04.796703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:04.823464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.825887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.829190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:04.835329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:04.838247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:04.838928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:05.135528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:05.146959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3101c993-07b4-41f6-995b-51a0f6b13c0f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.360917 Oct 15 20:49:05.165107 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 210/381] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:04 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:05.314511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:05.332677 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-32c6be02-93e2-43d1-aa35-02aeea4f9fb6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.797696 Oct 15 20:49:05.337803 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 172/382] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:04 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 803 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:05.561002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:05.561982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:05.578533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:05.580469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:05.581501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:05.584762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:05.586643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:05.588640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:05.741031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:05.741679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:05.753954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:05.754849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:05.757901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:05.759166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:05.761786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:05.764329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:05.820850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:05.835124 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3954bd69-2e4e-41f9-89c7-e154c93f1679 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1267 microversion: 2.1 time: 0.278966 Oct 15 20:49:05.841982 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 211/383] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:05 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1267 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:06.181465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:06.184750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:06.222514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:06.223874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:06.224609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:06.231098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:06.234313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:06.234984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:06.428498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:06.447335 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4c8b05db-3c00-4f94-842b-b5a4423e7845 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1312 microversion: 2.1 time: 0.711312 Oct 15 20:49:06.449719 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 173/384] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:05 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1312 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:06.461100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:06.461729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:06.490562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:06.490984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:06.491486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:06.492228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:06.492579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:06.493012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:07.114732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:07.128761 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5808ce8e-8b31-4ca8-96ea-91911f4ca4c5 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.677831 Oct 15 20:49:07.130663 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 174/385] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:06 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:07.150559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:07.151445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:07.165180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:07.174687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:07.177948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:07.182480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:07.183163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:07.183745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:07.774018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:07.788271 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6cfe3a6e-8736-4f2c-8c14-9373c52afa2b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1273 microversion: 2.1 time: 0.656354 Oct 15 20:49:07.799822 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 175/386] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:07 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1273 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:07.817261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:07.818343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:07.836540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:07.837365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:07.838130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:07.839549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:07.840185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:07.840923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:07.906379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:07.924533 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c25c5d6d-cb30-4e6e-9937-145355dbee4a tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 1.754340 Oct 15 20:49:07.926308 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 212/387] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:06 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 1758 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:08.556630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Calling method '.version_select at 0x7fe7543b90d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:08.560670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:08.562783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:08.563766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:08.565731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:08.567815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:08.568353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:08.610924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:08.623500 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c7e47b0-c140-4afd-8f25-f4b85eea95be tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1462 microversion: 2.1 time: 0.820429 Oct 15 20:49:08.625781 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 176/388] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:07 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1462 bytes in 825 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:08.636957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:08.637981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:08.646266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:08.648162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:08.648728 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c27c0d6d-eedb-4449-ad93-6ad5a1d7cb22 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.626271 Oct 15 20:49:08.649330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:08.650757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:08.651650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:08.655823 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 213/389] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 20:49:08 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:49:08.665442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:08.753725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46074663-e255-4ecc-9d78-87438e58867b tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:08.755223 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-46074663-e255-4ecc-9d78-87438e58867b tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin] HTTP exception thrown: Invalid timestamp for date invalid_time Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46074663-e255-4ecc-9d78-87438e58867b tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin] Request method failure captured: Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-instance_usage_audit_log/invalid_time HTTP/1.1 Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzEo4r4KxHWTMgFiXezPCqCPKH3A-TgQYG4zxu_uG5clKwSyn1jC66vlJwAJCtuOICWr-qEU9ajiTQW0Wm3MR66UBAoTBJlDce4nIzeJ6e3FhSpj8cYwxBmfcYrlEIQRvZLmy4IFOGfgNeqfbI5FSmCr9mnLYwBUIHzwi0ZeucQ85M Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2aa40e24e7b7418eac62e7e71bee0b69 Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:49:08.774767 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2aa40e24e7b7418eac62e7e71bee0b69"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2aa40e24e7b7418eac62e7e71bee0b69"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2aa40e24e7b7418eac62e7e71bee0b69"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2aa40e24e7b7418eac62e7e71bee0b69"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2aa40e24e7b7418eac62e7e71bee0b69 Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d798e3c137b8489fb4c0a66dcaa9fff6 Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:08.775990 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid timestamp for date invalid_time Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'invalid_time'} Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 51, in show Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi before_date = datetime.datetime.strptime(str(id), Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/_strptime.py", line 568, in _strptime_datetime Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi tt, fraction, gmtoff_fraction = _strptime(data_string, format) Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/_strptime.py", line 349, in _strptime Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise ValueError("time data %r does not match format %r" % Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ValueError: time data 'invalid_time' does not match format '%Y-%m-%d %H:%M:%S' Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:08.776614 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 55, in show Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=msg) Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid timestamp for date invalid_time Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid timestamp for date invalid_time Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46074663-e255-4ecc-9d78-87438e58867b tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin] Returning 400 to user: Invalid timestamp for date invalid_time {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:08.777874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46074663-e255-4ecc-9d78-87438e58867b tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848 tempest-InstanceUsageAuditLogNegativeTestJSON-2104578848-project-admin] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log/invalid_time" status: 400 len: 83 microversion: 2.1 time: 0.119495 Oct 15 20:49:08.778667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 214/390] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Oct 15 20:49:08 2022] GET /compute/v2.1/os-instance_usage_audit_log/invalid_time => generated 83 bytes in 121 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:49:08.943519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:08.944142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:08.953515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:08.954094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:08.954576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:08.957208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:08.958121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:08.958758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:09.045879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:09.059582 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a59e7445-e742-4550-a3f4-47e8cfe5b78a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.431019 Oct 15 20:49:09.060639 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 177/391] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:08 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:09.218410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9ec85c7f-767e-4015-852b-83633614f73b tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Calling method '.version_select at 0x7fe7542c31f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:09.241009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9ec85c7f-767e-4015-852b-83633614f73b tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Policy check for os_compute_api:os-hosts:list failed with credentials {'is_admin': False, 'user_id': 'a62d07846dda4594a29a627c97cd236c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22a57f99fc634af4b37a892143bf6665', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9ec85c7f-767e-4015-852b-83633614f73b tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Request method failure captured: Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts HTTP/1.1 Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzCHiZeWwiUsfz8C1o9sy79hWmrL6Uxgzz6_NZekSjRyGUzM1K471uBzvXEyNLfac-1kEUYsWeCNCPf1oPHOo7W6FUVDrpdTy61CfmkjFsU3La5TyMkANuph66tyOBu2Bqv2T0y3Na9_ZnV11LEPOtuqxgj-MkIm4IXAGP6eCY0LPc Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:09.247022 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22a57f99fc634af4b37a892143bf6665"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a62d07846dda4594a29a627c97cd236c Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:49:09.248568 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 74, in index Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'list', Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:09.249433 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Oct 15 20:49:09.250584 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:09.252931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9ec85c7f-767e-4015-852b-83633614f73b tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:list to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:09.255745 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9ec85c7f-767e-4015-852b-83633614f73b tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 403 len: 109 microversion: 2.1 time: 0.193684 Oct 15 20:49:09.261485 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 178/392] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 20:49:09 2022] GET /compute/v2.1/os-hosts => generated 109 bytes in 200 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:09.465181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-124b79b4-a259-4a62-97aa-7565123adaf7 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:09.467117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-124b79b4-a259-4a62-97aa-7565123adaf7 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Policy check for os_compute_api:os-instance-usage-audit-log:list failed with credentials {'is_admin': False, 'user_id': 'bd2aa13e39d644319136fd102c55cb95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89ed78bfa690487c816835dd282e2970', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-124b79b4-a259-4a62-97aa-7565123adaf7 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Request method failure captured: Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-instance_usage_audit_log HTTP/1.1 Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzC2Y5mbzDJu4irVJ0Lmh4auOPB6WiLxvIK_fLM8wurwLvJzhylqMLuWdCB6fsw1U_RudBgR7spnHIe7RrZRbQNeD3NXG66HY0QIj7VoozIkjNXRD4Mk_CQmLGKWRFxrBf--okw_c0-oXzfMl-FFAFww90_YmxWWm_ZIZM-Fnz40Ek Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 89ed78bfa690487c816835dd282e2970 Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:09.470101 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89ed78bfa690487c816835dd282e2970"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89ed78bfa690487c816835dd282e2970"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89ed78bfa690487c816835dd282e2970"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89ed78bfa690487c816835dd282e2970"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 89ed78bfa690487c816835dd282e2970 Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: bd2aa13e39d644319136fd102c55cb95 Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project Oct 15 20:49:09.471311 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 38, in index Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(iual_policies.BASE_POLICY_NAME % 'list', target={}) Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:09.472529 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Oct 15 20:49:09.475851 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:09.477820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-124b79b4-a259-4a62-97aa-7565123adaf7 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:09.477820 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-124b79b4-a259-4a62-97aa-7565123adaf7 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log" status: 403 len: 128 microversion: 2.1 time: 0.208830 Oct 15 20:49:09.477820 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 179/393] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 20:49:09 2022] GET /compute/v2.1/os-instance_usage_audit_log => generated 128 bytes in 212 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:09.492077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:09.492695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:09.499821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:09.500486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:09.501267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:09.502175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:09.502767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:09.504085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:09.606538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-28245a7d-1fdd-4206-b0e7-d3916cf23b0b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1613 microversion: 2.1 time: 0.827371 Oct 15 20:49:09.621482 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 215/394] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:08 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1613 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:09.629360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:09.629984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:09.645122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:09.645666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:09.646108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:09.646817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:09.647273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:09.647690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:09.660399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:10.190625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.201938 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-234cdaac-b5cc-4b07-a69d-57101b98a103 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.579299 Oct 15 20:49:10.204418 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 216/395] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:09 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:10.211471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa55735-d8ae-491d-a31c-35a8ec74dd5f tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Calling method '.version_select at 0x7fe7544adf70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.220699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0aa55735-d8ae-491d-a31c-35a8ec74dd5f tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Policy check for os_compute_api:os-hosts:reboot failed with credentials {'is_admin': False, 'user_id': 'a62d07846dda4594a29a627c97cd236c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22a57f99fc634af4b37a892143bf6665', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa55735-d8ae-491d-a31c-35a8ec74dd5f tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Request method failure captured: Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/reboot HTTP/1.1 Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzCHiZeWwiUsfz8C1o9sy79hWmrL6Uxgzz6_NZekSjRyGUzM1K471uBzvXEyNLfac-1kEUYsWeCNCPf1oPHOo7W6FUVDrpdTy61CfmkjFsU3La5TyMkANuph66tyOBu2Bqv2T0y3Na9_ZnV11LEPOtuqxgj-MkIm4IXAGP6eCY0LPc Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:10.225051 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22a57f99fc634af4b37a892143bf6665"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a62d07846dda4594a29a627c97cd236c Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ncdv-master-1'} Oct 15 20:49:10.226208 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 203, in reboot Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'reboot', Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.226888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Oct 15 20:49:10.227827 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.230131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa55735-d8ae-491d-a31c-35a8ec74dd5f tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.231210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0aa55735-d8ae-491d-a31c-35a8ec74dd5f tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1/reboot" status: 403 len: 111 microversion: 2.1 time: 0.023612 Oct 15 20:49:10.232196 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 217/396] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/reboot => generated 111 bytes in 25 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:10.242576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbe60e43-786e-4a65-bbdb-13e05bc03373 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.244660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbe60e43-786e-4a65-bbdb-13e05bc03373 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Policy check for os_compute_api:os-instance-usage-audit-log:show failed with credentials {'is_admin': False, 'user_id': 'bd2aa13e39d644319136fd102c55cb95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89ed78bfa690487c816835dd282e2970', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbe60e43-786e-4a65-bbdb-13e05bc03373 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Request method failure captured: Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-instance_usage_audit_log/2022-10-15%2020:49:09 HTTP/1.1 Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzC2Y5mbzDJu4irVJ0Lmh4auOPB6WiLxvIK_fLM8wurwLvJzhylqMLuWdCB6fsw1U_RudBgR7spnHIe7RrZRbQNeD3NXG66HY0QIj7VoozIkjNXRD4Mk_CQmLGKWRFxrBf--okw_c0-oXzfMl-FFAFww90_YmxWWm_ZIZM-Fnz40Ek Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 89ed78bfa690487c816835dd282e2970 Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:10.247225 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89ed78bfa690487c816835dd282e2970"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89ed78bfa690487c816835dd282e2970"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89ed78bfa690487c816835dd282e2970"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89ed78bfa690487c816835dd282e2970"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 89ed78bfa690487c816835dd282e2970 Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: bd2aa13e39d644319136fd102c55cb95 Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project Oct 15 20:49:10.248413 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '2022-10-15 20:49:09'} Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 45, in show Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(iual_policies.BASE_POLICY_NAME % 'show', target={}) Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.249025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Oct 15 20:49:10.250589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.252521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbe60e43-786e-4a65-bbdb-13e05bc03373 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.252643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.253918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbe60e43-786e-4a65-bbdb-13e05bc03373 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762 tempest-InstanceUsageAuditLogNegativeTestJSON-169144762-project] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log/2022-10-15 20:49:09" status: 403 len: 128 microversion: 2.1 time: 0.014746 Oct 15 20:49:10.254775 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 218/397] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-instance_usage_audit_log/2022-10-15%2020%3A49%3A09 => generated 128 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:10.268175 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2a629eec-0122-44d2-b203-bc713ba459cb tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.786017 Oct 15 20:49:10.271982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:10.274840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:10.286638 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 180/398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:09 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:10.291705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.291875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:10.293153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:10.293594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:10.294159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:10.296454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:10.296924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:10.297390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:10.303098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:10.304391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:10.304993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:10.306451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:10.307468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:10.308050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:10.634674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.642634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:10.650714 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-075f3cc1-9867-4a63-8bc0-66073c57d8c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.361232 Oct 15 20:49:10.653989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 181/399] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:10.670072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c5aa3b1-c6a0-4f07-a1b6-add4206629ba tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Calling method '.version_select at 0x7fe75429bb80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.688310 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-2c5aa3b1-c6a0-4f07-a1b6-add4206629ba tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c5aa3b1-c6a0-4f07-a1b6-add4206629ba tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot HTTP/1.1 Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:10.690386 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: exception: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'nonexistent_hostname'} Oct 15 20:49:10.692069 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:10.692917 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 205, in reboot Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="reboot") Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Oct 15 20:49:10.694175 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.695428 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.698453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c5aa3b1-c6a0-4f07-a1b6-add4206629ba tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.699802 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2c5aa3b1-c6a0-4f07-a1b6-add4206629ba tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot" status: 404 len: 99 microversion: 2.1 time: 0.041885 Oct 15 20:49:10.701102 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 182/400] 10.222.0.16 () {58 vars in 1212 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot => generated 99 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:49:10.706849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-de630fa7-72c6-464b-8fd2-87cfddd64a4b req-20856d44-aea6-4401-a92b-12459cff74ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d3ad0a6-3ac3-4a3d-933b-406e5f41c891", "tag": "83dfa0e3-077b-4825-b6b7-8d0d1feea6a6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:10.732886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-de630fa7-72c6-464b-8fd2-87cfddd64a4b req-20856d44-aea6-4401-a92b-12459cff74ec service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:10.734006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-de630fa7-72c6-464b-8fd2-87cfddd64a4b req-20856d44-aea6-4401-a92b-12459cff74ec service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:10.734891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-de630fa7-72c6-464b-8fd2-87cfddd64a4b req-20856d44-aea6-4401-a92b-12459cff74ec service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:10.762499 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-de630fa7-72c6-464b-8fd2-87cfddd64a4b req-20856d44-aea6-4401-a92b-12459cff74ec service nova] Creating event network-changed:83dfa0e3-077b-4825-b6b7-8d0d1feea6a6 for instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 on nch1-master-1 Oct 15 20:49:10.768060 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-de630fa7-72c6-464b-8fd2-87cfddd64a4b req-20856d44-aea6-4401-a92b-12459cff74ec service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065520 Oct 15 20:49:10.770010 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 183/401] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:10.775917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fa5aba6e-7bbf-4b2b-9793-73f646b347e1 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Calling method '.version_select at 0x7fe754256af0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.778157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fa5aba6e-7bbf-4b2b-9793-73f646b347e1 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Policy check for os_compute_api:os-hosts:show failed with credentials {'is_admin': False, 'user_id': 'a62d07846dda4594a29a627c97cd236c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22a57f99fc634af4b37a892143bf6665', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fa5aba6e-7bbf-4b2b-9793-73f646b347e1 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Request method failure captured: Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzCHiZeWwiUsfz8C1o9sy79hWmrL6Uxgzz6_NZekSjRyGUzM1K471uBzvXEyNLfac-1kEUYsWeCNCPf1oPHOo7W6FUVDrpdTy61CfmkjFsU3La5TyMkANuph66tyOBu2Bqv2T0y3Na9_ZnV11LEPOtuqxgj-MkIm4IXAGP6eCY0LPc Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:10.780060 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22a57f99fc634af4b37a892143bf6665"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a62d07846dda4594a29a627c97cd236c Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1'} Oct 15 20:49:10.781671 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 270, in show Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'show', Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.782554 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Oct 15 20:49:10.783739 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.786229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fa5aba6e-7bbf-4b2b-9793-73f646b347e1 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:show to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.787266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fa5aba6e-7bbf-4b2b-9793-73f646b347e1 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1" status: 403 len: 109 microversion: 2.1 time: 0.015605 Oct 15 20:49:10.787960 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 184/402] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1 => generated 109 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:10.801560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31575c3d-e2a3-4b49-9305-fed551b26def tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Calling method '.version_select at 0x7fe754231700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.814354 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-31575c3d-e2a3-4b49-9305-fed551b26def tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31575c3d-e2a3-4b49-9305-fed551b26def tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname HTTP/1.1 Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:10.820513 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: exception: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'nonexistent_hostname'} Oct 15 20:49:10.821882 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 274, in show Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.822681 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 282, in show Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.823891 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.827479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31575c3d-e2a3-4b49-9305-fed551b26def tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.829786 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31575c3d-e2a3-4b49-9305-fed551b26def tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname" status: 404 len: 99 microversion: 2.1 time: 0.034794 Oct 15 20:49:10.847815 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 185/403] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname => generated 99 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 15 20:49:10.853703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-462c2bc2-e9bb-45ed-b1ac-0f97d94f1f06 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Calling method '.version_select at 0x7fe75428e040>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.855478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-462c2bc2-e9bb-45ed-b1ac-0f97d94f1f06 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Policy check for os_compute_api:os-hosts:shutdown failed with credentials {'is_admin': False, 'user_id': 'a62d07846dda4594a29a627c97cd236c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22a57f99fc634af4b37a892143bf6665', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-462c2bc2-e9bb-45ed-b1ac-0f97d94f1f06 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Request method failure captured: Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown HTTP/1.1 Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzCHiZeWwiUsfz8C1o9sy79hWmrL6Uxgzz6_NZekSjRyGUzM1K471uBzvXEyNLfac-1kEUYsWeCNCPf1oPHOo7W6FUVDrpdTy61CfmkjFsU3La5TyMkANuph66tyOBu2Bqv2T0y3Na9_ZnV11LEPOtuqxgj-MkIm4IXAGP6eCY0LPc Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:10.857024 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22a57f99fc634af4b37a892143bf6665"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a62d07846dda4594a29a627c97cd236c Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1'} Oct 15 20:49:10.858381 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 195, in shutdown Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'shutdown', Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.859207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Oct 15 20:49:10.860427 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.861491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-462c2bc2-e9bb-45ed-b1ac-0f97d94f1f06 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.861491 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-462c2bc2-e9bb-45ed-b1ac-0f97d94f1f06 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown" status: 403 len: 113 microversion: 2.1 time: 0.009194 Oct 15 20:49:10.873732 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 186/404] 10.222.0.16 () {58 vars in 1197 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown => generated 113 bytes in 25 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:10.879378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-56812bc1-f316-440e-9b11-3a37bfd3f60c tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Calling method '.version_select at 0x7fe75429bf70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.902206 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-56812bc1-f316-440e-9b11-3a37bfd3f60c tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-56812bc1-f316-440e-9b11-3a37bfd3f60c tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown HTTP/1.1 Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:10.903343 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: exception: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'nonexistent_hostname'} Oct 15 20:49:10.904513 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:10.905334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 197, in shutdown Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="shutdown") Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Oct 15 20:49:10.906309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-56812bc1-f316-440e-9b11-3a37bfd3f60c tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.907601 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-56812bc1-f316-440e-9b11-3a37bfd3f60c tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown" status: 404 len: 99 microversion: 2.1 time: 0.029822 Oct 15 20:49:10.911654 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 187/405] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown => generated 99 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:49:10.916668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8aa212a9-8e5f-4f6b-973e-eba8b4849d2a tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Calling method '.version_select at 0x7fe75420e4c0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.925782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8aa212a9-8e5f-4f6b-973e-eba8b4849d2a tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Policy check for os_compute_api:os-hosts:start failed with credentials {'is_admin': False, 'user_id': 'a62d07846dda4594a29a627c97cd236c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22a57f99fc634af4b37a892143bf6665', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8aa212a9-8e5f-4f6b-973e-eba8b4849d2a tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Request method failure captured: Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/startup HTTP/1.1 Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzCHiZeWwiUsfz8C1o9sy79hWmrL6Uxgzz6_NZekSjRyGUzM1K471uBzvXEyNLfac-1kEUYsWeCNCPf1oPHOo7W6FUVDrpdTy61CfmkjFsU3La5TyMkANuph66tyOBu2Bqv2T0y3Na9_ZnV11LEPOtuqxgj-MkIm4IXAGP6eCY0LPc Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:10.938952 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22a57f99fc634af4b37a892143bf6665"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a62d07846dda4594a29a627c97cd236c Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1'} Oct 15 20:49:10.940211 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 187, in startup Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'start', Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.941528 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Oct 15 20:49:10.942660 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.944330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8aa212a9-8e5f-4f6b-973e-eba8b4849d2a tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:start to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.945340 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8aa212a9-8e5f-4f6b-973e-eba8b4849d2a tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1/startup" status: 403 len: 110 microversion: 2.1 time: 0.032483 Oct 15 20:49:10.952890 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 188/406] 10.222.0.16 () {58 vars in 1194 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/startup => generated 110 bytes in 40 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:10.959455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6deeab21-3fbf-46b0-ad52-26894384a7d2 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Calling method '.version_select at 0x7fe75429bb80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:10.974248 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-6deeab21-3fbf-46b0-ad52-26894384a7d2 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6deeab21-3fbf-46b0-ad52-26894384a7d2 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/startup HTTP/1.1 Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:10.976588 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: exception: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'nonexistent_hostname'} Oct 15 20:49:10.978149 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:10.978821 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 189, in startup Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="startup") Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Oct 15 20:49:10.979926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6deeab21-3fbf-46b0-ad52-26894384a7d2 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6deeab21-3fbf-46b0-ad52-26894384a7d2 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname/startup" status: 404 len: 99 microversion: 2.1 time: 0.024642 Oct 15 20:49:10.989444 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 189/407] 10.222.0.16 () {58 vars in 1215 bytes} [Sat Oct 15 20:49:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/startup => generated 99 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:49:10.994461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c5717c16-f197-43dd-9426-c27f71dc1239 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Action: 'update', calling method: .version_select at 0x7fe754231670>, body: {"status": "enable", "maintenance_mode": "invalid"} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c5717c16-f197-43dd-9426-c27f71dc1239 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 51 Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:10.999051 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: {"status": "enable", "maintenance_mode": "invalid"} Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:11.000191 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'enable', 'maintenance_mode': 'invalid'}} Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['maintenance_mode']: Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'disable', Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'Enable', Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'Disable', Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'ENABLE', Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'DISABLE'], Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'type': 'string'} Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance: Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'invalid' Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.000881 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.006193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.007183 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.007183 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:49:11.007183 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:49:11.007183 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.007183 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.007476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '81fdbd60e924457692039abcb456cb89', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '08e2c30e5266478893afa293c836a3ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:11.007580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c5717c16-f197-43dd-9426-c27f71dc1239 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 400 to user: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:11.007659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] flavor_id is None. Assuming migration. {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4196}} Oct 15 20:49:11.007659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 20:49:11.007796 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c5717c16-f197-43dd-9426-c27f71dc1239 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 199 microversion: 2.1 time: 0.010602 Oct 15 20:49:11.007796 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 190/408] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:49:10 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 199 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 15 20:49:11.017664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c4377a2-d7e6-44ed-a28d-38e787a69500 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Action: 'update', calling method: .version_select at 0x7fe75428e700>, body: {"status": "invalid", "maintenance_mode": "enable"} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c4377a2-d7e6-44ed-a28d-38e787a69500 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 51 Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:11.035160 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: {"status": "invalid", "maintenance_mode": "enable"} Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:11.036624 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'invalid', 'maintenance_mode': 'enable'}} Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['status']: Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'disable', Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'Enable', Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'Disable', Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'ENABLE', Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'DISABLE'], Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'type': 'string'} Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance: Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'invalid' Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.037523 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.039106 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.040707 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.040707 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:49:11.040707 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:49:11.040707 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Oct 15 20:49:11.040707 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.043982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c4377a2-d7e6-44ed-a28d-38e787a69500 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 400 to user: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:11.048143 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c4377a2-d7e6-44ed-a28d-38e787a69500 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 189 microversion: 2.1 time: 0.033777 Oct 15 20:49:11.049278 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 191/409] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:49:11 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 189 bytes in 38 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:49:11.072960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4e09721-2ec1-406a-aa14-f8d119262778 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Action: 'update', calling method: .version_select at 0x7fe754231f70>, body: {"status": "enable", "maintenance_mode": "enable"} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:11.086053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4e09721-2ec1-406a-aa14-f8d119262778 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Policy check for os_compute_api:os-hosts:update failed with credentials {'is_admin': False, 'user_id': 'a62d07846dda4594a29a627c97cd236c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22a57f99fc634af4b37a892143bf6665', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4e09721-2ec1-406a-aa14-f8d119262778 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Request method failure captured: Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 50 Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzCHiZeWwiUsfz8C1o9sy79hWmrL6Uxgzz6_NZekSjRyGUzM1K471uBzvXEyNLfac-1kEUYsWeCNCPf1oPHOo7W6FUVDrpdTy61CfmkjFsU3La5TyMkANuph66tyOBu2Bqv2T0y3Na9_ZnV11LEPOtuqxgj-MkIm4IXAGP6eCY0LPc Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:11.092412 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22a57f99fc634af4b37a892143bf6665"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22a57f99fc634af4b37a892143bf6665"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 22a57f99fc634af4b37a892143bf6665 Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-754014085 Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a62d07846dda4594a29a627c97cd236c Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-754014085-project Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: {"status": "enable", "maintenance_mode": "enable"} Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Oct 15 20:49:11.094228 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'enable', 'maintenance_mode': 'enable'}} Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 112, in update Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'update', Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:11.095399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Oct 15 20:49:11.096974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.099661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4e09721-2ec1-406a-aa14-f8d119262778 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:update to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:11.103528 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4e09721-2ec1-406a-aa14-f8d119262778 tempest-HostsAdminNegativeTestJSON-754014085 tempest-HostsAdminNegativeTestJSON-754014085-project] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 403 len: 111 microversion: 2.1 time: 0.043972 Oct 15 20:49:11.105970 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 192/410] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:49:11 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 111 bytes in 49 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:11.121485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0e44f3be-2103-4f43-adff-39d35bb31a97 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Action: 'update', calling method: .version_select at 0x7fe7544d1550>, body: {"status": null, "maintenance_mode": null} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0e44f3be-2103-4f43-adff-39d35bb31a97 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 42 Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:11.129617 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: {"status": null, "maintenance_mode": null} Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid input for field/attribute status. Value: None. None is not of type 'string' Oct 15 20:49:11.130825 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ncdv-master-1', 'body': {'status': None, 'maintenance_mode': None}} Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: None is not of type 'string' Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['status']: Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'disable', Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'Enable', Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'Disable', Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'ENABLE', Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'DISABLE'], Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi 'type': 'string'} Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance: Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi None Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.131570 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute status. Value: None. None is not of type 'string' Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.132941 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.136439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:49:11.136439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:49:11.136439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute status. Value: None. None is not of type 'string' Oct 15 20:49:11.136439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.137351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0e44f3be-2103-4f43-adff-39d35bb31a97 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 400 to user: Invalid input for field/attribute status. Value: None. None is not of type 'string' {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:11.137898 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0e44f3be-2103-4f43-adff-39d35bb31a97 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 127 microversion: 2.1 time: 0.026599 Oct 15 20:49:11.144335 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 193/411] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Oct 15 20:49:11 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 127 bytes in 28 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 15 20:49:11.154669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28732b98-aec8-4a4f-b173-c22c327aaa14 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Action: 'update', calling method: .version_select at 0x7fe75429bca0>, body: {"status": "enable", "maintenance_mode": "enable"} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:11.161974 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.hosts [None req-28732b98-aec8-4a4f-b173-c22c327aaa14 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Enabling host nonexistent_hostname. Oct 15 20:49:11.173359 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-28732b98-aec8-4a4f-b173-c22c327aaa14 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28732b98-aec8-4a4f-b173-c22c327aaa14 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Request method failure captured: Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/os-hosts/nonexistent_hostname HTTP/1.1 Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 50 Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzD-CDU5ndfUhd5SXkpRqOY9dYlMyFA9Cm57in9fgk8_uS2y2TKmYltuSXJJptakakq9Qw4Z1Nu9bWNAu3dTLm7yUT_WPRdjCKzIFYbbmD7z6N_QGBJdF77GKjobGcyh1uWpD-ocDqQnRCN7AD55aGsiOaDEpDHu2G0osmfwsVcYd8 Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:11.176093 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/62478424b22c42c88c3d53b21bac76cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 62478424b22c42c88c3d53b21bac76cc Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-2046781838 Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 7e4d521d64424a37ab5a12e58a752a1a Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-HostsAdminNegativeTestJSON-2046781838-project-admin Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: {"status": "enable", "maintenance_mode": "enable"} Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: exception: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:11.177251 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'nonexistent_hostname', 'body': {'status': 'enable', 'maintenance_mode': 'enable'}} Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 159, in _set_enabled_status Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = self.api.set_host_enabled(context, host_name, enabled) Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Oct 15 20:49:11.177991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:11.181680 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 124, in update Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result['status'] = self._set_enabled_status(context, id, status) Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 163, in _set_enabled_status Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:11.182770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28732b98-aec8-4a4f-b173-c22c327aaa14 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:11.183557 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28732b98-aec8-4a4f-b173-c22c327aaa14 tempest-HostsAdminNegativeTestJSON-2046781838 tempest-HostsAdminNegativeTestJSON-2046781838-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/nonexistent_hostname" status: 404 len: 99 microversion: 2.1 time: 0.032232 Oct 15 20:49:11.183557 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 194/412] 10.222.0.16 () {60 vars in 1211 bytes} [Sat Oct 15 20:49:11 2022] PUT /compute/v2.1/os-hosts/nonexistent_hostname => generated 99 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:49:11.227905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:11.229387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:11.242080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:11.248180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:11.249114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:11.250211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:11.251321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:11.252139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:11.410630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48afbd67-6912-4fed-b662-059cbdb3b46b tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114/action" status: 202 len: 0 microversion: 2.1 time: 1.154224 Oct 15 20:49:11.414865 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 219/413] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:10 2022] POST /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114/action => generated 0 bytes in 1160 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:11.441657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:11.442528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:11.454402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:11.454924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:11.455364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:11.456825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:11.457556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:11.458043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:11.765266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:11.780380 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c4df740-2b2d-4fa8-9568-d8cb5bf0451b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.555746 Oct 15 20:49:11.781791 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 195/414] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:11 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:11.791740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:11.796791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:11.809855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:11.819679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:11.821459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:11.823261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:11.826907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:11.828036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:12.383198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:12.405844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fce9ada0-d424-4769-8d7b-5c9be276c3de tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.978868 Oct 15 20:49:12.406768 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 220/415] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:11 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 991 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:12.436796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:12.439554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:12.453333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:12.453837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:12.454278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:12.455001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:12.455471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:12.455898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:12.773466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:12.788346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cb35d034-9cfc-4493-a290-f1aefd6ac635 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1471 microversion: 2.1 time: 1.002627 Oct 15 20:49:12.807069 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 196/416] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:11 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1471 bytes in 1024 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:12.831757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-dfd4ca19-8da9-456e-a333-104ca12ce3b8 req-689c20bf-573f-465a-ba9d-61b863a8a60d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad32c860-892c-4d29-989d-86aebcce74bd", "name": "network-vif-plugged", "status": "completed", "tag": "17bb0738-b329-473c-b57b-c47dd0a496d4"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:12.851894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dfd4ca19-8da9-456e-a333-104ca12ce3b8 req-689c20bf-573f-465a-ba9d-61b863a8a60d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:12.858468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dfd4ca19-8da9-456e-a333-104ca12ce3b8 req-689c20bf-573f-465a-ba9d-61b863a8a60d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:12.861552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dfd4ca19-8da9-456e-a333-104ca12ce3b8 req-689c20bf-573f-465a-ba9d-61b863a8a60d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:12.877795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:12.889903 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8f109d4-96e8-4a98-9880-aa1ce0007e8a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.471417 Oct 15 20:49:12.892669 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 221/417] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:12 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:12.898935 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-dfd4ca19-8da9-456e-a333-104ca12ce3b8 req-689c20bf-573f-465a-ba9d-61b863a8a60d service nova] Creating event network-vif-plugged:17bb0738-b329-473c-b57b-c47dd0a496d4 for instance ad32c860-892c-4d29-989d-86aebcce74bd on nch2-master-1 Oct 15 20:49:12.899725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:12.900697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:12.910859 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-dfd4ca19-8da9-456e-a333-104ca12ce3b8 req-689c20bf-573f-465a-ba9d-61b863a8a60d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099710 Oct 15 20:49:12.912510 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 197/418] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:12.934312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:12.935758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:12.938615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:12.942962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:12.955668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:12.959270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:13.288242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:13.296357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d494e750-36e4-4f54-8364-6f398180a96f tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.402233 Oct 15 20:49:13.296942 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 222/419] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:12 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:13.438178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:13.438935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:13.450990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:13.451664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:13.452108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:13.452864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:13.453690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:13.455202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:13.818314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:13.818934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:13.829523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:13.832331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:13.833563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:13.836853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:13.837745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:13.838373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:13.856911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:13.872331 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-88108f0c-47c8-4fb0-80d2-0f46f8867dea tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.439352 Oct 15 20:49:13.873318 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 198/420] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:13 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:13.922475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:13.930023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:13.945826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:13.948531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:13.951179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:13.953584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:13.956723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:13.957585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:14.081617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=5 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:14.185631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:14.228871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-511868d6-92ff-42f2-9263-bd2bffa7ada0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1471 microversion: 2.1 time: 0.413556 Oct 15 20:49:14.240646 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 223/421] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:13 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1471 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:14.278826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:14.291867 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-13a0866c-9e7f-4832-b16f-cd4594967ab2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.373458 Oct 15 20:49:14.295522 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 199/422] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:13 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:14.322352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:14.323504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:14.334517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:14.335051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:14.336177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:14.337190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:14.337880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:14.338465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:14.851556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:14.864653 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4321b58-d4cf-47ed-85a3-a75c6d6a4ad5 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1319 microversion: 2.1 time: 0.543096 Oct 15 20:49:14.865741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 224/423] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:14 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1319 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:14.898548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:14.900992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:14.911073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:14.913877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:14.914489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:14.915029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:14.916009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:14.916667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:14.917516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.014656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-91a96c96-1a2e-4cb8-b2af-5e82964ac9a1 req-39ca14f8-3177-4877-a27c-187e7f3d7e85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "683f8544-7601-473f-b81f-c5bf78f0c781", "name": "network-vif-plugged", "status": "completed", "tag": "c833f037-3f48-49a5-9761-1581e6331daf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:15.051096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-91a96c96-1a2e-4cb8-b2af-5e82964ac9a1 req-39ca14f8-3177-4877-a27c-187e7f3d7e85 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.051906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-91a96c96-1a2e-4cb8-b2af-5e82964ac9a1 req-39ca14f8-3177-4877-a27c-187e7f3d7e85 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.053501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-91a96c96-1a2e-4cb8-b2af-5e82964ac9a1 req-39ca14f8-3177-4877-a27c-187e7f3d7e85 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.103044 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-91a96c96-1a2e-4cb8-b2af-5e82964ac9a1 req-39ca14f8-3177-4877-a27c-187e7f3d7e85 service nova] Creating event network-vif-plugged:c833f037-3f48-49a5-9761-1581e6331daf for instance 683f8544-7601-473f-b81f-c5bf78f0c781 on nch2-master-1 Oct 15 20:49:15.115567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-91a96c96-1a2e-4cb8-b2af-5e82964ac9a1 req-39ca14f8-3177-4877-a27c-187e7f3d7e85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107538 Oct 15 20:49:15.119283 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 225/424] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:15.256384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:15.257363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:15.266338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.267233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.267930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.268856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.269493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.270118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.309836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:15.326235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-889e1d62-ff72-4c18-a5c0-730a4b4ec1f8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.432664 Oct 15 20:49:15.329089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 200/425] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:14 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:15.334184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:15.338475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:15.354623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.355530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.356237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.357315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.358085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.358880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.556915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:15.569411 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5905479b-9d39-4000-9090-eb294a21ea75 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.237995 Oct 15 20:49:15.582386 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 201/426] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:15 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:15.760024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:15.772679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-88e488fc-d11a-488e-8ea3-5d91753a082c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1471 microversion: 2.1 time: 0.522615 Oct 15 20:49:15.776746 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 226/427] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:15 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1471 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:15.892294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:15.893017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:15.903844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.904408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.905076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:15.908577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:15.911242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:15.911699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.238751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:16.259102 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c6aa6a08-3b28-4167-bd7a-45b41c84e9f1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1468 microversion: 2.1 time: 0.373955 Oct 15 20:49:16.273083 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 202/428] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:15 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1468 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:16.286839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:16.290231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:16.297434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.298087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.298795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.299738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.300970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.301629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.352741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:16.353395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:16.364366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.364868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.365756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.368042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.368710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.370791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.421979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0f373e3-dfd4-450c-a2ef-c0d8c047ed4e tempest-DeleteServersAdminTestJSON-502411808 tempest-DeleteServersAdminTestJSON-502411808-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd/action" status: 202 len: 0 microversion: 2.1 time: 0.140366 Oct 15 20:49:16.426384 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 227/429] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:16 2022] POST /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd/action => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:16.431947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:16.432774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:16.439409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.440580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.441213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.442224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.442867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.443448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.517519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.518723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.519189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.519721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lazy-loading 'fault' on Instance uuid ad32c860-892c-4d29-989d-86aebcce74bd {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:16.733302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:16.748760 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-17d8a8a9-f7d0-48d3-9cdd-ffd537b75bcd tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1451 microversion: 2.1 time: 0.321026 Oct 15 20:49:16.750408 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 228/430] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:16 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1451 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:16.773752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:16.774331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:16.781578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.782355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.783181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.784647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.785701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.787050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.902664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:16.922061 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dec1043d-5eea-4d6c-b4aa-c567928a7f32 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.573820 Oct 15 20:49:16.942283 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 203/431] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:16 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:16.949942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:16.951282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:16.961128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.961801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.963008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:16.963921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:16.965030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:16.965806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.039389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:49:17.170263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df68bf5d-c963-4dfc-8a3a-7d799840f849 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 204 len: 0 microversion: 2.1 time: 0.235924 Oct 15 20:49:17.172088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 204/432] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:49:16 2022] DELETE /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:49:17.195091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:17.207543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:17.208495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:17.210028 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38fcdcb2-2d0a-47d9-902c-45b2278a3e8c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1314 microversion: 2.1 time: 0.442758 Oct 15 20:49:17.211321 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 229/433] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:16 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1314 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:17.216340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:17.217099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:17.224825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.226088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.227037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.228214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.229092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.229788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.240089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.240763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.241383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.242419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.244613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.245268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.273395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.274857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.276156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.276839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lazy-loading 'fault' on Instance uuid ad32c860-892c-4d29-989d-86aebcce74bd {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:17.585482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:17.595121 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c862a302-c39c-473c-b9cb-3f0e70bfb6d5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1471 microversion: 2.1 time: 0.411532 Oct 15 20:49:17.595703 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 205/434] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:17 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1471 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:17.636198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:17.646758 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fba99505-87f9-4362-b2e6-f912f05e36bf tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1457 microversion: 2.1 time: 0.434077 Oct 15 20:49:17.667303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 230/435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:17 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1457 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:17.950467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:17.951086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:17.961715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.963464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.965299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:17.966358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:17.967540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:17.970438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.154223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Action: 'update', calling method: .version_select at 0x7fe75420e4c0>, body: {"quota_set": {"ram": "5120"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:18.317922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.319550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.323193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.324236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.328291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.329288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.486179 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e05e03d8-0f55-4811-b42b-4bc1c51290b2 tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4" status: 200 len: 240 microversion: 2.36 time: 0.561651 Oct 15 20:49:18.487182 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 206/436] 10.222.0.16 () {62 vars in 1303 bytes} [Sat Oct 15 20:49:17 2022] PUT /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4 => generated 240 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 20:49:18.492889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:18.493716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:18.501276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.502174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.503336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.504498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.505370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.505983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.588945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:18.622459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:18.629475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:18.635491 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d8217ea5-cb64-4ec3-a3b7-d6310abe41db tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.689687 Oct 15 20:49:18.638599 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 231/437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:17 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:18.647578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Calling method '.version_select at 0x7fe75425d670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:18.775430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.775925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.776368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.781998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.782748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.783659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.851848 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ebae9c7a-b813-4903-a5f4-69d4530e24bb tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4/detail" status: 200 len: 672 microversion: 2.36 time: 0.211334 Oct 15 20:49:18.859475 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 232/438] 10.222.0.16 () {60 vars in 1304 bytes} [Sat Oct 15 20:49:18 2022] GET /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4/detail => generated 672 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:18.868553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:18.869227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:18.880854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.881591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.882151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:18.893246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:18.894017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:18.894558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.233013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:19.243905 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1bc4a29b-6dab-4f2f-8c9c-594b5a601a76 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1464 microversion: 2.1 time: 0.755291 Oct 15 20:49:19.244502 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 207/439] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:18 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1464 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:19.251049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:19.252060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:19.264905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.265945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.267772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.279852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.281594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.282902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.365585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.366163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.366616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.367144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lazy-loading 'fault' on Instance uuid ad32c860-892c-4d29-989d-86aebcce74bd {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:19.532287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:19.545866 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d75295d-de2e-4506-b512-ff726e02d869 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.683768 Oct 15 20:49:19.547404 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 233/440] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:18 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:19.567116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Action: 'update', calling method: .version_select at 0x7fe75427f940>, body: {"quota_set": {"ram": "2048"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:19.731053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.735062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.736303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.738893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.741778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.742420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.802079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.802941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.804147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.805435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:19.806147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:19.806775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:19.889797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73566912-cc2c-4ea6-9727-3a3434e7803b tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4?user_id=dd8d07607d68445a81bbcd3b3c2dfe2b" status: 200 len: 240 microversion: 2.36 time: 0.339074 Oct 15 20:49:19.891057 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 234/441] 10.222.0.16 () {62 vars in 1384 bytes} [Sat Oct 15 20:49:19 2022] PUT /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4?user_id=dd8d07607d68445a81bbcd3b3c2dfe2b => generated 240 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:19.976643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:19.993872 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e9de5220-5bfe-4cbc-90ff-458abd0e0fc9 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1457 microversion: 2.1 time: 0.745575 Oct 15 20:49:19.996386 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 208/442] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:19 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1457 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:20.001670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-492a1602-3fce-4bdb-baf5-fced2bafe7c3 req-ea54bcaa-694b-4ccd-b587-7284b03c6f43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d3ad0a6-3ac3-4a3d-933b-406e5f41c891", "name": "network-vif-plugged", "status": "completed", "tag": "83dfa0e3-077b-4825-b6b7-8d0d1feea6a6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:20.042144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-492a1602-3fce-4bdb-baf5-fced2bafe7c3 req-ea54bcaa-694b-4ccd-b587-7284b03c6f43 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.043428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-492a1602-3fce-4bdb-baf5-fced2bafe7c3 req-ea54bcaa-694b-4ccd-b587-7284b03c6f43 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.046430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-492a1602-3fce-4bdb-baf5-fced2bafe7c3 req-ea54bcaa-694b-4ccd-b587-7284b03c6f43 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.093805 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-492a1602-3fce-4bdb-baf5-fced2bafe7c3 req-ea54bcaa-694b-4ccd-b587-7284b03c6f43 service nova] Creating event network-vif-plugged:83dfa0e3-077b-4825-b6b7-8d0d1feea6a6 for instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 on nch1-master-1 Oct 15 20:49:20.105690 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-492a1602-3fce-4bdb-baf5-fced2bafe7c3 req-ea54bcaa-694b-4ccd-b587-7284b03c6f43 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107687 Oct 15 20:49:20.110831 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 209/443] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:20.124984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:20.130060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:20.141322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.146112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.150513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.153239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:20.155380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.156033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.158719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.168362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'deleted': False, 'project_id': '5c7b5e1d939a4b4592b1a81e4a30bcab'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:20.179956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:20.182723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.183956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.184628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.186040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.187134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.189180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.464153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:20.498388 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b2d6ddd-691f-4bb2-9ec0-dc0d6d09d35b tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 0.383058 Oct 15 20:49:20.499489 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 210/444] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:20 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:20.512983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de1e1e85-2f79-4d39-9c49-dbf2399b769c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] Calling method '.version_select at 0x7fe75428e0d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:20.588879 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6fe58dad-842e-4752-859c-bf99c306b9c9 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 15 microversion: 2.1 time: 0.696504 Oct 15 20:49:20.591894 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 235/445] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:19 2022] GET /compute/v2.1/servers/detail => generated 15 bytes in 700 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:20.618243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-690d0146-c961-4387-8b8c-d8637db41d98 req-a4f6ebb0-465a-441b-88d5-50cd5ae0e941 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad32c860-892c-4d29-989d-86aebcce74bd", "name": "network-vif-deleted", "tag": "17bb0738-b329-473c-b57b-c47dd0a496d4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:20.630683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-690d0146-c961-4387-8b8c-d8637db41d98 req-a4f6ebb0-465a-441b-88d5-50cd5ae0e941 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.631426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-690d0146-c961-4387-8b8c-d8637db41d98 req-a4f6ebb0-465a-441b-88d5-50cd5ae0e941 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.632020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-690d0146-c961-4387-8b8c-d8637db41d98 req-a4f6ebb0-465a-441b-88d5-50cd5ae0e941 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.647811 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-690d0146-c961-4387-8b8c-d8637db41d98 req-a4f6ebb0-465a-441b-88d5-50cd5ae0e941 service nova] Creating event network-vif-deleted:17bb0738-b329-473c-b57b-c47dd0a496d4 for instance ad32c860-892c-4d29-989d-86aebcce74bd on nch2-master-1 Oct 15 20:49:20.654328 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-690d0146-c961-4387-8b8c-d8637db41d98 req-a4f6ebb0-465a-441b-88d5-50cd5ae0e941 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050619 Oct 15 20:49:20.655535 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 236/446] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:20.662010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:20.662926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:20.670539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.671575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.672197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.673093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.675124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.675745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.814242 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de1e1e85-2f79-4d39-9c49-dbf2399b769c tempest-QuotasAdminTestV236-717142290 tempest-QuotasAdminTestV236-717142290-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4?user_id=dd8d07607d68445a81bbcd3b3c2dfe2b" status: 200 len: 282 microversion: 2.36 time: 0.309729 Oct 15 20:49:20.817303 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 211/447] 10.222.0.16 () {60 vars in 1364 bytes} [Sat Oct 15 20:49:20 2022] GET /compute/v2.1/os-quota-sets/d6d1303ef7b949da9374cf07859ccfd4?user_id=dd8d07607d68445a81bbcd3b3c2dfe2b => generated 282 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:20.834950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:20.850208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'deleted': False} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:20.868441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.868963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.869545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:20.870321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:20.870787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:20.871219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:21.055966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:21.136210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.multi_cell_list [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Listed batch of 7 results from cell out of 1000 limit. Returned 7 total so far. {{(pid=138689) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:49:21.172992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:21.186290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b2954d9-ae57-41b5-8923-94de4edba762 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.529337 Oct 15 20:49:21.196858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 237/448] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:20 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:21.230011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:21.230821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:21.242476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:21.251904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:21.252646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:21.253651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:21.254241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:21.254821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:21.320144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:21.321206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:21.322785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:21.323394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lazy-loading 'fault' on Instance uuid ad32c860-892c-4d29-989d-86aebcce74bd {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:21.399212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5a178b6a74140d4910f77d811febc08', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c126854c21954928af16db403a0b8919', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:21.410052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0634d435-aa89-4962-b01e-ef4495d97c66 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 200 len: 1309 microversion: 2.1 time: 0.187675 Oct 15 20:49:21.421539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:21.422245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:21.422872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:21.427215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 238/449] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:21 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 1309 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:21.475328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=95,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:21.524484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:21.525329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:21.532578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:21.537763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:21.540455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:21.542270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:21.543077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:21.549457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.119806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e639dd7f-d864-4006-91b9-cc8f5e6233d3 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?all_tenants=" status: 200 len: 11194 microversion: 2.1 time: 1.301272 Oct 15 20:49:22.121385 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 212/450] 10.222.0.16 () {58 vars in 1171 bytes} [Sat Oct 15 20:49:20 2022] GET /compute/v2.1/servers/detail?all_tenants= => generated 11194 bytes in 1303 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:22.163916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:22.173143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'deleted': False, 'project_id': '5c7b5e1d939a4b4592b1a81e4a30bcab'} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:22.182857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.183349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.183780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.184522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.185760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.186462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.386463 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e6372092-d31c-4c36-ba15-c6bc7c50e3ad tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?tenant_id=1f0b466076164a309233726db31b0efd" status: 200 len: 15 microversion: 2.1 time: 0.226724 Oct 15 20:49:22.406127 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 213/451] 10.222.0.16 () {58 vars in 1231 bytes} [Sat Oct 15 20:49:22 2022] GET /compute/v2.1/servers/detail?tenant_id=1f0b466076164a309233726db31b0efd => generated 15 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:49:22.406127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:22.406127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:22.406127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.406127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.406813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.406813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.406813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.406813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.570018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:22.579168 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2cb4fa1d-6c84-4564-b346-11e156b0624a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.190147 Oct 15 20:49:22.594265 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 214/452] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:22 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:22.602989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:22.609410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'tenant_id': '1f0b466076164a309233726db31b0efd', 'deleted': False} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:22.614936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.615423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.615859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.616596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.617070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.617689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.653910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.multi_cell_list [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138689) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:49:22.756453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.757258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.757884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.804017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:22.813786 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba15ed63-2541-4534-8917-3e609535434e tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1315 microversion: 2.1 time: 1.293649 Oct 15 20:49:22.817518 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 239/453] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:21 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1315 bytes in 1298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:22.831681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:22.833827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:22.850785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.854195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.854764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.857472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.857982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.858422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.902768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] HTTP exception thrown: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Request method failure captured: Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd HTTP/1.1 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyGbYOmxMZXrEpFWF7vRs2xfglVYJXAOH1UU0pRObKCbtySTA8XW1DChJ2qnp1sm_nh2knXzbJAg1paYMFrR98Dq3dhA7PvNeIbIqHnv9hqFwykMNxqqs_LUMwz3BqfTGsh1sXFVDDfvHbooX6HCHoHmfh6tryekXE1U5ir8UhNNC4 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:22.904601 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c126854c21954928af16db403a0b8919"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c126854c21954928af16db403a0b8919"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5a178b6a74140d4910f77d811febc08 Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:22.905914 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ad32c860-892c-4d29-989d-86aebcce74bd'} Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:22.906591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:22.907735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:22.909566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Returning 404 to user: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:22.911851 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef896dda-3910-401f-b174-55a0f6e134b1 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 404 len: 111 microversion: 2.1 time: 0.085975 Oct 15 20:49:22.912667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 240/454] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:22 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:22.926185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:22.928531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:22.945751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.951368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.955242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:22.956529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:22.957238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:22.959319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.023801 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] HTTP exception thrown: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Request method failure captured: Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd HTTP/1.1 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyGbYOmxMZXrEpFWF7vRs2xfglVYJXAOH1UU0pRObKCbtySTA8XW1DChJ2qnp1sm_nh2knXzbJAg1paYMFrR98Dq3dhA7PvNeIbIqHnv9hqFwykMNxqqs_LUMwz3BqfTGsh1sXFVDDfvHbooX6HCHoHmfh6tryekXE1U5ir8UhNNC4 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:23.025889 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c126854c21954928af16db403a0b8919"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c126854c21954928af16db403a0b8919"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5a178b6a74140d4910f77d811febc08 Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.027276 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ad32c860-892c-4d29-989d-86aebcce74bd'} Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:23.028046 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.029336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.038055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Returning 404 to user: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:23.040000 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95f910fe-c929-4641-84c1-00cc02151365 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 404 len: 111 microversion: 2.1 time: 0.120961 Oct 15 20:49:23.046188 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 241/455] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:49:22 2022] DELETE /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 111 bytes in 128 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:49:23.058000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.059127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: ad32c860-892c-4d29-989d-86aebcce74bd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:23.063638 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6f5df07f-7837-4430-ba70-9eff27aa398a tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=1f0b466076164a309233726db31b0efd" status: 200 len: 4825 microversion: 2.1 time: 0.468044 Oct 15 20:49:23.064399 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 215/456] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 20:49:22 2022] GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=1f0b466076164a309233726db31b0efd => generated 4825 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:23.075542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.075983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.076430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.089687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.090657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.091350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.097504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.107683 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] HTTP exception thrown: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Request method failure captured: Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd HTTP/1.1 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyGbYOmxMZXrEpFWF7vRs2xfglVYJXAOH1UU0pRObKCbtySTA8XW1DChJ2qnp1sm_nh2knXzbJAg1paYMFrR98Dq3dhA7PvNeIbIqHnv9hqFwykMNxqqs_LUMwz3BqfTGsh1sXFVDDfvHbooX6HCHoHmfh6tryekXE1U5ir8UhNNC4 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:23.109583 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c126854c21954928af16db403a0b8919"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c126854c21954928af16db403a0b8919"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5a178b6a74140d4910f77d811febc08 Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.110736 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ad32c860-892c-4d29-989d-86aebcce74bd'} Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:23.111372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. Oct 15 20:49:23.115091 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.117004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'tenant_id': '5c7b5e1d939a4b4592b1a81e4a30bcab', 'deleted': False} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:23.117990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Returning 404 to user: Instance ad32c860-892c-4d29-989d-86aebcce74bd could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:23.119356 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6eb04a1-0b9a-4762-96ca-0e0f2c9152f0 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd" status: 404 len: 111 microversion: 2.1 time: 0.067446 Oct 15 20:49:23.120808 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 242/457] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:23 2022] GET /compute/v2.1/servers/ad32c860-892c-4d29-989d-86aebcce74bd => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:23.128314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.130990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.132734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.133301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.134018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:23.134229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.134866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.135306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.151632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.152578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.153387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.154436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.157653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.158411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.195281 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6769d0ca-73b4-43db-a5c8-62e3a60aec93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=5c7b5e1d939a4b4592b1a81e4a30bcab" status: 200 len: 15 microversion: 2.1 time: 0.104398 Oct 15 20:49:23.213106 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 216/458] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 20:49:23 2022] GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=5c7b5e1d939a4b4592b1a81e4a30bcab => generated 15 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:49:23.226866 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] HTTP exception thrown: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.230237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1894a112-f223-4fb0-88ca-d77233c97877 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Request method failure captured: Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d HTTP/1.1 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyGbYOmxMZXrEpFWF7vRs2xfglVYJXAOH1UU0pRObKCbtySTA8XW1DChJ2qnp1sm_nh2knXzbJAg1paYMFrR98Dq3dhA7PvNeIbIqHnv9hqFwykMNxqqs_LUMwz3BqfTGsh1sXFVDDfvHbooX6HCHoHmfh6tryekXE1U5ir8UhNNC4 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:23.231610 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c126854c21954928af16db403a0b8919"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c126854c21954928af16db403a0b8919"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5a178b6a74140d4910f77d811febc08 Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.232973 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd6bfd383-07fb-4c4e-9d98-00a96ea7274d'} Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:23.233659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.234870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.236720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Returning 404 to user: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:23.241635 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22aa2fe2-e252-4756-a32d-8313ca07c250 tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 404 len: 111 microversion: 2.1 time: 0.113126 Oct 15 20:49:23.246621 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 243/459] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:49:23 2022] DELETE /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 111 bytes in 121 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:23.248349 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1894a112-f223-4fb0-88ca-d77233c97877 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?status=invalid_status" status: 200 len: 15 microversion: 2.1 time: 0.020987 Oct 15 20:49:23.256821 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 217/460] 10.222.0.16 () {58 vars in 1189 bytes} [Sat Oct 15 20:49:23 2022] GET /compute/v2.1/servers/detail?status=invalid_status => generated 15 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:23.258872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.262441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] [instance: d6bfd383-07fb-4c4e-9d98-00a96ea7274d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:23.271646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.272515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.274058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.275083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.275850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.276544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.286530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:23.289004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:23.303115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] HTTP exception thrown: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Request method failure captured: Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d HTTP/1.1 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyGbYOmxMZXrEpFWF7vRs2xfglVYJXAOH1UU0pRObKCbtySTA8XW1DChJ2qnp1sm_nh2knXzbJAg1paYMFrR98Dq3dhA7PvNeIbIqHnv9hqFwykMNxqqs_LUMwz3BqfTGsh1sXFVDDfvHbooX6HCHoHmfh6tryekXE1U5ir8UhNNC4 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:23.304868 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c126854c21954928af16db403a0b8919"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c126854c21954928af16db403a0b8919"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c126854c21954928af16db403a0b8919"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c126854c21954928af16db403a0b8919 Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-815785770 Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5a178b6a74140d4910f77d811febc08 Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersAdminTestJSON-815785770-project Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.306396 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd6bfd383-07fb-4c4e-9d98-00a96ea7274d'} Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:23.307157 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. Oct 15 20:49:23.308315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:23.310260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] Returning 404 to user: Instance d6bfd383-07fb-4c4e-9d98-00a96ea7274d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:23.312445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9871a49c-aacb-405f-827d-effff6f4902b tempest-DeleteServersAdminTestJSON-815785770 tempest-DeleteServersAdminTestJSON-815785770-project] 10.222.0.16 "GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d" status: 404 len: 111 microversion: 2.1 time: 0.058812 Oct 15 20:49:23.312962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.313360 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 244/461] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:23 2022] GET /compute/v2.1/servers/d6bfd383-07fb-4c4e-9d98-00a96ea7274d => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:23.313528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.313939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.314712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.315689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.316206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.608841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.608987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:23.623520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.634529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.635121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.635885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.636369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.636846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.732551 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-15f0f147-054b-445e-8724-c829f3134e7c tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 0 microversion: 2.1 time: 0.449636 Oct 15 20:49:23.734580 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 218/462] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:23 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 0 bytes in 452 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:49:23.750131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:23.761488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:23.765933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Searching by: {'vm_state': ['error'], 'deleted': False, 'project_id': '1f0b466076164a309233726db31b0efd'} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:23.781880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.782376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.782821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:23.784127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:23.789805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:23.790679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:24.035625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.multi_cell_list [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138689) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:49:24.107998 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6cec1909-fe61-4bbe-9c47-dc73def2ae35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers?status=error" status: 200 len: 338 microversion: 2.1 time: 0.367332 Oct 15 20:49:24.116512 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 219/463] 10.222.0.16 () {58 vars in 1150 bytes} [Sat Oct 15 20:49:23 2022] GET /compute/v2.1/servers?status=error => generated 338 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:24.132583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:24.133775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:24.141953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:24.142762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:24.143546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:24.144432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:24.145083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:24.145730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:24.356383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:24.366602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bfeb6a47-adc0-40ef-a11d-ea229bd6148b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.763156 Oct 15 20:49:24.367180 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 245/464] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:23 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:24.392091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "active"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:24.395018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:24.401732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:24.402827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:24.403528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:24.404627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:24.405974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:24.406623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:24.559560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b58c955-3669-4e33-9889-c24dc70e9442 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 0 microversion: 2.1 time: 0.172545 Oct 15 20:49:24.560185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 246/465] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:24 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:24.571510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:24.576413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:24.591295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:24.593567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:24.594241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:24.594895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:24.601482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:24.602044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.009072 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf6ff65f-0cd5-49e0-89f6-ec22ab199ded tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1614 microversion: 2.1 time: 0.440444 Oct 15 20:49:25.010024 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 247/466] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:24 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1614 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:25.041647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:25.042276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:25.056240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.056963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.059025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.061591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.063055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.063684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.155926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:25.168076 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2109f645-1a9b-4cd8-8335-60544faf69e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1502 microversion: 2.1 time: 1.050119 Oct 15 20:49:25.173114 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 220/467] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:24 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1502 bytes in 1056 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:25.202769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:25.203657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:25.218465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.221518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.222406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.225068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.226967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.231810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.273151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:49:25.436727 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ae78dfc-b6cb-44e1-95d0-9e84f6c7a9d6 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 204 len: 0 microversion: 2.1 time: 0.239221 Oct 15 20:49:25.456122 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 221/468] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:49:25 2022] DELETE /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:49:25.491404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:25.493505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:25.521143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.526264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.529498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.530362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.530936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.531288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.573500 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-275d7cfe-c92d-4f32-8596-958ac224358b tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1614 microversion: 2.1 time: 0.536133 Oct 15 20:49:25.576372 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 248/469] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:25 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1614 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:25.594136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:25.595577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:25.620061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.623534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.645980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.648255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.651363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.652714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.907225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:25.921941 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-670f0372-ae55-4c2d-a41e-9d3e95f15d47 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.462846 Oct 15 20:49:25.923796 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 222/470] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:25 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:25.938104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:25.954227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'host': 'nch2-master-1', 'deleted': False} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:25.975242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.975926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.977259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:25.980622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:25.981394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:25.991914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.133983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:26.143332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5689494-a301-4139-b5a1-14380c8ece73 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1508 microversion: 2.1 time: 0.563061 Oct 15 20:49:26.147289 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 249/471] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:25 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1508 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:26.161938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:26.162900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:26.179400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.180603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.181350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.183268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.185607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.186459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.253645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.multi_cell_list [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138689) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:49:26.275637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:49:26.531415 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-67f58726-e025-4d28-8476-d69cb848a67d tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers?host=nch2-master-1&all_tenants=1" status: 200 len: 988 microversion: 2.1 time: 0.604896 Oct 15 20:49:26.551617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=1,nova_cell1:UPDATE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:26.555481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 223/472] 10.222.0.16 () {58 vars in 1190 bytes} [Sat Oct 15 20:49:25 2022] GET /compute/v2.1/servers?host=nch2-master-1&all_tenants=1 => generated 988 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:26.575169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:26.591621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Searching by: {'host': 'nonexistent_host', 'deleted': False} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:49:26.618021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.620096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.620570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.621422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.621886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.628625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.634678 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b830f601-2e8b-4334-a323-c1565caee073 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 204 len: 0 microversion: 2.1 time: 0.485174 Oct 15 20:49:26.637275 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 250/473] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:49:26 2022] DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:49:26.690024 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3ec4bf16-81f0-44f5-b77e-42818a582d93 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers?host=nonexistent_host&all_tenants=1" status: 200 len: 15 microversion: 2.1 time: 0.125262 Oct 15 20:49:26.694166 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 224/474] 10.222.0.16 () {58 vars in 1196 bytes} [Sat Oct 15 20:49:26 2022] GET /compute/v2.1/servers?host=nonexistent_host&all_tenants=1 => generated 15 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:26.710511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:26.713973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:26.728007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.729086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.729806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.731391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.732454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.733119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.869447 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0acb0219-e919-41b3-b96f-4a0ff6559724 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 0 microversion: 2.1 time: 0.165656 Oct 15 20:49:26.870176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 251/475] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:26 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:26.880925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:26.884269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:26.902859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.903371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.903806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.904523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:26.904972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:26.909642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:26.928968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersWithSpecificFlavorTestJSON-server-1481652076", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "1723496106", "adminPass": "***", "networks": [{"uuid": "44be626a-f6e0-4bbc-b8f7-82aacab3bfed"}], "security_groups": [{"name": "tempest-securitygroup--1012426555"}], "key_name": "tempest-keypair-1690905189", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:27.090808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:49:27.092010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:49:27.092774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:49:27.093482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:49:27.094104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:49:27.156571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:27.171986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] validate_networks() for [('44be626a-f6e0-4bbc-b8f7-82aacab3bfed', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:49:27.237196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:27.246882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:27.248072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:27.248735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:27.249754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:27.250482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:27.251396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:27.313702 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64ef0488-8077-4d27-a0f9-b8021d5e0ec9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 1067 microversion: 2.1 time: 0.432406 Oct 15 20:49:27.326536 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 225/476] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:26 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 1067 bytes in 452 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Oct 15 20:49:27.336290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:27.344542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:27.359436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:27.360317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:27.360976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:27.364681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:27.365889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:27.367217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:27.699366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:27.723255 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b9a96ebb-4c2f-4b36-85d0-3c0d7a987175 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.392844 Oct 15 20:49:27.726193 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 226/477] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:27 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:27.748901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:27.750241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:27.768814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:27.769746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:27.771991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:27.774472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:27.776011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:27.777216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:28.432976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:28.446715 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-afaa5e59-601d-4ace-be11-fdc2334a1516 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1508 microversion: 2.1 time: 0.718252 Oct 15 20:49:28.448250 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 227/478] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:27 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1508 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:28.466254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:28.467227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:28.487626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:28.490777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:28.491517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:28.495073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:28.495906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:28.496904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:28.919123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:28.928525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f8940ef-5376-4c6e-99ef-3a223ed02419 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.477618 Oct 15 20:49:28.932144 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 228/479] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:28 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:29.057740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:49:29.061492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:49:29.063491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:49:29.063938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:49:29.065963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:49:29.096889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:29.102006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:29.102517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:29.103332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:29.106501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:29.106988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:29.169992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Getting quotas for project f856d315b3b84bdcac94b6d300ef8b2b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:49:29.183075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Getting quotas for user 14bd717b05ce4e40a1cd54c2db8474a6 and project f856d315b3b84bdcac94b6d300ef8b2b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:49:29.229802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:49:29.247401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=-1,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='blank',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:49:29.381649 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d11a7d13-3b4c-41b6-babf-23c3bdbd336f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.458091 Oct 15 20:49:29.391708 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 252/480] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:26 2022] POST /compute/v2.1/servers => generated 402 bytes in 2468 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:49:29.408415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:29.409832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:29.430762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:29.433130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:29.437357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:29.438497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:29.439168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:29.440461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:29.480968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-1463459543", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "6bc761ff-f597-43f8-85e3-57edd2825dc3"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:29.646779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] validate_networks() for [('6bc761ff-f597-43f8-85e3-57edd2825dc3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:49:30.178385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:30.206748 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26d0cf44-5930-4244-8c0e-8c70f64adaaf tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.811935 Oct 15 20:49:30.221663 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 253/481] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:29 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:30.234813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c27b086c-4c91-4a5e-8d0f-3bc8653d5586 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:30.242942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c27b086c-4c91-4a5e-8d0f-3bc8653d5586 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:30.512508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c27b086c-4c91-4a5e-8d0f-3bc8653d5586 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:30.524152 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c27b086c-4c91-4a5e-8d0f-3bc8653d5586 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1265 microversion: 2.1 time: 0.300982 Oct 15 20:49:30.525408 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 254/482] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:30 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1265 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:30.558216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:30.559169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:30.581781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:30.582299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:30.585420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:30.591215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:30.591831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:30.596607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:30.705503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:30.706945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:30.709809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:30.712551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lazy-loading 'fault' on Instance uuid 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:30.900175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '507faa1518b34884b3e095bfae431bc2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86408ec8ca9f4e33a16bdf9a23aa9faa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:30.910542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-287a59c4-8e46-4d9e-9624-978cb165e120 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 200 len: 1305 microversion: 2.1 time: 0.383876 Oct 15 20:49:30.916876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 255/483] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:30 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 1305 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:30.933980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:30.936231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:30.968820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:30.974203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:30.976132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:30.978393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:30.978886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:30.979337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:31.640378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:31.658803 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f6d6630-c0e4-4d55-9581-090e6f55080c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.739943 Oct 15 20:49:31.664244 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 256/484] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:30 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:31.670415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-44b08161-c01b-47af-bb69-98153d37299c req-1b357e10-6f30-41a7-a058-64d4943f89d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d3ad0a6-3ac3-4a3d-933b-406e5f41c891", "name": "network-vif-deleted", "tag": "83dfa0e3-077b-4825-b6b7-8d0d1feea6a6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:31.715362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44b08161-c01b-47af-bb69-98153d37299c req-1b357e10-6f30-41a7-a058-64d4943f89d0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:31.725019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44b08161-c01b-47af-bb69-98153d37299c req-1b357e10-6f30-41a7-a058-64d4943f89d0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:31.726428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44b08161-c01b-47af-bb69-98153d37299c req-1b357e10-6f30-41a7-a058-64d4943f89d0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:31.747118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-44b08161-c01b-47af-bb69-98153d37299c req-1b357e10-6f30-41a7-a058-64d4943f89d0 service nova] Creating event network-vif-deleted:83dfa0e3-077b-4825-b6b7-8d0d1feea6a6 for instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 on nch1-master-1 Oct 15 20:49:31.763351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-44b08161-c01b-47af-bb69-98153d37299c req-1b357e10-6f30-41a7-a058-64d4943f89d0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097491 Oct 15 20:49:31.764574 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 257/485] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:31.824967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:49:31.826179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:49:31.831999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:49:31.832785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:49:31.836250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:49:31.878495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:31.887665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:31.888127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:31.897817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:31.898347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:31.898800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:32.012849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:49:32.048318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for user c613201f55b94a528c6c36e5b5772799 and project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:49:32.112283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:49:32.124212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:49:32.377776 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5a52337-c3ef-45c0-8f8f-a2a1d0209d1c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.437027 Oct 15 20:49:32.389940 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 229/486] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 3457 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:49:32.410669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:32.416308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:32.424158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-UnshelveToHostMultiNodesTest-server-1306139748", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:32.444681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:32.445835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:32.448601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:32.450076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:32.453646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:32.457014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:32.605651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:49:32.606097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:49:32.609730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:49:32.610163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:49:32.610590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:49:32.639228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:32.648073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:32.648841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:32.649793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:32.650277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:32.654867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:32.734130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Getting quotas for project ab4db46c126d4607943fef56702f5eaf. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:49:32.756698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Getting quotas for user 58842ce40beb4def95f41884b083ec49 and project ab4db46c126d4607943fef56702f5eaf. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:49:32.803581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:49:32.821079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:49:32.997838 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da595271-d981-43b1-a89f-612205a80fad tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.91 time: 1.219779 Oct 15 20:49:33.013528 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 258/487] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:49:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1248 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:49:33.027295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:33.031971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:33.056463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.059337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.060206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:33.068905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.069534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.070614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:33.651751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:33.672235 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3baedb26-3098-4ed3-ad74-f39a78709dd5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1323 microversion: 2.1 time: 0.649366 Oct 15 20:49:33.672800 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 259/488] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:33 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1323 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:33.688131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bbe2edbd-bb59-4652-93a0-a23a25ba15ac req-f60404ed-255c-4847-acbe-a9d687a22046 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "56f00c4c-5642-40b7-b0a7-62c3ec8b3b56", "name": "network-vif-deleted", "tag": "1ed285bf-73a2-45c3-8c63-b59f9d0797cc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:33.727346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bbe2edbd-bb59-4652-93a0-a23a25ba15ac req-f60404ed-255c-4847-acbe-a9d687a22046 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.731756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bbe2edbd-bb59-4652-93a0-a23a25ba15ac req-f60404ed-255c-4847-acbe-a9d687a22046 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.733584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bbe2edbd-bb59-4652-93a0-a23a25ba15ac req-f60404ed-255c-4847-acbe-a9d687a22046 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:33.771526 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-bbe2edbd-bb59-4652-93a0-a23a25ba15ac req-f60404ed-255c-4847-acbe-a9d687a22046 service nova] Creating event network-vif-deleted:1ed285bf-73a2-45c3-8c63-b59f9d0797cc for instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 on nch1-master-1 Oct 15 20:49:33.787135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bbe2edbd-bb59-4652-93a0-a23a25ba15ac req-f60404ed-255c-4847-acbe-a9d687a22046 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109989 Oct 15 20:49:33.790847 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 260/489] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:33.807311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:33.808437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:33.828030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.831154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.832766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:33.835886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.838499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.839977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:33.875884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:33.891097 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8f460768-33ee-4fac-88d8-f5f167c6b27f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 1.498761 Oct 15 20:49:33.894369 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 230/490] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:32 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 1504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:33.898843 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 HTTP/1.1 Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:33.906787 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:33.909131 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5d3ad0a6-3ac3-4a3d-933b-406e5f41c891'} Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:33.910031 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:33.911182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:33.912803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:33.916956 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a58ac677-3e37-48a5-9e1b-74212bf4fed8 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 404 len: 111 microversion: 2.1 time: 0.123497 Oct 15 20:49:33.917264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:33.918960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:33.919750 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 261/491] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:33 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 111 bytes in 128 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:33.964930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:33.965537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.967198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:33.969423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.972339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:33.974179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:33.978913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:33.983485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.009663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:34.010425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:34.012567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.021152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:34.022069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:34.024213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.340333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:34.360458 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-338aec76-5a26-4f23-bf48-b62ba30992db tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1215 microversion: 2.1 time: 0.461322 Oct 15 20:49:34.364119 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 231/492] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:33 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1215 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:34.382430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:34.383321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:34.410468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:34.411578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:34.412737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.415816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:34.418741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:34.422102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.442910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:34.456366 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b66c4750-1abe-4c07-be31-49777ff3a1c7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.530338 Oct 15 20:49:34.456986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 262/493] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:33 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:34.469241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-1858109902", "metadata": {"image_type": "test"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:34.477146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:34.482684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:34.491385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:34.495611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:34.496105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.498156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:34.500589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:34.502686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:34.587300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891/action HTTP/1.1 Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 112 Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:34.598008 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-1858109902", "metadata": {"image_type": "test"}}} Oct 15 20:49:34.599357 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5d3ad0a6-3ac3-4a3d-933b-406e5f41c891', 'body': {'createImage': {'name': 'tempest-ImagesNegativeTestJSON-image-1858109902', 'metadata': {'image_type': 'test'}}}} Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:34.600160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:34.601380 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:34.603381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:34.603381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:34.603381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:34.603381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:34.603381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:34.606936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:34.612270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8590d163-0e58-42da-bc92-78a472269f5a tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "POST /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891/action" status: 404 len: 111 microversion: 2.1 time: 0.153052 Oct 15 20:49:34.619378 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 263/494] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Oct 15 20:49:34 2022] POST /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891/action => generated 111 bytes in 162 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:34.982670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:35.001616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:35.006540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:35.018171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7b0e4ac-3ebd-4364-a675-2f4dba0a8ea9 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1382955672", "id": 1145173451, "os-flavor-access:is_public": "False"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:35.050638 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-71ebc4bf-101f-462a-ae18-25a13cfd4283 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1458 microversion: 2.91 time: 0.683080 Oct 15 20:49:35.051233 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 232/495] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:34 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1458 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:35.062113 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7b0e4ac-3ebd-4364-a675-2f4dba0a8ea9 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.441370 Oct 15 20:49:35.081882 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 264/496] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:34 2022] POST /compute/v2.1/flavors => generated 439 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:35.086611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7131b1fd-fc53-4e28-8a4a-e1b03aaf92cf tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-330830546", "metadata": {"image_type": "test"}}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:35.097083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:35.099314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7131b1fd-fc53-4e28-8a4a-e1b03aaf92cf tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Failed to fetch instance by id tempest-invalid-1799686529 {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 20:49:35.103474 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7131b1fd-fc53-4e28-8a4a-e1b03aaf92cf tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance tempest-invalid-1799686529 could not be found. Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7131b1fd-fc53-4e28-8a4a-e1b03aaf92cf tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/tempest-invalid-1799686529/action HTTP/1.1 Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 111 Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:35.106222 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-330830546", "metadata": {"image_type": "test"}}} Oct 15 20:49:35.107397 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance tempest-invalid-1799686529 could not be found. Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'tempest-invalid-1799686529', 'body': {'createImage': {'name': 'tempest-ImagesNegativeTestJSON-image-330830546', 'metadata': {'image_type': 'test'}}}} Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance tempest-invalid-1799686529 could not be found. Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:35.108023 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance tempest-invalid-1799686529 could not be found. Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance tempest-invalid-1799686529 could not be found. Oct 15 20:49:35.109087 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.113325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7131b1fd-fc53-4e28-8a4a-e1b03aaf92cf tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance tempest-invalid-1799686529 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:35.114741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:35.115590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7131b1fd-fc53-4e28-8a4a-e1b03aaf92cf tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "POST /compute/v2.1/servers/tempest-invalid-1799686529/action" status: 404 len: 101 microversion: 2.1 time: 0.062711 Oct 15 20:49:35.123704 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 233/497] 10.222.0.16 () {60 vars in 1249 bytes} [Sat Oct 15 20:49:35 2022] POST /compute/v2.1/servers/tempest-invalid-1799686529/action => generated 101 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:35.129758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:35.131035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:35.139666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:35.140558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:35.141244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:35.144019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:35.145019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:35.145663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:35.161125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:35.163599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:35.163795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:35.167958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:35.168458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:35.177084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:35.701494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:35.714834 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52467ff1-7bea-4d53-81bd-6de2cdca47bc tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.630138 Oct 15 20:49:35.732741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 265/498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:35 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:35.744669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e942d0ff-fb91-47a9-a809-b4d035148d8d tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:35.907405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e942d0ff-fb91-47a9-a809-b4d035148d8d tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1145173451/action" status: 200 len: 97 microversion: 2.1 time: 0.173120 Oct 15 20:49:35.921901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 266/499] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:49:35 2022] POST /compute/v2.1/flavors/1145173451/action => generated 97 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:35.932879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84ba2cbe-f319-43e4-a0ad-b984b464e226 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-741141175"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:35.935710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84ba2cbe-f319-43e4-a0ad-b984b464e226 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Failed to fetch instance by id aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 20:49:35.946205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-84ba2cbe-f319-43e4-a0ad-b984b464e226 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84ba2cbe-f319-43e4-a0ad-b984b464e226 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action HTTP/1.1 Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 56 Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:35.950167 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: {"createImage": {"name": "tempest-test-snap-741141175"}} Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:35.951584 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'body': {'createImage': {'name': 'tempest-test-snap-741141175'}}} Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:35.952479 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:35.953706 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:35.957563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84ba2cbe-f319-43e4-a0ad-b984b464e226 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:35.957563 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-84ba2cbe-f319-43e4-a0ad-b984b464e226 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action" status: 404 len: 110 microversion: 2.1 time: 0.032414 Oct 15 20:49:35.964640 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 267/500] 10.222.0.16 () {60 vars in 1275 bytes} [Sat Oct 15 20:49:35 2022] POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action => generated 110 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:35.980216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:35.980849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:36.011784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:36.014230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:36.015544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:36.018041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:36.018856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:36.020105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:36.331502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:36.345949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e56c9fb0-98cc-4b09-9985-94683c44d9bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1269 microversion: 2.1 time: 0.377935 Oct 15 20:49:36.365645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 268/501] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:35 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1269 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:36.408667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:36.409824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:36.422190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:36.422686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:36.423175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:36.424617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:36.425133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:36.425572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:36.777144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:36.790105 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9e4f232-0c37-4caf-b9b9-298ba70d55fc tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.419238 Oct 15 20:49:36.822313 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 269/502] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:36 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:36.832101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7f3b9ab-b90e-42de-ad03-729d1f711987 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:36.959366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:36.991130 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e909843-676e-4878-8c94-8bbdc39d5d8f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 1.865620 Oct 15 20:49:37.002362 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 234/503] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:35 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 1878 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:37.003943 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d7f3b9ab-b90e-42de-ad03-729d1f711987 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] HTTP exception thrown: Flavor access already exists for flavor 34 and project 43aaf2ff5067452883510aad3a05adc3 combination. Oct 15 20:49:37.020199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a0152431-5320-4bc5-8fce-98384e6136c9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1017128252"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:37.028831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a0152431-5320-4bc5-8fce-98384e6136c9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Failed to fetch instance by id aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 20:49:37.038435 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a0152431-5320-4bc5-8fce-98384e6136c9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a0152431-5320-4bc5-8fce-98384e6136c9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action HTTP/1.1 Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 57 Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:37.040694 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: {"createImage": {"name": "tempest-test-snap-1017128252"}} Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:49:37.042060 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'body': {'createImage': {'name': 'tempest-test-snap-1017128252'}}} Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:37.042804 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Oct 15 20:49:37.043967 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7f3b9ab-b90e-42de-ad03-729d1f711987 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Request method failure captured: Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/flavors/1145173451/action HTTP/1.1 Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 67 Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzbeT19G9p_ac_7of5B2KwioRFspUiNUINL-xpYeCx-9sOxamqsXFBkJk2gvWmLCPFh9x-8Ixpx6g1IOrUYOBuST-jengi0slZJxSVup_YOlrkDqjU3cAfkpxvOAhjQgn09BbwuA-I0bDE4lKAKmmaUvlcfAhso457OAGB-icEF1do Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d66f78230e7f4217a939bda430a3d9f6 Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:49:37.045010 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d66f78230e7f4217a939bda430a3d9f6 Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fa3564c93bca41fc947c9d0762320f25 Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: {"addTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} Oct 15 20:49:37.047982 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: exception: Flavor access already exists for flavor 34 and project 43aaf2ff5067452883510aad3a05adc3 combination. Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1145173451', 'body': {'addTenantAccess': {'tenant': '43aaf2ff5067452883510aad3a05adc3'}}} Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = self._query(query) Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi conn.query(q) Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result.read() Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:49:37.049971 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry '34-43aaf2ff5067452883510aad3a05adc3' for key 'flavor_projects.uniq_flavor_projects0flavor_id0project_id'") Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 73, in _flavor_add_project Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi project.save(context.session) Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi session.flush() Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._flush(objects) Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:49:37.053478 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi compat.raise_( Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi flush_context.execute() Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi rec.execute(self) Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:49:37.056908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = self._query(query) Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi conn.query(q) Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:49:37.060022 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result.read() Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '34-43aaf2ff5067452883510aad3a05adc3' for key 'flavor_projects.uniq_flavor_projects0flavor_id0project_id'") Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO flavor_projects (created_at, updated_at, flavor_id, project_id) VALUES (%(created_at)s, %(updated_at)s, %(flavor_id)s, %(project_id)s)] Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 10, 15, 20, 49, 36, 993698), 'updated_at': None, 'flavor_id': 34, 'project_id': '43aaf2ff5067452883510aad3a05adc3'}] Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.063961 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 79, in _add_tenant_access Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi flavor.add_access(tenant) Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 411, in add_access Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._add_access(project_id) Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 404, in _add_access Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._flavor_add_project(self._context, self.id, project_id) Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 397, in _flavor_add_project Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _flavor_add_project(context, flavor_id, project_id) Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 75, in _flavor_add_project Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FlavorAccessExists(flavor_id=flavor_id, Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FlavorAccessExists: Flavor access already exists for flavor 34 and project 43aaf2ff5067452883510aad3a05adc3 combination. Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.066659 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 83, in _add_tenant_access Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=err.format_message()) Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Flavor access already exists for flavor 34 and project 43aaf2ff5067452883510aad3a05adc3 combination. Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:37.070312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:37.074257 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor access already exists for flavor 34 and project 43aaf2ff5067452883510aad3a05adc3 combination. Oct 15 20:49:37.074257 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.074257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7f3b9ab-b90e-42de-ad03-729d1f711987 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Returning 409 to user: Flavor access already exists for flavor 34 and project 43aaf2ff5067452883510aad3a05adc3 combination. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:37.074257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d7f3b9ab-b90e-42de-ad03-729d1f711987 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1145173451/action" status: 409 len: 152 microversion: 2.1 time: 0.221288 Oct 15 20:49:37.074257 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 270/504] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:49:36 2022] POST /compute/v2.1/flavors/1145173451/action => generated 152 bytes in 223 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:49:37.074257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:37.075021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:37.075021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.075021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.075021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.076275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.076275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.076275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.079448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a0152431-5320-4bc5-8fce-98384e6136c9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:37.080822 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a0152431-5320-4bc5-8fce-98384e6136c9 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action" status: 404 len: 112 microversion: 2.1 time: 0.073957 Oct 15 20:49:37.082763 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 235/505] 10.222.0.16 () {60 vars in 1281 bytes} [Sat Oct 15 20:49:37 2022] POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action => generated 112 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:37.091492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:37.093804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:37.121092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.121863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.122386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.123641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.124177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.125418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.371094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:37.396245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:37.406748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:37.418429 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d03961f8-bb3b-49ba-814f-185193a13c11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1512 microversion: 2.91 time: 0.370685 Oct 15 20:49:37.443224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 271/506] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:37 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1512 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:37.456697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-26085e14-9258-4399-ace2-a9c1ffd7dc95 req-a12b7154-f6c0-4054-b831-7f51093eccf7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1dd90a8-ba2b-462d-b891-c244f6a1722b", "tag": "110098ee-818d-4570-838f-f57f60ecb527"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:37.477154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-26085e14-9258-4399-ace2-a9c1ffd7dc95 req-a12b7154-f6c0-4054-b831-7f51093eccf7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.477982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-26085e14-9258-4399-ace2-a9c1ffd7dc95 req-a12b7154-f6c0-4054-b831-7f51093eccf7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.478656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-26085e14-9258-4399-ace2-a9c1ffd7dc95 req-a12b7154-f6c0-4054-b831-7f51093eccf7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.499744 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-26085e14-9258-4399-ace2-a9c1ffd7dc95 req-a12b7154-f6c0-4054-b831-7f51093eccf7 service nova] Creating event network-changed:110098ee-818d-4570-838f-f57f60ecb527 for instance d1dd90a8-ba2b-462d-b891-c244f6a1722b on nch2-master-1 Oct 15 20:49:37.506494 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-26085e14-9258-4399-ace2-a9c1ffd7dc95 req-a12b7154-f6c0-4054-b831-7f51093eccf7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061643 Oct 15 20:49:37.507795 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 272/507] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:37.515661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86f1949a-335f-4da5-aa3c-b7e510228828 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:37.533172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:37.558630 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78a006ab-3bc9-4986-a403-0084ca2665ef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.474274 Oct 15 20:49:37.559529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 236/508] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:37 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:37.567037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:37.567792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:37.574844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.586284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.589367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.590473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.592096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.593616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.640667 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 HTTP/1.1 Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:37.643169 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:37.644400 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '5d3ad0a6-3ac3-4a3d-933b-406e5f41c891'} Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:37.645244 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:37.646362 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:37.647515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:37.647515 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b424a5d9-f3e2-480a-a1c3-01b3e519a2e5 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 404 len: 111 microversion: 2.1 time: 0.082486 Oct 15 20:49:37.659478 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 237/509] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:49:37 2022] DELETE /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:37.667567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:37.679992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:37.694193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.694727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.695444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.696200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:37.696669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:37.728318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:37.916910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:37.945545 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2223be7b-48e6-4011-84d7-28d7ac0442b9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1269 microversion: 2.1 time: 0.284363 Oct 15 20:49:37.946962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 238/510] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:37 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1269 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:37.971493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:37.972224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] [instance: 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:38.012945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.014822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.015588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.019340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.020464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.024469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.069278 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86f1949a-335f-4da5-aa3c-b7e510228828 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1145173451/action" status: 200 len: 21 microversion: 2.1 time: 0.558843 Oct 15 20:49:38.073673 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] HTTP exception thrown: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:38.076158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 273/511] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:49:37 2022] POST /compute/v2.1/flavors/1145173451/action => generated 21 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Request method failure captured: Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 HTTP/1.1 Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxy-LKtPPw_fvXWBGL40DbnMhCe9-hrlC-wdCbB6xgLaHnqHF5bg07RuF14uNOcvtwCtNd0RYt9RGBFytYBaRcdCB40QyhZ38C3jOE5bBOa9CvI0ySZP-N3PGz6EKKEDI3oyRVvGv1GRyj2vPpPiAUXopYRh1lzR-wtDhaYQetw__Qg Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:49:38.078698 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/86408ec8ca9f4e33a16bdf9a23aa9faa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 86408ec8ca9f4e33a16bdf9a23aa9faa Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1178038769 Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 507faa1518b34884b3e095bfae431bc2 Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesNegativeTestJSON-1178038769-project Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:38.080619 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '5d3ad0a6-3ac3-4a3d-933b-406e5f41c891'} Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:38.081548 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. Oct 15 20:49:38.082922 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:38.084408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] Returning 404 to user: Instance 5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:38.084408 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-21e8798c-d578-49b6-a776-b82c06822bc0 tempest-ImagesNegativeTestJSON-1178038769 tempest-ImagesNegativeTestJSON-1178038769-project] 10.222.0.16 "GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891" status: 404 len: 111 microversion: 2.1 time: 0.133992 Oct 15 20:49:38.085749 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 239/512] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:37 2022] GET /compute/v2.1/servers/5d3ad0a6-3ac3-4a3d-933b-406e5f41c891 => generated 111 bytes in 138 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:38.090700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:38.097992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:38.101665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:38.102082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:38.129778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.130402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.131121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.131847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.132490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.133136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.133955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.134467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.136815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.138422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.140143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.140768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.683233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:38.693833 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2be3c397-c9cc-455a-8d75-d84573240f35 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.616319 Oct 15 20:49:38.696259 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 274/513] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:38 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:38.710571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e24a35f2-45d1-46aa-a650-61a53105c4b7 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:38.764531 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e24a35f2-45d1-46aa-a650-61a53105c4b7 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1145173451" status: 202 len: 0 microversion: 2.1 time: 0.066085 Oct 15 20:49:38.769788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 275/514] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:49:38 2022] DELETE /compute/v2.1/flavors/1145173451 => generated 0 bytes in 73 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:38.790387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:38.792191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:38.809820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.814910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.815617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.818918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.819564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.822358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.826009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:38.834720 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-79c4cc48-24a2-4c6b-8200-0ea188440b81 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.747551 Oct 15 20:49:38.846323 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 240/515] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:38 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:38.859816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:38.861382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:38.879801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.881804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.885544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.886584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:38.887256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:38.887893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:38.990380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:38.997717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:39.010641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:39.024876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:39.039281 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a5de273-28ad-42a7-be5c-ac30cd42daa9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1533 microversion: 2.91 time: 0.267124 Oct 15 20:49:39.058252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 276/516] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:38 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1533 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:39.074922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c97a87cb-d410-4f27-8ab7-e3b608394a27 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:39.107650 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c97a87cb-d410-4f27-8ab7-e3b608394a27 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5815 microversion: 2.1 time: 0.047163 Oct 15 20:49:39.109158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 277/517] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:39 2022] GET /compute/v2.1/flavors/detail => generated 5815 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:39.114978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:39.115872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:39.138373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:39.141663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:39.143306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:39.144322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:39.145141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:39.145808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:39.531700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:39.546795 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-823639dc-1e9d-4c99-a937-f40cba15aa18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.699002 Oct 15 20:49:39.548367 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 241/518] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:38 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:39.794843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:39.795852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:39.814561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:39.815063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:39.815506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:39.816232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:39.820713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:39.821202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.115479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:40.129502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lazy-loading 'services' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:40.134779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-661c2feb-e21d-49fe-a992-42a930e3e480 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 1.024262 Oct 15 20:49:40.158747 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 278/519] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:39 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 1034 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:40.158747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5ec7bc8-ce0e-4f6e-a593-da8b0d00b86b tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-910397519", "id": 418531693, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:40.182854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d5ec7bc8-ce0e-4f6e-a593-da8b0d00b86b tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 434 microversion: 2.1 time: 0.036146 Oct 15 20:49:40.197420 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 279/520] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:40 2022] POST /compute/v2.1/flavors => generated 434 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:40.204904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.205929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:40.216965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.217910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.219059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.220971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.221759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.222624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.242430 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0103d059-c57e-4a40-bfef-0640c3267eef tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1938 microversion: 2.91 time: 0.692466 Oct 15 20:49:40.243436 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 242/521] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:39 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1938 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:40.255388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.256121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:40.265720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.266286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.267368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.268169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.268631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.269270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.574010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:40.590525 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c97c8f0-8598-4c14-bb72-08a1f2cfb609 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.396349 Oct 15 20:49:40.591208 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 280/522] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:40 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:40.601765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1f93e22-ba73-44e9-a7f7-034628e38933 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.618287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e1f93e22-ba73-44e9-a7f7-034628e38933 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] HTTP exception thrown: Access list not available for public flavors. Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1f93e22-ba73-44e9-a7f7-034628e38933 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Request method failure captured: Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/flavors/418531693/os-flavor-access HTTP/1.1 Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzbeT19G9p_ac_7of5B2KwioRFspUiNUINL-xpYeCx-9sOxamqsXFBkJk2gvWmLCPFh9x-8Ixpx6g1IOrUYOBuST-jengi0slZJxSVup_YOlrkDqjU3cAfkpxvOAhjQgn09BbwuA-I0bDE4lKAKmmaUvlcfAhso457OAGB-icEF1do Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d66f78230e7f4217a939bda430a3d9f6 Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:49:40.620064 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d66f78230e7f4217a939bda430a3d9f6 Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fa3564c93bca41fc947c9d0762320f25 Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: exception: Access list not available for public flavors. Oct 15 20:49:40.621625 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'flavor_id': '418531693'} Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 52, in index Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Access list not available for public flavors. Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Access list not available for public flavors. Oct 15 20:49:40.622546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:40.625378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1f93e22-ba73-44e9-a7f7-034628e38933 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Returning 404 to user: Access list not available for public flavors. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:40.627633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1f93e22-ba73-44e9-a7f7-034628e38933 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/418531693/os-flavor-access" status: 404 len: 91 microversion: 2.1 time: 0.034919 Oct 15 20:49:40.628970 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 281/523] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 20:49:40 2022] GET /compute/v2.1/flavors/418531693/os-flavor-access => generated 91 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:49:40.633976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.635117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:40.648460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.650208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.650208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.650208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.650624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.651060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.672465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:40.687070 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-41a1eb2a-3eb9-4854-a743-b53603221f07 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.437904 Oct 15 20:49:40.688035 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 243/524] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:40 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:40.706781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.707513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:40.725509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.726541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.728344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.729481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.730201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.731047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.827033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lazy-loading 'services' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:40.899330 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a11e292-8b70-4879-8235-7a6b9818b847 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1938 microversion: 2.91 time: 0.268523 Oct 15 20:49:40.900530 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 282/525] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:40 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1938 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:40.906031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51e85ff6-6371-45be-ae27-7b4ecff193ae tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.943880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51e85ff6-6371-45be-ae27-7b4ecff193ae tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/418531693" status: 202 len: 0 microversion: 2.1 time: 0.041891 Oct 15 20:49:40.945156 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 283/526] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:49:40 2022] DELETE /compute/v2.1/flavors/418531693 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:40.950647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:40.963974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.964546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.965333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:40.966531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:40.967158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:40.967779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.007241 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fd364ae-4bfc-43db-a7a6-1e47d0230cae tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 498 microversion: 2.91 time: 0.060517 Oct 15 20:49:41.016636 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 284/527] 10.222.0.16 () {60 vars in 1217 bytes} [Sat Oct 15 20:49:40 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 498 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:49:41.035967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eeda489f-024a-46c8-af9a-a2a13d58db39 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:41.061165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eeda489f-024a-46c8-af9a-a2a13d58db39 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5815 microversion: 2.1 time: 0.038460 Oct 15 20:49:41.061911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 285/528] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:41 2022] GET /compute/v2.1/flavors/detail => generated 5815 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:41.071868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:41.072529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:41.079446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.080271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.080789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.081565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.082094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.082596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.109327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lazy-loading 'flavor' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:41.198404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:41.212747 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1ad12e30-c74f-4f49-96cb-e809607f0639 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.522091 Oct 15 20:49:41.226497 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 244/529] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:40 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:41.242403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e87f74a3-3fc6-4798-9534-218fa11bbd49 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-508733235", "id": 1098398426, "os-flavor-access:is_public": "False"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:41.270139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:41.292073 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e87f74a3-3fc6-4798-9534-218fa11bbd49 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 438 microversion: 2.1 time: 0.062359 Oct 15 20:49:41.293394 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 245/530] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:41 2022] POST /compute/v2.1/flavors => generated 438 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:41.344067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:41.346213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:41.371916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.372724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.375350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.376371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.389503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.396568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.400796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:41.579660 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b85a1ce-9452-44a3-93e6-aa3aabe82e20 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action" status: 202 len: 0 microversion: 2.91 time: 0.511606 Oct 15 20:49:41.580906 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 286/531] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:49:41 2022] POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action => generated 0 bytes in 518 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:49:41.821872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05238141-831d-4d44-a26d-8b8519a58a19 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:41.825551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-05238141-831d-4d44-a26d-8b8519a58a19 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Policy check for os_compute_api:os-flavor-access:add_tenant_access failed with credentials {'is_admin': False, 'user_id': '94d53e544c7047539a60d1cd0dd681e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '43aaf2ff5067452883510aad3a05adc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05238141-831d-4d44-a26d-8b8519a58a19 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Request method failure captured: Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/flavors/1098398426/action HTTP/1.1 Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 67 Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzaH4CyX0ECYoIQTOys3ZPvRZzcatNi3idlMTjxvM8aEtWcO0dAyzPQKhtq1FabuNep7j494rpcQW3T8vZqIWRjo3YNYZ4dJmi2p49XzxmpXmkQV5jx-McnI2T9gDGLEwUTybKrlG3kBh38fowYM-mJjOnhVMcyaEm04tgep14qkMo Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 43aaf2ff5067452883510aad3a05adc3 Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1960827844 Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:41.829019 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1960827844 Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 43aaf2ff5067452883510aad3a05adc3 Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1960827844 Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsAccessNegativeTestJSON-1960827844-project Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 94d53e544c7047539a60d1cd0dd681e3 Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1960827844-project Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: {"addTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} Oct 15 20:49:41.831041 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1098398426', 'body': {'addTenantAccess': {'tenant': '43aaf2ff5067452883510aad3a05adc3'}}} Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 66, in _add_tenant_access Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(fa_policies.POLICY_ROOT % "add_tenant_access", target={}) Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:41.831749 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Oct 15 20:49:41.833556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:41.835094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05238141-831d-4d44-a26d-8b8519a58a19 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:41.835094 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-05238141-831d-4d44-a26d-8b8519a58a19 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] 10.222.0.16 "POST /compute/v2.1/flavors/1098398426/action" status: 403 len: 130 microversion: 2.1 time: 0.221717 Oct 15 20:49:41.835094 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 287/532] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:49:41 2022] POST /compute/v2.1/flavors/1098398426/action => generated 130 bytes in 251 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:41.840396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:41.841343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:41.849246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.850444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.851001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.852417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.853018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.853693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.936606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:41.949384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=18 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:41.951362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:41.953963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:41.964827 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de30a192-75c4-4728-9abd-0b72e92b1f5d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.129067 Oct 15 20:49:41.966371 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 288/533] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:41 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:41.974349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:41.976038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:41.984343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.985688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:41.987821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:41.990239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:41.991250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:42.010849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:42.084386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=22 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:42.287206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:42.301918 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e9bcdfe5-dae0-4b22-8232-3270c4a78187 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 0.983525 Oct 15 20:49:42.302576 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 246/534] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:41 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 1008 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:42.326461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b17a7b01-0ecb-4018-be7e-392b0f06244c req-cc6f9a6d-d7f1-4750-902f-f16224cdf8ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4d275e72-929a-469e-9cdd-c5b4c4968506", "tag": "374f7499-815c-480d-9b88-199bfe2d2340"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:42.344155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b17a7b01-0ecb-4018-be7e-392b0f06244c req-cc6f9a6d-d7f1-4750-902f-f16224cdf8ac service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:42.345476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b17a7b01-0ecb-4018-be7e-392b0f06244c req-cc6f9a6d-d7f1-4750-902f-f16224cdf8ac service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:42.346084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b17a7b01-0ecb-4018-be7e-392b0f06244c req-cc6f9a6d-d7f1-4750-902f-f16224cdf8ac service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:42.363493 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b17a7b01-0ecb-4018-be7e-392b0f06244c req-cc6f9a6d-d7f1-4750-902f-f16224cdf8ac service nova] Creating event network-changed:374f7499-815c-480d-9b88-199bfe2d2340 for instance 4d275e72-929a-469e-9cdd-c5b4c4968506 on nch1-master-1 Oct 15 20:49:42.371922 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b17a7b01-0ecb-4018-be7e-392b0f06244c req-cc6f9a6d-d7f1-4750-902f-f16224cdf8ac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048194 Oct 15 20:49:42.374306 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 247/535] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:42.391889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:42.394334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:42.401982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:42.403609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:42.404694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:42.405663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:42.406338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:42.406989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:42.728863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:42.737882 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51c5a4b6-b207-4fd2-aed5-e739757c0210 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1471 microversion: 2.1 time: 0.770151 Oct 15 20:49:42.755180 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 289/536] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:41 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1471 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:42.763228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2029a876-af2f-4997-a2cb-3e69150d6473 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:42.785935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=28 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:42.786549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:42.795218 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2029a876-af2f-4997-a2cb-3e69150d6473 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1098398426" status: 202 len: 0 microversion: 2.1 time: 0.038193 Oct 15 20:49:42.800098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 290/537] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:49:42 2022] DELETE /compute/v2.1/flavors/1098398426 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:42.806554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:42.809672 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5752bcf8-bb0d-48ff-ac0f-fca6a1303257 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.433601 Oct 15 20:49:42.810272 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 248/538] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:42 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:42.815718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:42.827908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6da939dc-6fa5-408d-9864-ae318d4368f8 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:42.839946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:42.840635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:42.842178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:42.843121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:42.852095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:42.852853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:42.881250 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6da939dc-6fa5-408d-9864-ae318d4368f8 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5815 microversion: 2.1 time: 0.069751 Oct 15 20:49:42.884725 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 249/539] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:42 2022] GET /compute/v2.1/flavors/detail => generated 5815 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:42.914384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5b1ee39-a9e9-4bed-a528-46bdbe849f40 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-714644395", "id": 68096766, "os-flavor-access:is_public": "False"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:42.952204 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5b1ee39-a9e9-4bed-a528-46bdbe849f40 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 432 microversion: 2.1 time: 0.041551 Oct 15 20:49:42.964240 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 250/540] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:42 2022] POST /compute/v2.1/flavors => generated 432 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:49:42.971630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ded83452-bde9-4cd5-bcba-92a13d6d94e6 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:43.126972 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ded83452-bde9-4cd5-bcba-92a13d6d94e6 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/68096766/action" status: 200 len: 95 microversion: 2.1 time: 0.161005 Oct 15 20:49:43.128234 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 251/541] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:49:42 2022] POST /compute/v2.1/flavors/68096766/action => generated 95 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:43.133158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:43.134044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:43.145731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:43.146293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:43.147725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:43.147725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:43.147963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:43.148550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:43.489844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:43.502175 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e2854b5-df1d-4869-8385-e53a9730a402 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.700622 Oct 15 20:49:43.505604 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 291/542] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:42 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:43.540885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-62cc1e96-e5e0-4449-9185-5c6e5112ae08 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:43.544482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-62cc1e96-e5e0-4449-9185-5c6e5112ae08 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Policy check for os_compute_api:os-flavor-access:remove_tenant_access failed with credentials {'is_admin': False, 'user_id': '94d53e544c7047539a60d1cd0dd681e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '43aaf2ff5067452883510aad3a05adc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-62cc1e96-e5e0-4449-9185-5c6e5112ae08 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Request method failure captured: Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/flavors/68096766/action HTTP/1.1 Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 70 Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzaH4CyX0ECYoIQTOys3ZPvRZzcatNi3idlMTjxvM8aEtWcO0dAyzPQKhtq1FabuNep7j494rpcQW3T8vZqIWRjo3YNYZ4dJmi2p49XzxmpXmkQV5jx-McnI2T9gDGLEwUTybKrlG3kBh38fowYM-mJjOnhVMcyaEm04tgep14qkMo Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 43aaf2ff5067452883510aad3a05adc3 Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1960827844 Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:49:43.554343 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/43aaf2ff5067452883510aad3a05adc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1960827844 Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 43aaf2ff5067452883510aad3a05adc3 Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1960827844 Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsAccessNegativeTestJSON-1960827844-project Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 94d53e544c7047539a60d1cd0dd681e3 Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1960827844-project Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: {"removeTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} Oct 15 20:49:43.555926 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '68096766', 'body': {'removeTenantAccess': {'tenant': '43aaf2ff5067452883510aad3a05adc3'}}} Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 91, in _remove_tenant_access Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can( Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:49:43.556774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Oct 15 20:49:43.558284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:49:43.559640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-62cc1e96-e5e0-4449-9185-5c6e5112ae08 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:43.560787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-62cc1e96-e5e0-4449-9185-5c6e5112ae08 tempest-FlavorsAccessNegativeTestJSON-1960827844 tempest-FlavorsAccessNegativeTestJSON-1960827844-project] 10.222.0.16 "POST /compute/v2.1/flavors/68096766/action" status: 403 len: 133 microversion: 2.1 time: 0.049157 Oct 15 20:49:43.566090 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 292/543] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:49:43 2022] POST /compute/v2.1/flavors/68096766/action => generated 133 bytes in 55 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:43.569522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:43.570144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:43.581336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:43.581842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:43.582387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:43.583148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:43.583623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:43.584057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:43.911026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:43.930517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:43.933946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:43.955112 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da7832d4-e560-4bb8-ae36-61794e01fe77 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.816585 Oct 15 20:49:43.966525 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 252/544] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:43 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 836 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:43.980039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c24bbeaf-33b7-418a-93ba-8c9da8d13aed tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "43aaf2ff5067452883510aad3a05adc3"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:44.090962 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c24bbeaf-33b7-418a-93ba-8c9da8d13aed tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/68096766/action" status: 200 len: 21 microversion: 2.1 time: 0.122191 Oct 15 20:49:44.094021 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 253/545] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 20:49:43 2022] POST /compute/v2.1/flavors/68096766/action => generated 21 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:49:44.103883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.104582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:44.126101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.127235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.128067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.129159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.134691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.135192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.246444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:44.259998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a33c4f1-04fa-4220-adb0-b2101918f61c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 0.694415 Oct 15 20:49:44.265573 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 293/546] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:43 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:44.280973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.285723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:44.299053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.301437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.303748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.304675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.306558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.310957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.351923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:44.367848 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-29997865-657b-4d40-9b0d-381477ca35ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.272215 Oct 15 20:49:44.368666 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 254/547] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:44 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:44.374308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7577431e-5a21-4ebe-9af5-78c9d6e054ec tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.401392 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7577431e-5a21-4ebe-9af5-78c9d6e054ec tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/68096766" status: 202 len: 0 microversion: 2.1 time: 0.030343 Oct 15 20:49:44.403277 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 255/548] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 20:49:44 2022] DELETE /compute/v2.1/flavors/68096766 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:44.411235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-6672595e-6b69-4782-85f6-43d82157dda2 req-6bad6a13-55b3-4435-b853-0cea30096369 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d1ddd46-47f6-4948-a897-3b963780991e", "name": "network-vif-unplugged", "status": "completed", "tag": "7d14c08a-67ff-4f01-a95d-0c19d04975e3"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:44.436785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6672595e-6b69-4782-85f6-43d82157dda2 req-6bad6a13-55b3-4435-b853-0cea30096369 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.437557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6672595e-6b69-4782-85f6-43d82157dda2 req-6bad6a13-55b3-4435-b853-0cea30096369 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.438241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6672595e-6b69-4782-85f6-43d82157dda2 req-6bad6a13-55b3-4435-b853-0cea30096369 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.460014 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-6672595e-6b69-4782-85f6-43d82157dda2 req-6bad6a13-55b3-4435-b853-0cea30096369 service nova] Creating event network-vif-unplugged:7d14c08a-67ff-4f01-a95d-0c19d04975e3 for instance 5d1ddd46-47f6-4948-a897-3b963780991e on nch2-master-1 Oct 15 20:49:44.469101 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-6672595e-6b69-4782-85f6-43d82157dda2 req-6bad6a13-55b3-4435-b853-0cea30096369 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.062674 Oct 15 20:49:44.470977 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 256/549] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:44 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:44.480474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b4528e94-670d-4b04-909d-56aca5c54ada tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.509465 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b4528e94-670d-4b04-909d-56aca5c54ada tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5815 microversion: 2.1 time: 0.034831 Oct 15 20:49:44.510472 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 257/550] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:44 2022] GET /compute/v2.1/flavors/detail => generated 5815 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:44.531226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.538038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:44.547254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.548816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.550664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.551448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:44.551911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:44.552487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:44.713821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:44.730298 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-08e314d8-97ef-4322-bf09-275a1e9f6a62 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.204174 Oct 15 20:49:44.740146 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 258/551] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:44 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:44.747708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c3469ed0-5b7f-4761-bcd7-31b919ecd6a8 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1250535027", "id": 1755670932, "os-flavor-access:is_public": "False"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:44.784745 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c3469ed0-5b7f-4761-bcd7-31b919ecd6a8 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.042105 Oct 15 20:49:44.792981 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 259/552] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:49:44 2022] POST /compute/v2.1/flavors => generated 439 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:44.799181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b458194c-af47-47df-81ec-a345721d3a71 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "2f2cdf5281e744898df90c7e32baca94"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:44.874961 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-b458194c-af47-47df-81ec-a345721d3a71 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] HTTP exception thrown: Flavor access not found for 38 / 2f2cdf5281e744898df90c7e32baca94 combination. Oct 15 20:49:44.877513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b458194c-af47-47df-81ec-a345721d3a71 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Request method failure captured: Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/flavors/1755670932/action HTTP/1.1 Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 70 Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxzbeT19G9p_ac_7of5B2KwioRFspUiNUINL-xpYeCx-9sOxamqsXFBkJk2gvWmLCPFh9x-8Ixpx6g1IOrUYOBuST-jengi0slZJxSVup_YOlrkDqjU3cAfkpxvOAhjQgn09BbwuA-I0bDE4lKAKmmaUvlcfAhso457OAGB-icEF1do Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d66f78230e7f4217a939bda430a3d9f6 Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:49:44.878341 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d66f78230e7f4217a939bda430a3d9f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d66f78230e7f4217a939bda430a3d9f6 Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-952969313 Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: fa3564c93bca41fc947c9d0762320f25 Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: {"removeTenantAccess": {"tenant": "2f2cdf5281e744898df90c7e32baca94"}} Oct 15 20:49:44.879613 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: exception: Flavor access not found for 38 / 2f2cdf5281e744898df90c7e32baca94 combination. Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '1755670932', 'body': {'removeTenantAccess': {'tenant': '2f2cdf5281e744898df90c7e32baca94'}}} Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 104, in _remove_tenant_access Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi flavor.remove_access(tenant) Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 423, in remove_access Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._remove_access(project_id) Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 416, in _remove_access Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._flavor_del_project(self._context, self.id, project_id) Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 401, in _flavor_del_project Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return _flavor_del_project(context, flavor_id, project_id) Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 86, in _flavor_del_project Oct 15 20:49:44.880436 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.FlavorAccessNotFound(flavor_id=flavor_id, Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.FlavorAccessNotFound: Flavor access not found for 38 / 2f2cdf5281e744898df90c7e32baca94 combination. Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 107, in _remove_tenant_access Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor access not found for 38 / 2f2cdf5281e744898df90c7e32baca94 combination. Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:44.881650 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:49:44.884364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:49:44.884364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:49:44.884364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:49:44.884364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:49:44.884364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor access not found for 38 / 2f2cdf5281e744898df90c7e32baca94 combination. Oct 15 20:49:44.884364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:49:44.886070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b458194c-af47-47df-81ec-a345721d3a71 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Returning 404 to user: Flavor access not found for 38 / 2f2cdf5281e744898df90c7e32baca94 combination. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:49:44.887179 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b458194c-af47-47df-81ec-a345721d3a71 tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1755670932/action" status: 404 len: 124 microversion: 2.1 time: 0.092423 Oct 15 20:49:44.888358 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 260/553] 10.222.0.16 () {60 vars in 1200 bytes} [Sat Oct 15 20:49:44 2022] POST /compute/v2.1/flavors/1755670932/action => generated 124 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:49:44.888489 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99bb0f89-42cc-4e18-85af-03ccea7922d1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1476 microversion: 2.1 time: 0.621819 Oct 15 20:49:44.888964 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 294/554] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:44 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1476 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:44.913344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c7bf800-7a6f-48c2-8739-79ee36bf888d tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.944107 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c7bf800-7a6f-48c2-8739-79ee36bf888d tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1755670932" status: 202 len: 0 microversion: 2.1 time: 0.040974 Oct 15 20:49:44.944923 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 261/555] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:49:44 2022] DELETE /compute/v2.1/flavors/1755670932 => generated 0 bytes in 43 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:49:44.952841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4f071995-89cc-4c22-948b-06c0af7c53cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.988117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:44.988741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:44.998338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4f071995-89cc-4c22-948b-06c0af7c53cb tempest-FlavorsAccessNegativeTestJSON-952969313 tempest-FlavorsAccessNegativeTestJSON-952969313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5815 microversion: 2.1 time: 0.049022 Oct 15 20:49:44.998945 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 295/556] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:44 2022] GET /compute/v2.1/flavors/detail => generated 5815 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:45.009837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.009837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.009837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.009837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.010140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.010140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.216047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:45.225593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:45.227078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:45.248903 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f070146-d388-4ca7-bb55-f1de0cb478aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.265662 Oct 15 20:49:45.250089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 262/557] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:44 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:49:45.284144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:45.285508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:45.296821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.299400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.300071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.301072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.306778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.307543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.396677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:45.397596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:45.410356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.411963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.412881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.413908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.414618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.415297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.761616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:45.773362 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6be8a865-3f1a-4024-b10e-b45a9ca22a85 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.378816 Oct 15 20:49:45.773981 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 263/558] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:45 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:45.815302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:45.816137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:45.824283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.825113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.825631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:45.826613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:45.827258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:45.827789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.127071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:46.138068 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-045e3c9a-932e-40a7-9dc1-3af9b58352ee tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 0.857822 Oct 15 20:49:46.154824 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 296/559] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:45 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 876 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:46.163380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:46.165286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:46.173271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=4,nova_api:DELETE=10 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:46.179038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.179981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.180696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.181749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.182693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.183334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.358810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:46.374094 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-03dca430-653a-45b0-9b5b-b5d252d4eb18 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.586750 Oct 15 20:49:46.375722 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 264/560] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:45 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:46.380961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:46.387843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:46.402611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.403120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.403563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.404283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.404740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.418733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.577214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:46.588277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:46.590117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:46.601418 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0a73a11-e807-4127-b347-cc51a11feeb4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.224166 Oct 15 20:49:46.602692 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 265/561] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:46 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:46.608827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1dd90a8-ba2b-462d-b891-c244f6a1722b", "name": "network-vif-plugged", "status": "completed", "tag": "110098ee-818d-4570-838f-f57f60ecb527"}, {"server_uuid": "e597eee9-ea21-44d4-8f17-5726355a6114", "name": "network-vif-unplugged", "status": "completed", "tag": "a1988c6d-abca-4574-9f81-2e94f32f51da"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:46.625478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.627147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.627773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.647492 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Creating event network-vif-plugged:110098ee-818d-4570-838f-f57f60ecb527 for instance d1dd90a8-ba2b-462d-b891-c244f6a1722b on nch2-master-1 Oct 15 20:49:46.648261 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Creating event network-vif-unplugged:a1988c6d-abca-4574-9f81-2e94f32f51da for instance e597eee9-ea21-44d4-8f17-5726355a6114 on nch1-master-1 Oct 15 20:49:46.666222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] Instance e597eee9-ea21-44d4-8f17-5726355a6114 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:49:46.682553 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-198bf02c-ba3c-4900-bff6-17fa2949bfb1 req-981019fc-4abb-472c-a41b-c7614f6a110f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.078237 Oct 15 20:49:46.684034 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 266/562] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:46 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:46.802728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:46.803533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:46.821586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.822191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.822702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:46.823501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:46.824055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:46.824554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:47.036485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:47.057225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1c8b9b2-bee3-48cd-a59a-6ad2614f4271 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1475 microversion: 2.1 time: 0.900844 Oct 15 20:49:47.059620 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 297/563] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:46 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1475 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:47.174583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:47.176908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:47.199711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:47.206336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:47.207060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:47.208583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:47.209539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:47.210185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:47.259858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:47.273263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da24775b-8790-4df5-a6e1-5b7935d62709 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.475667 Oct 15 20:49:47.273781 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 267/564] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:46 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:47.401471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:47.402090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:47.415654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:47.421926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:47.422739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:47.427450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:47.428186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:47.428994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:47.958924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:47.982673 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-72f45552-b9fc-4cfb-a1b3-ec7b33ebcd70 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1391 microversion: 2.1 time: 0.584074 Oct 15 20:49:47.983966 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 268/565] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:47 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1391 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:47.989199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:47.990394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:47.991229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:48.000700 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4b0f598-fc59-4631-859f-4d0b55e1454b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 0.836636 Oct 15 20:49:48.005610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:48.006268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 298/566] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:47 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:48.008066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:48.008860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:48.010394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:48.011235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:48.011880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:48.084878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:48.085509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:48.097837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:48.101518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:48.101518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:48.101518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:48.101518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:48.101854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:48.294526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:48.310719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:48.314460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:48.321806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ad23e4d1-0569-4b4d-bded-bf9b5a5cc421 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.336474 Oct 15 20:49:48.322698 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 269/567] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:47 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:48.327746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:48.329127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:48.343309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:48.349634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:48.351018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:48.353350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:48.357413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:48.358037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:48.457233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:48.471875 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d1878976-0b0c-4ba7-9f6b-794ef8c3374f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1475 microversion: 2.1 time: 0.390622 Oct 15 20:49:48.472731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 299/568] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:48 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1475 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:48.908123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:48.918755 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e2922dbf-518a-4987-b15e-c9d620fbe720 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.594758 Oct 15 20:49:48.945444 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 270/569] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:48 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:49.012341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:49.012982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:49.029792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.030321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.034050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.034326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:49.035228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.035353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:49.036180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.039057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.048077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.048590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.049025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.068576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.069090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.071046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.328889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:49.337018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:49.355938 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5480ff14-d08d-41fc-98bf-a4082dc13ac5 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1589 microversion: 2.1 time: 0.342061 Oct 15 20:49:49.357550 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 300/570] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:49 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1589 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:49.369019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e597eee9-ea21-44d4-8f17-5726355a6114", "tag": "a1988c6d-abca-4574-9f81-2e94f32f51da"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:49.387212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.390438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.391837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.426147 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] Creating event network-changed:a1988c6d-abca-4574-9f81-2e94f32f51da for instance e597eee9-ea21-44d4-8f17-5726355a6114 on nch1-master-1 Oct 15 20:49:49.452034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] Instance e597eee9-ea21-44d4-8f17-5726355a6114 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:49:49.482857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4d0e1cf1-3ff1-49e9-92fb-9ca8746737ed req-2dfbb1ce-0606-49db-855f-9af08e901875 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.123645 Oct 15 20:49:49.483823 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 301/571] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:49.490581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:49.490581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:49.516517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.517578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.517998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.518726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.519191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.519624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.725246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:49.741557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:49.743529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:49.755212 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31d7658c-c703-4afe-89d5-d2299eb87acd tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.270173 Oct 15 20:49:49.755361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:49.757530 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 302/572] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:49 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:49.766329 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-adff5d98-48da-4235-937a-8d5a8f7ca985 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 0.735914 Oct 15 20:49:49.768554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:49.770326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:49.782435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.787922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.788637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.789670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.790352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.791037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.794645 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 271/573] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:49 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 751 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:49.800345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:49.800965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:49.823447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.824532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.827198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:49.828320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:49.828868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:49.831735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.087522 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d0cb93f-32a8-41cf-a2af-7d3b5e30a524 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.324485 Oct 15 20:49:50.088132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 303/574] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:49:49 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b/os-interface => generated 284 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:50.102128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:50.102980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:50.121528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:50.124474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:50.125141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.127000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:50.131821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:50.132506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.341066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:50.352876 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b1784f6-3d56-48fb-a496-94422e169a21 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1475 microversion: 2.1 time: 0.556676 Oct 15 20:49:50.353871 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 272/575] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:49 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1475 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:50.793266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:50.794264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:50.803180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:50.804377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:50.805105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.806596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:50.807388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:50.808880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.832650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:50.838673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:50.846112 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6dc257e0-94a9-477a-888d-b8a05d90b79d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.748543 Oct 15 20:49:50.866646 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 304/576] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:50 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:50.879019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:50.880573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:50.893694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:50.894418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:50.895559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.897188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:50.898149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:50.898816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:50.960141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:50.979061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=15 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:50.979646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:50.985296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:50.999757 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-503cfe66-a347-4912-a2df-f7aff199c75e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.210793 Oct 15 20:49:51.000771 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 273/577] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:50 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:49:51.063300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a865d488-56d2-4edb-b8ec-15a6b7deb3ac req-55d237da-3fd7-43be-9f6d-883804ed3671 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d275e72-929a-469e-9cdd-c5b4c4968506", "name": "network-vif-plugged", "status": "completed", "tag": "374f7499-815c-480d-9b88-199bfe2d2340"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:51.077671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a865d488-56d2-4edb-b8ec-15a6b7deb3ac req-55d237da-3fd7-43be-9f6d-883804ed3671 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:51.079772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a865d488-56d2-4edb-b8ec-15a6b7deb3ac req-55d237da-3fd7-43be-9f6d-883804ed3671 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:51.086394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a865d488-56d2-4edb-b8ec-15a6b7deb3ac req-55d237da-3fd7-43be-9f6d-883804ed3671 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:51.118009 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a865d488-56d2-4edb-b8ec-15a6b7deb3ac req-55d237da-3fd7-43be-9f6d-883804ed3671 service nova] Creating event network-vif-plugged:374f7499-815c-480d-9b88-199bfe2d2340 for instance 4d275e72-929a-469e-9cdd-c5b4c4968506 on nch1-master-1 Oct 15 20:49:51.131738 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a865d488-56d2-4edb-b8ec-15a6b7deb3ac req-55d237da-3fd7-43be-9f6d-883804ed3671 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071810 Oct 15 20:49:51.132906 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 274/578] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:49:51.382951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:51.384116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:51.400774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:51.401444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:51.403007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:51.406003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:51.407919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:51.410893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:51.642259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:51.651126 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3dd3cf7d-8413-4459-beb5-bb8b98b7d5eb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1473 microversion: 2.1 time: 0.274805 Oct 15 20:49:51.667298 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 275/579] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:51 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1473 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:51.826214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-155a529a-71ac-48a1-bb88-e12869cbb084 tempest-ListImagesTestJSON-710344632 tempest-ListImagesTestJSON-710344632-project] Calling method '.version_select at 0x7fe74ff6bdc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:51.889841 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-155a529a-71ac-48a1-bb88-e12869cbb084 tempest-ListImagesTestJSON-710344632 tempest-ListImagesTestJSON-710344632-project] 10.222.0.16 "GET /compute/v2.1/images/77903f56-0555-4c00-82ae-bbf42e11a2a9" status: 200 len: 784 microversion: 2.1 time: 0.221212 Oct 15 20:49:51.905271 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 276/580] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:49:51 2022] GET /compute/v2.1/images/77903f56-0555-4c00-82ae-bbf42e11a2a9 => generated 784 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:51.928360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:51.929207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:51.937208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:51.938652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:51.939289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:51.940670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:51.941405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:51.942064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:52.246495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:52.256309 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df06c754-45c8-474d-ba6b-3f7f6c134db8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 1.388464 Oct 15 20:49:52.267745 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 305/581] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:50 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 1400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:52.273437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20b3d8a2-83e0-4cc2-bdc7-dc5c5e27244e tempest-ListImagesTestJSON-710344632 tempest-ListImagesTestJSON-710344632-project] Calling method '.version_select at 0x7fe75443f9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:52.406405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20b3d8a2-83e0-4cc2-bdc7-dc5c5e27244e tempest-ListImagesTestJSON-710344632 tempest-ListImagesTestJSON-710344632-project] 10.222.0.16 "GET /compute/v2.1/images" status: 200 len: 445 microversion: 2.1 time: 0.137333 Oct 15 20:49:52.407355 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 306/582] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:49:52 2022] GET /compute/v2.1/images => generated 445 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:52.420930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:52.421891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:52.431168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:52.432384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:52.432999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:52.435005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:52.435634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:52.436329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:52.639424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:52.644322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:52.660828 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f879e8f5-5bd5-478e-84e7-5c869e10e02b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.763308 Oct 15 20:49:52.664722 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 277/583] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:51 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:52.668238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:52.673146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:52.676607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d43b8f01-3ba9-409f-85e9-7da56f46b796 tempest-ListImagesTestJSON-710344632 tempest-ListImagesTestJSON-710344632-project] Calling method '.version_select at 0x7fe7546ce280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:52.683689 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca7671f5-a308-4284-8464-cb4ac94eb5aa tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.267335 Oct 15 20:49:52.688464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 307/584] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:52 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:52.700047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:52.700696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:52.708335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:52.708925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:52.710068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:52.710789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:52.711761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:52.712200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:52.766120 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d43b8f01-3ba9-409f-85e9-7da56f46b796 tempest-ListImagesTestJSON-710344632 tempest-ListImagesTestJSON-710344632-project] 10.222.0.16 "GET /compute/v2.1/images/detail" status: 200 len: 787 microversion: 2.1 time: 0.099747 Oct 15 20:49:52.780763 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 278/585] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:49:52 2022] GET /compute/v2.1/images/detail => generated 787 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:53.162339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-99496a97-d595-4db9-b308-3359b28a20da req-395bb6ce-3179-42df-828a-9ce8ab542cb3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d1ddd46-47f6-4948-a897-3b963780991e", "name": "network-vif-plugged", "status": "completed", "tag": "7d14c08a-67ff-4f01-a95d-0c19d04975e3"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:53.176311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-99496a97-d595-4db9-b308-3359b28a20da req-395bb6ce-3179-42df-828a-9ce8ab542cb3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.176971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-99496a97-d595-4db9-b308-3359b28a20da req-395bb6ce-3179-42df-828a-9ce8ab542cb3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.177694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-99496a97-d595-4db9-b308-3359b28a20da req-395bb6ce-3179-42df-828a-9ce8ab542cb3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.193621 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-99496a97-d595-4db9-b308-3359b28a20da req-395bb6ce-3179-42df-828a-9ce8ab542cb3 service nova] Creating event network-vif-plugged:7d14c08a-67ff-4f01-a95d-0c19d04975e3 for instance 5d1ddd46-47f6-4948-a897-3b963780991e on nch2-master-1 Oct 15 20:49:53.199931 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-99496a97-d595-4db9-b308-3359b28a20da req-395bb6ce-3179-42df-828a-9ce8ab542cb3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040978 Oct 15 20:49:53.201412 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 279/586] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:53.295867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:53.296574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:53.300239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=3,nova_api:DELETE=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:53.324125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.325504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.326338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.328310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.331250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.332520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.365364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:53.375087 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c873b64-d42a-4e1b-bacc-6ca787aeac48 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1470 microversion: 2.1 time: 0.685097 Oct 15 20:49:53.378898 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 308/587] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:52 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1470 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:53.411638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:53.412773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:53.430218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.430972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.432043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.433695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.434392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.435315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.567155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4c83dd0-2f4a-4f03-80aa-1dd891e80283 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114/action" status: 204 len: 0 microversion: 2.1 time: 0.169831 Oct 15 20:49:53.568592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 309/588] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:53 2022] POST /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114/action => generated 0 bytes in 172 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:49:53.683909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:53.707451 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3a227078-5257-4d76-a7ad-c5792f829495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1311 microversion: 2.1 time: 0.420560 Oct 15 20:49:53.711473 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 280/589] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:53 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1311 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:53.724590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:53.727710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:53.742155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.744640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.745246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.747806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.748399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.751248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.851001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Calling method '.version_select at 0x7fe75427fb80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:53.853352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.854520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.855063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.895738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.896704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.902428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.931056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.935069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.935855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.956775 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4a5d044-741c-422d-b671-f7ce168ce813 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.385691 Oct 15 20:49:53.958021 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 310/590] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:53 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:53.963034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:53.966705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:53.983891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.985484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.986727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:53.988307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:53.989120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:53.989801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.224134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:54.233842 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5e1cbe45-c782-44c9-98f2-9a163ce2afd0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1470 microversion: 2.1 time: 0.520849 Oct 15 20:49:54.245634 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 281/591] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:53 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1470 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:54.257126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:54.258108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:54.264848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.265585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.267107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.268065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.268713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.269336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:54.269585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.280385 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be0d4dc0-a3bb-4355-bc26-ca981c9ba8f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1477 microversion: 2.1 time: 0.320908 Oct 15 20:49:54.294345 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 311/592] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:53 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1477 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:54.305804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:54.306680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:54.320201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.332790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.334696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.336197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.336844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.337523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.409797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:54.423818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:54.427458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:54.441411 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-acef4766-14fd-4809-b7aa-855f2b69f2c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.193755 Oct 15 20:49:54.444053 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 282/593] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:54 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:54.457306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a5343956-e1c8-46a1-bd3d-eaaf49b41bb1 req-c30722b7-ae1d-461e-bd0a-18b254bb82e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1dd90a8-ba2b-462d-b891-c244f6a1722b", "tag": "110098ee-818d-4570-838f-f57f60ecb527"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:54.476267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a5343956-e1c8-46a1-bd3d-eaaf49b41bb1 req-c30722b7-ae1d-461e-bd0a-18b254bb82e3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.477018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a5343956-e1c8-46a1-bd3d-eaaf49b41bb1 req-c30722b7-ae1d-461e-bd0a-18b254bb82e3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.477682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a5343956-e1c8-46a1-bd3d-eaaf49b41bb1 req-c30722b7-ae1d-461e-bd0a-18b254bb82e3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.493221 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a5343956-e1c8-46a1-bd3d-eaaf49b41bb1 req-c30722b7-ae1d-461e-bd0a-18b254bb82e3 service nova] Creating event network-changed:110098ee-818d-4570-838f-f57f60ecb527 for instance d1dd90a8-ba2b-462d-b891-c244f6a1722b on nch2-master-1 Oct 15 20:49:54.498680 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a5343956-e1c8-46a1-bd3d-eaaf49b41bb1 req-c30722b7-ae1d-461e-bd0a-18b254bb82e3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050893 Oct 15 20:49:54.499290 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 283/594] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:54.504673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Calling method '.version_select at 0x7fe74ff124c0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:54.507464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.508127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.508738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.524614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.526085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.526688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.542775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:54.559597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.560391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.560996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.568116 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-59092f02-f05e-4c7a-9fc4-9c5a3b377463 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1589 microversion: 2.1 time: 0.272501 Oct 15 20:49:54.568598 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1d4e901b-20dc-44e2-bb09-be450a8904de tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nch1-master-1/servers" status: 200 len: 433 microversion: 2.1 time: 0.067366 Oct 15 20:49:54.569651 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 284/595] 10.222.0.16 () {58 vars in 1212 bytes} [Sat Oct 15 20:49:54 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/servers => generated 433 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:54.572854 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 312/596] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:54 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1589 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:54.585605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Calling method '.version_select at 0x7fe75428ed30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:54.595721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.596488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.597391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.637730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.638260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.638708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.653946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.654454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.654900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.662457 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2038f4bd-4800-4d9d-bb55-dbd478364842 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.084675 Oct 15 20:49:54.663406 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 285/597] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:49:54 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:54.674013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Calling method '.version_select at 0x7fe7546d5430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:54.677223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.677223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.677619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.712711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.713443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.714046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.723950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de25497e-e1ec-408b-a6cd-87858f0f1505 tempest-HypervisorAdminUnderV252Test-1522162380 tempest-HypervisorAdminUnderV252Test-1522162380-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nch1-master-1/search" status: 200 len: 104 microversion: 2.1 time: 0.054160 Oct 15 20:49:54.725186 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 313/598] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 20:49:54 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/search => generated 104 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:54.737894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:54.738803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:54.749996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.750536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.751127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:54.751859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:54.753648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:54.754011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:55.261824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:55.262454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:55.278337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:55.279404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:55.280036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:55.283915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:55.285393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:55.288675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:55.618339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:55.627391 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-02f5a7e6-4e51-4baa-9325-499f4eebd883 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1494 microversion: 2.1 time: 0.894387 Oct 15 20:49:55.627938 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 286/599] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:54 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1494 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:55.649618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:55.650236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:55.661255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:55.661751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:55.662392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:55.663073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:55.663527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:55.663951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:55.962353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:55.972119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b36db058-323f-419d-ba4b-3d72d8a87c19 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1470 microversion: 2.1 time: 0.714307 Oct 15 20:49:55.972815 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 314/600] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:55 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1470 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:55.981085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:55.981999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:55.989975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:55.991135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:55.991663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:55.992643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:55.993249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:55.994061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.002317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:56.020505 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e149ba37-3580-4fb2-bf83-369b876da6ea tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1463 microversion: 2.1 time: 0.387350 Oct 15 20:49:56.026021 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 287/601] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:55 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1463 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:56.134475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:56.147900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:56.149579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:56.156505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-89273432-994e-4857-9886-ec3c2f279401 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.181914 Oct 15 20:49:56.157513 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 315/602] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:55 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:56.162557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:56.163682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:56.175402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.176297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.176988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.178113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.178806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.179528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.309815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:56.310781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:56.320423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.321011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.321599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.322606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.323326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.323826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.464838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:56.474654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6adb56b-d63b-438d-b7f0-e6be9a0e79f8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1463 microversion: 2.1 time: 0.315709 Oct 15 20:49:56.475248 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 316/603] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:56 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1463 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:56.500907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:56.505539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:56.516079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.516751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.517231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.517965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.518428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.518858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.618810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:49:56.619293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:49:56.619777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:49:56.620253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:49:56.620675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:49:56.642554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:49:56.697971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:56.705852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.707017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.707017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.707939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.708461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.708972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.746645 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-84fae4f2-ea6e-4b52-a37d-aa6b501d04a8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 1067 microversion: 2.1 time: 0.251010 Oct 15 20:49:56.749414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 317/604] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:56 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 1067 bytes in 255 msecs (HTTP/1.1 202) 10 headers in 452 bytes (2 switches on core 0) Oct 15 20:49:56.771689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:56.772314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:56.780243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.780757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.782200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.783385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:56.783915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:56.784351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:56.987376 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81e7aee2-1fb3-437e-90cf-8f012072daf8 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1645 microversion: 2.1 time: 0.964746 Oct 15 20:49:57.000815 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 288/605] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:56 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1645 bytes in 975 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:57.005589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:57.007272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:57.013332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:57.015128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.026867 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b1cc7d5-971a-4a98-8e18-a958347afdde tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.259875 Oct 15 20:49:57.027722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 318/606] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:56 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:57.027876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.028708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.029756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.030237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.030668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.042581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:57.043772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:57.057114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.057747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.058278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.059349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.059969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.060553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.358356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:57.373674 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-48c5a563-1f8f-4e7d-9a54-0c04760d7c62 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1470 microversion: 2.1 time: 0.374940 Oct 15 20:49:57.374565 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 289/607] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:56 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1470 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:57.379056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0dc24c13-d159-40ad-88ac-5895cb5d11a9 req-9e8440c7-361e-4948-8a13-e163d9ce5c24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e597eee9-ea21-44d4-8f17-5726355a6114", "name": "network-vif-plugged", "status": "completed", "tag": "a1988c6d-abca-4574-9f81-2e94f32f51da"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:57.388413 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb0b8350-b143-4a53-85cc-8e3476f2d920 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1645 microversion: 2.1 time: 0.358125 Oct 15 20:49:57.390405 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 319/608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:57 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1645 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:57.400310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:57.402417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:57.404272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0dc24c13-d159-40ad-88ac-5895cb5d11a9 req-9e8440c7-361e-4948-8a13-e163d9ce5c24 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.406603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0dc24c13-d159-40ad-88ac-5895cb5d11a9 req-9e8440c7-361e-4948-8a13-e163d9ce5c24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.407237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0dc24c13-d159-40ad-88ac-5895cb5d11a9 req-9e8440c7-361e-4948-8a13-e163d9ce5c24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.425168 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0dc24c13-d159-40ad-88ac-5895cb5d11a9 req-9e8440c7-361e-4948-8a13-e163d9ce5c24 service nova] Creating event network-vif-plugged:a1988c6d-abca-4574-9f81-2e94f32f51da for instance e597eee9-ea21-44d4-8f17-5726355a6114 on nch1-master-1 Oct 15 20:49:57.433000 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0dc24c13-d159-40ad-88ac-5895cb5d11a9 req-9e8440c7-361e-4948-8a13-e163d9ce5c24 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057177 Oct 15 20:49:57.434719 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 290/609] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:49:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:57.434864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.435558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.435935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.437154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.437560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.438171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.441345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:57.446760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.447418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.448027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.449150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.450160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.450834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.499285 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8de0e1a5-2213-4e07-b423-04c76e7ccc76 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.063093 Oct 15 20:49:57.500075 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 291/610] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:49:57 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:49:57.516372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:49:57.519878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:57.528464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.529516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.530138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.531723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:57.532605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:57.533054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:57.540792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:57.557586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:57.559404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:57.568010 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6eeafd76-e182-42b5-89c7-f5f87cff52d5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.175675 Oct 15 20:49:57.581077 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 320/611] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:57 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:57.654980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lazy-loading 'flavor' on Instance uuid 4d275e72-929a-469e-9cdd-c5b4c4968506 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:49:57.718494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Going to try to live migrate instance to nch2-master-1 {{(pid=138689) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:49:58.057653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:58.059577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:58.067708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:58.071615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:58.073026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:58.075325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:58.075811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:58.077240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:58.397093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:58.418981 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e5e911b-2dc3-4912-901a-5c834fd6be09 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.368412 Oct 15 20:49:58.419601 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 321/612] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:58 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:49:58.434441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:58.437085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:58.452807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:58.453888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:58.454334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:58.455162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:58.455634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:58.456068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:59.005645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:59.020400 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b662565f-fdd7-4ce5-8331-3b457ed499bb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1462 microversion: 2.1 time: 0.597286 Oct 15 20:49:59.027393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 322/613] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:58 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1462 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:49:59.035793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:59.036427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:59.069978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:59.070672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:59.070958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:59.073706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:59.074183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:59.074615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:59.770502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:59.779894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:59.784956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:49:59.791375 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a22ec03d-fbe1-454f-8ac5-6774ca97520b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.762721 Oct 15 20:49:59.805923 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 323/614] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:49:59 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 778 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:49:59.820488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:49:59.821416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:49:59.834157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:59.834746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:59.835201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:49:59.836062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:49:59.836393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:49:59.838272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.145353 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04f7a888-b0ab-40a9-8796-172b1d52543a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1613 microversion: 2.1 time: 0.337984 Oct 15 20:50:00.146714 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 324/615] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:49:59 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1613 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:00.159551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:00.160466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:00.179709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.180440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.181209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.182088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.183004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.183652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.427317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:00.435074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-56274a4a-21ed-4ca4-acb2-d5a7bf395356 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.287087 Oct 15 20:50:00.435868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 325/616] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:00 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:00.448312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aaf69c42-4acf-4e30-8e9b-89a31a76e18d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '.version_select at 0x7fe74fe96280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:00.452260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf69c42-4acf-4e30-8e9b-89a31a76e18d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.453055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf69c42-4acf-4e30-8e9b-89a31a76e18d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.454710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf69c42-4acf-4e30-8e9b-89a31a76e18d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.474468 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aaf69c42-4acf-4e30-8e9b-89a31a76e18d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 746 microversion: 2.1 time: 0.037369 Oct 15 20:50:00.478116 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 326/617] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:50:00 2022] GET /compute/v2.1/os-migrations => generated 746 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:00.701686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Calling method '.version_select at 0x7fe74fefc8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:00.703955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.704453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.704896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.748441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.748816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.749488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.765721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.766709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.767438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:00.775109 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a1c305ba-1310-43f6-9f3e-ca8d894108df tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.28 time: 0.293067 Oct 15 20:50:00.776314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 327/618] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 20:50:00 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 20:50:00.789511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-750339761", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:00.877191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:50:00.948849 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a84295ca-0359-4217-8309-51bd36225914 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506/action" status: 202 len: 0 microversion: 2.1 time: 3.437403 Oct 15 20:50:00.954854 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 292/619] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:49:57 2022] POST /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506/action => generated 0 bytes in 3444 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:00.962355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Calling method '.version_select at 0x7fe7546204c0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:00.968601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:00.970244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:00.971254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.000261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:01.019509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.021063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.021953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.034522 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-796d241d-ee50-4bb7-baab-7c8fb2b33107 tempest-HypervisorAdminV228Test-948228821 tempest-HypervisorAdminV228Test-948228821-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 718 microversion: 2.28 time: 0.077397 Oct 15 20:50:01.035878 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 293/620] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 20:50:00 2022] GET /compute/v2.1/os-hypervisors/1 => generated 718 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:50:01.041240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:01.042100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:01.056584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.057439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.058123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.060076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.060707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.061466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.182482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:01.196636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:01.216614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:01.223714 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2e3c445e-4b5c-4835-8f2d-0cd50ecd4a82 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.186545 Oct 15 20:50:01.241182 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 294/621] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:01 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:01.241182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:01.241182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:01.241182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.241182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.242041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.242041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.242337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.242702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.791066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:01.800522 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-058fcb2b-f748-49f2-b999-6024f7063817 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.574145 Oct 15 20:50:01.813477 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 295/622] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:01 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:01.824814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:01.825587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:01.833609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:01.834593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:01.835252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:01.835835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:01.836576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:01.837692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.838264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.838658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.840361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.840953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.841378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.855686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.856297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.856620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.857386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:01.857852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:01.858275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:01.904517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:50:01.927857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for user e32a1c53b4be44188b07c768405961ed and project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:50:01.945902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:50:01.948762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:50:02.025819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5143fa5c-e58b-4c1d-86ad-8c505f8f08e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.245841 Oct 15 20:50:02.027263 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 328/623] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:50:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 1250 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:50:02.040965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f11182c8-21b8-4f1b-affe-3a75859f6b1e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:02.042411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f11182c8-21b8-4f1b-affe-3a75859f6b1e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:02.234253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f11182c8-21b8-4f1b-affe-3a75859f6b1e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:02.244880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f11182c8-21b8-4f1b-affe-3a75859f6b1e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1212 microversion: 2.1 time: 0.208840 Oct 15 20:50:02.245544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 329/624] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:02 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1212 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:02.257323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:02.257984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:02.278682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:02.279193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:02.279638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:02.280581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:02.282078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:02.293832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:02.521838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:02.537647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:02.539474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:02.553334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f095db9a-1c66-42b0-991b-62751331ab28 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.302252 Oct 15 20:50:02.570001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 330/625] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:02 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:02.644958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:02.656536 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9d835db8-a5a9-48fd-8b42-5f8e431deab0 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.841250 Oct 15 20:50:02.661794 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 296/626] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:01 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:02.826794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:02.827627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:02.830655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:02.873692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:02.876298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:02.877933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:02.878875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:02.881995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:02.882668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.273100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:03.273765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:03.285233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:03.300920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:03.301842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.306075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:03.306864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:03.310523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.500142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:03.511249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ef5cfd9-d7d2-4ace-87e4-c834e1e01f77 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.689116 Oct 15 20:50:03.513425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 331/627] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:02 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:03.593557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:03.594934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:03.595736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:03.605258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:03.606618 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b180493b-e423-48a0-8d3c-cd2a4d1da812 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1264 microversion: 2.1 time: 0.338181 Oct 15 20:50:03.612224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:03.612690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.613455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:03.614350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:03.615041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.615208 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 297/628] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:03 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1264 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:03.694083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:03.697595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:03.714252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:03.716801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:03.718978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.720822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:03.724654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:03.725578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:03.728709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:03.737530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:03.740147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:03.752317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ee72a4a-6c4f-4062-99c7-7bbd400123eb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.164092 Oct 15 20:50:03.756937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 332/629] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:03 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:50:04.210976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:04.228498 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31dff6fe-3319-44a2-8d89-57242932fe3d tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.539881 Oct 15 20:50:04.241354 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 298/630] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:03 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:04.538476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:04.539083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:04.548316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:04.553933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:04.554745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:04.555653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:04.556256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:04.556834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:04.632441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:04.633414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:04.641900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:04.642665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:04.643878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:04.644769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:04.645628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:04.646348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:04.744635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:04.756554 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da30f2be-10e4-49c4-9d45-9b33d567e06c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1270 microversion: 2.1 time: 0.128489 Oct 15 20:50:04.757851 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 299/631] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:04 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1270 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:04.795038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:04.797320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:04.810576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:04.811465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:04.812693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:04.813677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:04.814357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:04.815752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:04.930453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:04.946682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:04.953447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:04.993968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:04.996347 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-afcdf0b7-2bf0-4137-8d90-318844952e02 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.207758 Oct 15 20:50:04.998119 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 300/632] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:04 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:05.004883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63d0d763-4a8a-4b34-83ad-19d441793335 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.470750 Oct 15 20:50:05.019944 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 333/633] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:04 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:05.086068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a8ce53ae-73c7-43d7-be86-6df2230324a2 req-924c15e6-cb4e-414b-b8fd-884473d02d3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d275e72-929a-469e-9cdd-c5b4c4968506", "name": "network-vif-unplugged", "status": "completed", "tag": "374f7499-815c-480d-9b88-199bfe2d2340"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:05.105797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a8ce53ae-73c7-43d7-be86-6df2230324a2 req-924c15e6-cb4e-414b-b8fd-884473d02d3b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:05.106958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a8ce53ae-73c7-43d7-be86-6df2230324a2 req-924c15e6-cb4e-414b-b8fd-884473d02d3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:05.107416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a8ce53ae-73c7-43d7-be86-6df2230324a2 req-924c15e6-cb4e-414b-b8fd-884473d02d3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:05.133050 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a8ce53ae-73c7-43d7-be86-6df2230324a2 req-924c15e6-cb4e-414b-b8fd-884473d02d3b service nova] Creating event network-vif-unplugged:374f7499-815c-480d-9b88-199bfe2d2340 for instance 4d275e72-929a-469e-9cdd-c5b4c4968506 on nch1-master-1 Oct 15 20:50:05.149402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a8ce53ae-73c7-43d7-be86-6df2230324a2 req-924c15e6-cb4e-414b-b8fd-884473d02d3b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.067755 Oct 15 20:50:05.156622 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 301/634] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:50:05.257626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:05.258767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:05.298753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:05.299267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:05.304034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:05.306917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:05.307539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:05.309198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:05.783800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:05.785818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:05.798970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:05.807911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:05.808429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:05.811160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:05.813337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:05.814019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:05.814183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c1ce710-f84b-4707-b5a2-1244d9df421a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.561250 Oct 15 20:50:05.814938 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 334/635] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:05 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:05.815561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:06.028931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:06.030424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:06.070265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:06.074768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:06.076978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:06.081353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:06.082331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:06.083290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:06.138096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:06.446445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:06.462124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:06.465553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:06.490384 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-436897af-9cec-43a3-beb9-0ff66a37ad81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.467001 Oct 15 20:50:06.505329 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 335/636] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:06 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:50:06.505329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:06.505329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:06.533610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:06.539257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:06.539854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:06.540689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:06.541205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:06.544892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:06.668298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:06.681687 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1b12e597-33df-4960-9dda-ebcd248c9c14 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.902135 Oct 15 20:50:06.687490 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 302/637] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:05 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 911 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:06.841350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:06.842016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:06.860739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:06.861278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:06.861722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:06.862438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:06.862893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:06.863314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:07.066811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:07.077284 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d36f894e-3464-4623-9451-c564a3ceba99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.581948 Oct 15 20:50:07.096077 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 336/638] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:06 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:07.239590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:07.253026 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1b40f4de-2ef9-4a7f-88cd-1339b04baba7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.415771 Oct 15 20:50:07.267212 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 303/639] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:06 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:07.289758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-cbeaf30e-6430-46ef-86b0-1a29d283ef8f req-9167be21-2c54-4e20-a0a3-b3f9e23bf076 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4d275e72-929a-469e-9cdd-c5b4c4968506", "tag": "374f7499-815c-480d-9b88-199bfe2d2340"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:07.305939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cbeaf30e-6430-46ef-86b0-1a29d283ef8f req-9167be21-2c54-4e20-a0a3-b3f9e23bf076 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:07.307073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cbeaf30e-6430-46ef-86b0-1a29d283ef8f req-9167be21-2c54-4e20-a0a3-b3f9e23bf076 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:07.311559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cbeaf30e-6430-46ef-86b0-1a29d283ef8f req-9167be21-2c54-4e20-a0a3-b3f9e23bf076 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:07.326435 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-cbeaf30e-6430-46ef-86b0-1a29d283ef8f req-9167be21-2c54-4e20-a0a3-b3f9e23bf076 service nova] Creating event network-changed:374f7499-815c-480d-9b88-199bfe2d2340 for instance 4d275e72-929a-469e-9cdd-c5b4c4968506 on nch1-master-1 Oct 15 20:50:07.331649 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-cbeaf30e-6430-46ef-86b0-1a29d283ef8f req-9167be21-2c54-4e20-a0a3-b3f9e23bf076 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049284 Oct 15 20:50:07.332484 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 304/640] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:07.519250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:07.524145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:07.525419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:07.548669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:07.560842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:07.561332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:07.562239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:07.562706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:07.563136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:07.635775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52e17207-8125-49fd-883e-74eca1ed7c9f tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe754706ca0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-851721492", "description": "tempest-description-272852173"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:07.692898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:07.701520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:07.703809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:07.722762 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-58a16b2b-1c2d-4ae1-9502-669f3f14a84e tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.204338 Oct 15 20:50:07.723826 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 305/641] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:07 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:50:07.728603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:07.729450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:07.735854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:07.736565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:07.737210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:07.738083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:07.738706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:07.739299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:07.955466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:07.977828 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8bd0b0a4-b85c-4020-b6db-fe6c20a93b85 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.252582 Oct 15 20:50:07.979604 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 306/642] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:07 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:08.106150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:08.106791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:08.122855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:08.123990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:08.125585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:08.126701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:08.127422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:08.135401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:08.310093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52e17207-8125-49fd-883e-74eca1ed7c9f tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 242 microversion: 2.1 time: 1.213387 Oct 15 20:50:08.317238 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 337/643] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:07 2022] POST /compute/v2.1/os-security-groups => generated 242 bytes in 1215 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:08.330308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:08.332560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:08.349915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:08.350852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:08.351838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:08.352897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:08.353694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:08.354215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:08.516967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:08.533968 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1cc410cf-1687-49a3-bab2-2d0630f50c13 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.431387 Oct 15 20:50:08.535569 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 307/644] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:08 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:08.557704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-41786bef-7544-405e-ace4-76a8a54c7e6e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe74fb44e50>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1522083607", "description": "tempest-description-642413525"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:08.757786 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-41786bef-7544-405e-ace4-76a8a54c7e6e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.204184 Oct 15 20:50:08.758657 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 308/645] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:08 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:08.777489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:08.790254 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6cd41c1f-8599-4638-b27e-a77e557789d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.473904 Oct 15 20:50:08.791135 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 338/646] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:08 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:08.808577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:08.809544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:08.829891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:08.830897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:08.832764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:08.835235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:08.835966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:08.838339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.003560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:09.013729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:09.016197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:09.021624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ec23197-7f9f-49f6-a62d-208863e04581 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.229290 Oct 15 20:50:09.022468 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 339/647] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:08 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:09.028989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bc6c6ca5-29ef-46af-bb97-124cd0ac2a1e tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '.version_select at 0x7fe74fb4e700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:09.044323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SecurityGroupsTestJSON-server-167383053", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "b3ffd293-16ae-49a6-a8ac-0b3025295760"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:09.121650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] validate_networks() for [('b3ffd293-16ae-49a6-a8ac-0b3025295760', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:50:09.131766 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bc6c6ca5-29ef-46af-bb97-124cd0ac2a1e tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8" status: 200 len: 371 microversion: 2.1 time: 0.371438 Oct 15 20:50:09.132868 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 309/648] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 20:50:08 2022] GET /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8 => generated 371 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:09.138796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:09.139848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:09.149097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.149841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.151359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.152466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.153204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.153857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.390801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:09.400327 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7a1df05e-f284-42f0-a6cd-e751b6ebcc3f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.265896 Oct 15 20:50:09.401814 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 310/649] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:09 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:09.411870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Action: 'update', calling method: .version_select at 0x7fe74ff12af0>, body: {"quota_set": {"ram": "5120"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:09.557846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.564628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.565415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.566727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.567395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.568013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.673001 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e8776421-08b7-438b-b83b-d66a8a253bf6 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8" status: 200 len: 328 microversion: 2.1 time: 0.269777 Oct 15 20:50:09.674994 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 311/650] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:50:09 2022] PUT /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8 => generated 328 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:09.680501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-08fd0a94-13aa-4f08-8076-518fa8ca218b req-baee4d05-c799-4380-b4c1-a5d254ee00de service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e84ed5b-8560-4a2c-a144-382839509e8f", "tag": "5531cb3d-9d57-432f-9e05-8b2e5b9f4831"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:09.693280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-08fd0a94-13aa-4f08-8076-518fa8ca218b req-baee4d05-c799-4380-b4c1-a5d254ee00de service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.694144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-08fd0a94-13aa-4f08-8076-518fa8ca218b req-baee4d05-c799-4380-b4c1-a5d254ee00de service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.694840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-08fd0a94-13aa-4f08-8076-518fa8ca218b req-baee4d05-c799-4380-b4c1-a5d254ee00de service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.709597 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-08fd0a94-13aa-4f08-8076-518fa8ca218b req-baee4d05-c799-4380-b4c1-a5d254ee00de service nova] Creating event network-changed:5531cb3d-9d57-432f-9e05-8b2e5b9f4831 for instance 0e84ed5b-8560-4a2c-a144-382839509e8f on nch2-master-1 Oct 15 20:50:09.715748 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-08fd0a94-13aa-4f08-8076-518fa8ca218b req-baee4d05-c799-4380-b4c1-a5d254ee00de service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039312 Oct 15 20:50:09.717237 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 312/651] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:09.726383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:09.727281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:09.735605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.736372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.737008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:09.737900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:09.738661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:09.739578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.083900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:10.084918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:10.085684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:10.086335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:10.086965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:10.098605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.099465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.100120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.101145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.101843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.102485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.122418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Getting quotas for project 869f1ee4c91c4067a3e32e39155b30a6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:50:10.129298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Getting quotas for user 03bc9235fe1c4fc1b3097a913cbe1052 and project 869f1ee4c91c4067a3e32e39155b30a6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:50:10.147763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:50:10.150730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:50:10.212323 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ccaf23c-e88d-4030-9e33-7989bbe5f666 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.173519 Oct 15 20:50:10.219665 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 340/652] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:50:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 1175 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:50:10.227276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb849d44-ecf5-441f-9a11-f2e45c98a387 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:10.243560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb849d44-ecf5-441f-9a11-f2e45c98a387 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8" status: 202 len: 0 microversion: 2.1 time: 0.021949 Oct 15 20:50:10.244586 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 341/653] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 20:50:10 2022] DELETE /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:10.252291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:10.254731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:10.262259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.263061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.264005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.282024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.282533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.282994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.415140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:10.442856 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d70b48e-9efa-4a62-85aa-55230b1874eb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.719767 Oct 15 20:50:10.444151 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 313/654] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:09 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:10.455347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:10.455997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:10.480649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.481188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.481632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.482387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.482806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.483238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.569082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:10.583558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-987b1f79-c6c6-4494-a106-aeef15214d5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.337116 Oct 15 20:50:10.585639 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 342/655] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:10 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:10.599973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95abdc9c-4d89-4dbd-b2cf-a1ab89090fed tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:10.600787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95abdc9c-4d89-4dbd-b2cf-a1ab89090fed tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:10.618974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:10.630984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:10.637683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:10.640706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:10.651765 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-093c817c-d0b7-4ad9-a80a-f5e7d48f1235 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.205047 Oct 15 20:50:10.652715 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 314/656] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:10 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:10.659003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4b479f54-a7b7-404f-bd2a-15788461db23 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '.version_select at 0x7fe74fb6e430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:10.682819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-95abdc9c-4d89-4dbd-b2cf-a1ab89090fed tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:10.699364 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95abdc9c-4d89-4dbd-b2cf-a1ab89090fed tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1215 microversion: 2.1 time: 0.111517 Oct 15 20:50:10.721758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 343/657] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:10 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1215 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:10.730647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:10.731829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:10.747344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.748063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.751176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.753238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:10.754506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:10.756407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:10.799295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:10.835523 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4b479f54-a7b7-404f-bd2a-15788461db23 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8" status: 200 len: 371 microversion: 2.1 time: 0.181251 Oct 15 20:50:10.836633 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 315/658] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 20:50:10 2022] GET /compute/v2.1/os-quota-sets/5f70e93455584211995b4a24d8cd71a8 => generated 371 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:11.018611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:11.029706 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90626dd8-1c74-4baf-b7cc-4a6b8ad1af24 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.306330 Oct 15 20:50:11.030501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 344/659] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:10 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:11.092513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28fec8bb-6b61-4549-b8be-f40a645631ed tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '.version_select at 0x7fe754231820>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:11.330876 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28fec8bb-6b61-4549-b8be-f40a645631ed tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306/defaults" status: 200 len: 371 microversion: 2.1 time: 0.242034 Oct 15 20:50:11.332144 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 316/660] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:11 2022] GET /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306/defaults => generated 371 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:11.479537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:11.480161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:11.497089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:11.498430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:11.500653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:11.504134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:11.506579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:11.508142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:11.610577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:11.611221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:11.627242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:11.627819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:11.628262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:11.628983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:11.629470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:11.629897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:11.961947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:11.977820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:11.978133 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-524ba0b8-b0dc-48a6-ac61-602c82cf0e5f tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.372167 Oct 15 20:50:11.979329 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 317/661] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:11 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:11.989356 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6109424-6b5d-4ef9-8588-0b4a6ff06763 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.519735 Oct 15 20:50:11.990959 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 345/662] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:11 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:11.998598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:12.000629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:12.001954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:12.003184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:12.025959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.027607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.027607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.027607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.028050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.028470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.028957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.031242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.032124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.033497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.037317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.038438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.171664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:12.186471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b6f7b8e-c350-4d9c-abe8-d74358b1844c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1267 microversion: 2.1 time: 0.194098 Oct 15 20:50:12.188392 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 346/663] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:11 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1267 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:12.199017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Action: 'update', calling method: .version_select at 0x7fe7544180d0>, body: {"quota_set": {"ram": "5120"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:12.295806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:12.307958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:12.319768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:12.335863 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a4526c8b-14b5-481c-b547-7f534222ec24 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.355249 Oct 15 20:50:12.344393 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 318/664] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:11 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:12.362828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:12.365650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:12.384006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.386882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.388673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.399268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.401334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.402940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.488813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.489645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.490116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.491113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.491852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.492286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.672830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da7896ed-d049-4cee-91c7-6bce98b719f3 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9" status: 200 len: 328 microversion: 2.1 time: 0.477005 Oct 15 20:50:12.673643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 347/665] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:50:12 2022] PUT /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9 => generated 328 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:12.700354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '.version_select at 0x7fe754818160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:12.857673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.858162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.868695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.869955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:12.870451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:12.870891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:12.913390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:12.923011 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6d4d7989-1281-46b9-978b-6f1de34966fb tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.576558 Oct 15 20:50:12.924262 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c13cd14-1207-4b41-8029-774e196d4ef8 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9/detail" status: 200 len: 916 microversion: 2.1 time: 0.240471 Oct 15 20:50:12.927582 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 319/666] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:12 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:12.945606 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 348/667] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:50:12 2022] GET /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9/detail => generated 916 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:13.016925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:13.018043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:13.024270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:13.025441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:13.036680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.038981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.043924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.047184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.049607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.049833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.050396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.051036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.051687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.052735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.056129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.061534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.392814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:13.405660 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0c0cc43e-521e-4122-8991-3916dcce8a06 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.392514 Oct 15 20:50:13.407153 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 320/668] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:13 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:13.425622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:13.426979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:13.437454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.438490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.439229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.440113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.440739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.441377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.513020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:13.522878 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c79df80a-8043-4aa2-92c4-ea799393981e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.503733 Oct 15 20:50:13.523464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 349/669] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:13 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:13.529194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Action: 'update', calling method: .version_select at 0x7fe74fec6280>, body: {"quota_set": {"ram": "2048"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:13.557852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:13.573973 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-86964fdf-75c3-4682-a10c-575d1136437b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1273 microversion: 2.1 time: 0.165515 Oct 15 20:50:13.577174 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 321/670] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:13 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1273 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:13.594076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:13.595834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:13.605525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.606249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.606911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.607859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.609214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.610028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.651594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.652905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.653605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.654692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.655666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.656502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.718137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.718137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.718624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.719267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.719728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.720178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.723652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:13.737940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:13.743400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:13.756217 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b251f507-61f8-4e94-bf6b-a9648783808b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1560 microversion: 2.91 time: 0.176483 Oct 15 20:50:13.756819 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 322/671] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:13 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:13.849840 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9713fdc-9624-470c-85f3-210a48b23d16 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9?user_id=0fc384ae0e2c446e93677acd6c342657" status: 200 len: 328 microversion: 2.1 time: 0.323979 Oct 15 20:50:13.858180 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 350/672] 10.222.0.16 () {60 vars in 1343 bytes} [Sat Oct 15 20:50:13 2022] PUT /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9?user_id=0fc384ae0e2c446e93677acd6c342657 => generated 328 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:13.864404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd732581-2bf9-42dd-9970-9e1559563b9b tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '.version_select at 0x7fe74fb44310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:13.953122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:13.954620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:13.961640 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cd732581-2bf9-42dd-9970-9e1559563b9b tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9?user_id=0fc384ae0e2c446e93677acd6c342657" status: 200 len: 370 microversion: 2.1 time: 0.101920 Oct 15 20:50:13.962476 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 323/673] 10.222.0.16 () {58 vars in 1323 bytes} [Sat Oct 15 20:50:13 2022] GET /compute/v2.1/os-quota-sets/7c6248db5a234d1f92f68484f827a9f9?user_id=0fc384ae0e2c446e93677acd6c342657 => generated 370 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:50:13.968511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.969551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.976774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:13.981987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:13.982486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:13.982933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:14.276180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:14.291847 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14ed1030-d9c8-4d2d-983a-ab33993d8712 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.344820 Oct 15 20:50:14.293164 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 351/674] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:13 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:14.440686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:14.442323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:14.454386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:14.457709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:14.458612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:14.459598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:14.463799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:14.464716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:14.503699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4f245d46-5306-4069-9bef-556e60dd6313 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Calling method '.version_select at 0x7fe7547f45e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:14.894100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=5,nova_api:DELETE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:14.896573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4f245d46-5306-4069-9bef-556e60dd6313 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306/defaults" status: 200 len: 371 microversion: 2.1 time: 0.396793 Oct 15 20:50:14.897276 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 352/675] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:14 2022] GET /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306/defaults => generated 371 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:14.902726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:14.911385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:14.932795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:14.933544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:14.935696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:14.938701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:14.940346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:14.940948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.257544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:15.265920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:15.273726 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2c8e984e-45ee-4af9-a169-4b5d8d1943a6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.845879 Oct 15 20:50:15.277968 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 324/676] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:14 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 852 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:15.288725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:15.290089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:15.299368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-630bb756-c917-494f-b1c4-9a5c160c8ade tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.400528 Oct 15 20:50:15.300044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.301183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.301946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.302890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.303576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.304220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.321932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 353/677] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:14 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:15.327102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:15.327720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:15.351345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.351859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.352305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.353275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.354585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.355195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.521665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:15.535489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:15.540691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:15.554985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5761bc61-a5f0-4294-906f-c10d9f321119 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.231567 Oct 15 20:50:15.578695 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 354/678] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:15 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:15.589647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Action: 'update', calling method: .version_select at 0x7fe7546e03a0>, body: {"quota_set": {"force": true, "metadata_items": 256, "ram": 10240, "key_pairs": 200, "instances": 20, "server_groups": 20, "server_group_members": 20, "cores": 2, "fixed_ips": 10, "floating_ips": 20, "security_group_rules": 20, "security_groups": 20, "injected_file_content_bytes": 20480, "injected_file_path_bytes": 512, "injected_files": 10}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:15.719004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:15.728210 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-499a671b-5225-4cc9-beff-761394374404 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.446445 Oct 15 20:50:15.735609 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 325/679] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:15 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:15.741089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:15.747292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:15.755368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.768431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.768931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.769398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.777850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.778661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.780624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.781230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:15.781935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.782621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:15.783272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:15.784235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:16.260125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5f1e8b65-d7fe-4b15-bf5b-c5a015d07ac7 tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306" status: 200 len: 329 microversion: 2.1 time: 0.679368 Oct 15 20:50:16.260790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 355/680] 10.222.0.16 () {60 vars in 1263 bytes} [Sat Oct 15 20:50:15 2022] PUT /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306 => generated 329 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:16.271871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d09f7a7e-a3d8-453f-af5a-a752858a91b1 req-b3cd1f20-fb4e-41ab-bbf5-292fa3e993ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "72d34005-b40a-4e73-8f98-b0a6d2d0106b", "tag": "54d23914-40e3-4e01-9162-9682f70a408f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:16.289949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d09f7a7e-a3d8-453f-af5a-a752858a91b1 req-b3cd1f20-fb4e-41ab-bbf5-292fa3e993ba service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:16.290437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d09f7a7e-a3d8-453f-af5a-a752858a91b1 req-b3cd1f20-fb4e-41ab-bbf5-292fa3e993ba service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:16.290872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d09f7a7e-a3d8-453f-af5a-a752858a91b1 req-b3cd1f20-fb4e-41ab-bbf5-292fa3e993ba service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:16.302669 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d09f7a7e-a3d8-453f-af5a-a752858a91b1 req-b3cd1f20-fb4e-41ab-bbf5-292fa3e993ba service nova] Creating event network-changed:54d23914-40e3-4e01-9162-9682f70a408f for instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b on nch2-master-1 Oct 15 20:50:16.309177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d09f7a7e-a3d8-453f-af5a-a752858a91b1 req-b3cd1f20-fb4e-41ab-bbf5-292fa3e993ba service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046413 Oct 15 20:50:16.310139 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 356/681] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:16.316216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Action: 'update', calling method: .version_select at 0x7fe754590160>, body: {"quota_set": {"cores": 20, "fixed_ips": -1, "floating_ips": -1, "injected_file_content_bytes": 10240, "injected_file_path_bytes": 255, "injected_files": 5, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "security_group_rules": -1, "security_groups": -1, "server_group_members": 10, "server_groups": 10}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:16.398336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:16.398840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:16.399282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:16.400019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:16.400481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:16.400911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:16.497470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:16.518402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bfa0cef4-936f-4507-845a-6c452a21179e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.780505 Oct 15 20:50:16.521780 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 326/682] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:15 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:16.554866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:16.558743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:16.575111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:16.577387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:16.588357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:16.589895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:16.590617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:16.591279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:16.889277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=25,nova_api:UPDATE=10 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:16.901694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:17.089265 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6be806b8-c195-4fa4-a52c-70b042f5d0cc tempest-QuotasAdminTestJSON-1556686450 tempest-QuotasAdminTestJSON-1556686450-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306" status: 200 len: 329 microversion: 2.1 time: 0.777699 Oct 15 20:50:17.090303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 357/683] 10.222.0.16 () {60 vars in 1263 bytes} [Sat Oct 15 20:50:16 2022] PUT /compute/v2.1/os-quota-sets/358f9c122d6c4f49ad13d0d08e224306 => generated 329 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:17.106565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:17.107481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:17.117412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.118220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.124719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.125788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.126448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.127211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.394115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:17.411505 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b15924b7-55f9-48d6-b5cb-71ca6544d272 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.880215 Oct 15 20:50:17.414569 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 327/684] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:16 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 887 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:17.439241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:17.441709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:17.452101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.453085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.453849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.456023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.456961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.457929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.476828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:17.499829 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-456972bb-a338-4ea9-8234-23b410cc5139 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.408255 Oct 15 20:50:17.501755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 358/685] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:17 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:17.515269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:17.517166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:17.528239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.529164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.530553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.531534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.532278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.532931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.695202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:17.708670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:17.723354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:17.752169 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-123caf10-a505-4396-bbdd-d5cce82767ef tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.334383 Oct 15 20:50:17.767485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:17.777612 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c0b5475-20a6-45df-b3f5-1056dbfb20d6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.273138 Oct 15 20:50:17.778481 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 359/686] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:17 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:17.783031 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 328/687] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:17 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:17.795157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:17.796331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:17.803939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4c050a50-2dda-4071-8d29-5fa2be1d160d req-75022312-c67c-419d-b52d-2f18c28707b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e84ed5b-8560-4a2c-a144-382839509e8f", "name": "network-vif-plugged", "status": "completed", "tag": "5531cb3d-9d57-432f-9e05-8b2e5b9f4831"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:17.817119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4c050a50-2dda-4071-8d29-5fa2be1d160d req-75022312-c67c-419d-b52d-2f18c28707b5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.818734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4c050a50-2dda-4071-8d29-5fa2be1d160d req-75022312-c67c-419d-b52d-2f18c28707b5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.818734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4c050a50-2dda-4071-8d29-5fa2be1d160d req-75022312-c67c-419d-b52d-2f18c28707b5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.832349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:17.849495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.850106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.850571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.851314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:17.851775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:17.852214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:17.872103 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4c050a50-2dda-4071-8d29-5fa2be1d160d req-75022312-c67c-419d-b52d-2f18c28707b5 service nova] Creating event network-vif-plugged:5531cb3d-9d57-432f-9e05-8b2e5b9f4831 for instance 0e84ed5b-8560-4a2c-a144-382839509e8f on nch2-master-1 Oct 15 20:50:17.895011 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4c050a50-2dda-4071-8d29-5fa2be1d160d req-75022312-c67c-419d-b52d-2f18c28707b5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100358 Oct 15 20:50:17.895666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 360/688] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:18.190574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:18.200154 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a672b2ef-d154-4079-96ce-cd02df7087cd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.408883 Oct 15 20:50:18.200747 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 329/689] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:17 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:18.444795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:18.445956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:18.480094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.484418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.488124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.489766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.495150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.495987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.527675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:18.528615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:18.540232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.541108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.541631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.542529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.543071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.543575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.850301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:18.862984 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7df3d1ca-791d-4b6c-b84d-c2fd50d95eea tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 0.339713 Oct 15 20:50:18.864179 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 330/690] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:18 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:18.869380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:18.870282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:18.878835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.881657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.882730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.884549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.886092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.886795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.921066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:18.934891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2750cfc3-fea8-4448-837c-eb5818d88e95 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.496293 Oct 15 20:50:18.935471 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 361/691] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:18 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:18.945173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:18.945860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:18.974289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.974788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.975228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:18.975937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:18.976395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:18.976818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:19.040742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:19.053261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:19.068940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:19.083454 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-85ed3f8a-be8f-448e-93c8-0a5456fd4ea2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.216412 Oct 15 20:50:19.087090 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 331/692] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:18 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:19.225168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:19.226447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:19.236126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:19.237901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:19.240147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:19.241157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:19.243617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:19.244277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:19.305054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:19.315181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d97397f-e656-4445-8cde-f0735519c378 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.378091 Oct 15 20:50:19.330210 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 362/693] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:18 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:19.462283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:19.479323 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5ec5a781-4e08-4b0e-9d92-cfb0c37604c5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.258023 Oct 15 20:50:19.480579 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 332/694] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:19 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:19.891797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:19.892414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:19.913938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:19.914437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:19.917388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:19.918146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:19.918644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:19.919070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:19.966220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:19.967587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:19.977457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:19.982107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:19.982765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:19.983821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:19.984971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:19.985505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:19.986090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:20.316708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:20.329643 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-afdfd8e3-1afa-43b0-9f92-366c6e4ea99a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.368979 Oct 15 20:50:20.352634 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 333/695] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:19 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:20.365508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:20.368851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:20.389212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:20.392042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:20.394409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:20.395552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:20.396226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:20.397221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:20.445261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:20.549110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:20.582685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:20.585385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:20.600207 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3b262b88-80b8-42b9-b4a8-b8e0d8c95d71 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.245666 Oct 15 20:50:20.600810 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 334/696] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:20 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:20.611211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:20.611852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:20.629135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:20.629642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:20.630082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:20.630819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:20.641832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:20.642497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:20.938367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:20.955814 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c80f966b-3854-4db2-a798-43c72d67ff8b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.353827 Oct 15 20:50:20.957437 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 335/697] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:20 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:20.982219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:20.984237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:21.006797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.007678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.008564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.021550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.022948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.023637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.298415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:21.306726 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fac79c54-0dde-4159-b9cc-fecbcc4ebcf2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1504 microversion: 2.1 time: 1.418736 Oct 15 20:50:21.317397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 363/698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:19 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1504 bytes in 1430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:21.360745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:21.362372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:21.380289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.390286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.394641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.396392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.402461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.407987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.532752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:21.542089 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fde167fa-5d2a-4d77-bc71-6f5f9f364b60 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.570124 Oct 15 20:50:21.563510 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 336/699] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:20 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:21.634092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:21.634762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:21.649349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.649855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.657273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.661754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.662322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.665675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.893510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:21.910443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27bf8efe-cd7e-4473-bda7-0cba549b4cf6 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.545190 Oct 15 20:50:21.911680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 364/700] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:21 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:21.963870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:21.976784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:21.984659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:21.985601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:21.985743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:21.992591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.993754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.994356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:21.995401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:21.996035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:21.996621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:22.006820 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-42749c84-dd8c-428b-b93d-913b09cc2c61 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.373721 Oct 15 20:50:22.010624 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 337/701] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:21 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:22.340558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:22.341310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:22.367294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:22.367798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:22.368243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:22.371583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:22.372782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:22.374410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:22.519389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:22.528583 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da0bd947-0746-4044-8c49-0004a9b8f533 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.548568 Oct 15 20:50:22.549075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 365/702] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:21 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:22.573615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:22.574243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:22.596508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:22.597020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:22.597496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:22.600899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:22.601894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:22.603755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.012606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:23.023144 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-54142dab-d561-499a-ae2a-134890aa4f1f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1494 microversion: 2.1 time: 0.690718 Oct 15 20:50:23.033069 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 338/703] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:22 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1494 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:23.033069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:23.033069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:23.050248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.056007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.056912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.059883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.069782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.070898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.143175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:23.153351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3117f7f1-9937-4052-8983-834a8a84ebf3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1312 microversion: 2.1 time: 0.584521 Oct 15 20:50:23.165483 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 366/704] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:22 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1312 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:23.165483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86093df2-5129-453e-9599-92bee22bfb9e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '.version_select at 0x7fe75425d310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:23.166128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86093df2-5129-453e-9599-92bee22bfb9e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.171017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86093df2-5129-453e-9599-92bee22bfb9e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.171393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86093df2-5129-453e-9599-92bee22bfb9e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.183031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86093df2-5129-453e-9599-92bee22bfb9e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 791 microversion: 2.1 time: 0.023920 Oct 15 20:50:23.184259 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 367/705] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:50:23 2022] GET /compute/v2.1/os-migrations => generated 791 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:23.193832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:23.195781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:23.206764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.207338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.207789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.208558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.209175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.209618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.392910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:23.402689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:23.404155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:23.424258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5171975-26d7-49f5-b6d5-30d75307589a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.238606 Oct 15 20:50:23.425165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 368/706] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:23 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:23.440099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:23.441611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:23.456560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.457090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.457716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.458625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.459257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.465839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.626168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:23.641488 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6fc8617c-d25c-4e3f-89b1-991326651260 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.610587 Oct 15 20:50:23.652984 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 339/707] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:23 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:23.679130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:23.679756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:23.698807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.700725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.703921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.707731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:23.710422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:23.719221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:23.924696 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-283e4919-bc31-4a89-aadc-d89c926e7a5a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1645 microversion: 2.1 time: 0.491006 Oct 15 20:50:23.945569 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 369/708] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:23 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1645 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:23.950052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:24.114052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:24.126497 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ece7cf7-88ca-4b89-ab13-37db9e2a6564 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.455496 Oct 15 20:50:24.141082 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 340/709] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:23 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:24.180407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:24.182381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:24.198347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:24.199071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:24.199306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:24.222368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:24.222943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:24.223445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:24.252884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-1226692919", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "6bc761ff-f597-43f8-85e3-57edd2825dc3"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:24.386380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] validate_networks() for [('6bc761ff-f597-43f8-85e3-57edd2825dc3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:50:24.597465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:24.610766 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a15875e3-63b7-44bd-8f46-b51cb217f36f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1462 microversion: 2.1 time: 0.434298 Oct 15 20:50:24.614994 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 370/710] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:24 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1462 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:24.627164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:24.628560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:24.648964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:24.659075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:24.659855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:24.660786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:24.661655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:24.663607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:24.775108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:24.787014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:24.788780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:24.797074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9dd62eb4-7aca-42c3-9990-6e0504cb0364 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.179807 Oct 15 20:50:24.797921 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 371/711] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:24 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:24.804054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:24.808747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:24.825276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:24.827032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:24.828388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:24.829622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:24.830490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:24.831539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.057788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:25.068153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Old instance type m1.heat2, new instance type m1.heat {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 20:50:25.081014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:25.082097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:25.083029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:25.083663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:25.086054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:25.268874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4e64011-a49a-4cb4-bc65-53f722e15c84 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f/action" status: 202 len: 0 microversion: 2.1 time: 0.469408 Oct 15 20:50:25.269799 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 372/712] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:24 2022] POST /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f/action => generated 0 bytes in 471 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:25.275243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:25.276098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:25.284185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:25.284826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:25.285452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.286407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:25.287133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:25.287751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.574673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:25.591406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b7d836d0-f7f1-4508-947c-3a16c26f6fe9 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1472 microversion: 2.1 time: 0.317821 Oct 15 20:50:25.593415 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 373/713] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:25 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1472 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:25.610583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:25.611529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:25.638158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:25.643598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:25.644579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.650566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:25.651310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:25.653652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.888404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:25.891064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:25.893087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:25.893731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:25.894440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:25.925305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:25.927185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:25.928431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.931844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:25.932544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:25.933253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:25.997865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:50:26.018638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for user c613201f55b94a528c6c36e5b5772799 and project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:50:26.081777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:50:26.091153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:50:26.104889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:26.130019 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c200141-0d29-4ffc-be6a-887f756798ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.533170 Oct 15 20:50:26.147851 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 374/714] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:25 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:26.159918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:26.161735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:26.178400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:26.181338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:26.182672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:26.184094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:26.186154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:26.188010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:26.267769 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-97462b36-20fe-4cd0-a5b5-4cd081d0fac4 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.024039 Oct 15 20:50:26.274455 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 341/715] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:50:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 2031 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:50:26.293304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:26.295799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:26.334889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:26.337627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:26.338079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:26.346579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:26.347391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:26.348382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:26.411315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=17 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:26.566905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:26.581248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:26.590060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:26.607161 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c007d586-2353-49e5-9e13-2fbbfcc30601 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.328196 Oct 15 20:50:26.609114 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 342/716] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:26 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:26.614831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:26.623370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec055d0f-2d8d-40e9-b499-56c292f671f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:26.626558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec055d0f-2d8d-40e9-b499-56c292f671f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:26.629970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99c18f5d-bff7-45ce-b93f-a623644bd2e5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1471 microversion: 2.1 time: 0.479811 Oct 15 20:50:26.632755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 375/717] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:26 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1471 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:26.647099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:26.649186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:26.662867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:26.663722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:26.665609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:26.666497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:26.667812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:26.669911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.039082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:27.042865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:27.061093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fa512eb-a9cf-4fe5-983c-f095aee41bec tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.423492 Oct 15 20:50:27.065179 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 376/718] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:26 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:27.154914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:27.155710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:27.167954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:27.179557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.180305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.183412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.183920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec055d0f-2d8d-40e9-b499-56c292f671f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:27.184385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.184385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.184385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.190318 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec055d0f-2d8d-40e9-b499-56c292f671f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1211 microversion: 2.1 time: 0.577147 Oct 15 20:50:27.195483 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 343/719] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:26 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1211 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:27.523207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:27.540624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc3f43cc-37a6-4cba-8915-30ca56e684f8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.390245 Oct 15 20:50:27.541619 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 377/720] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:27 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:27.636858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-15221089-0ab0-4e90-a5eb-2b63e426b7c5 req-5d79e66e-4677-425e-b459-9109e0ba6e94 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "72d34005-b40a-4e73-8f98-b0a6d2d0106b", "name": "network-vif-plugged", "status": "completed", "tag": "54d23914-40e3-4e01-9162-9682f70a408f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:27.647896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:27.653887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:27.681885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.683444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.686089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.688486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.690663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.691909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-15221089-0ab0-4e90-a5eb-2b63e426b7c5 req-5d79e66e-4677-425e-b459-9109e0ba6e94 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.694843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-15221089-0ab0-4e90-a5eb-2b63e426b7c5 req-5d79e66e-4677-425e-b459-9109e0ba6e94 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.695790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.697720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-15221089-0ab0-4e90-a5eb-2b63e426b7c5 req-5d79e66e-4677-425e-b459-9109e0ba6e94 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.756017 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-15221089-0ab0-4e90-a5eb-2b63e426b7c5 req-5d79e66e-4677-425e-b459-9109e0ba6e94 service nova] Creating event network-vif-plugged:54d23914-40e3-4e01-9162-9682f70a408f for instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b on nch2-master-1 Oct 15 20:50:27.784720 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-15221089-0ab0-4e90-a5eb-2b63e426b7c5 req-5d79e66e-4677-425e-b459-9109e0ba6e94 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.148053 Oct 15 20:50:27.787482 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 344/721] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:27.796918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:27.818791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:27.846832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.847633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.848298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.849593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:27.850339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:27.851057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:27.948636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:27.963747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:27.978539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:27.999065 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9bbc4f07-c7ff-46be-b3ef-473426e33cca tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.358263 Oct 15 20:50:28.009861 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 378/722] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:27 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:28.091091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:28.092268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:28.109265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:28.110025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:28.110478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:28.111301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:28.111939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:28.112508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:28.402281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:28.415942 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03ddf795-3855-4971-bab5-0deedacb6c40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.327801 Oct 15 20:50:28.416770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 379/723] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:28 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:28.426173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:28.430933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:28.446788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:28.448942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:28.450302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:28.453690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:28.454473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:28.457299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:28.569788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:28.586110 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78c90033-c264-47c3-8d57-9df44c9eeb3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1471 microversion: 2.1 time: 0.794734 Oct 15 20:50:28.587658 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 345/724] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:27 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1471 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:28.597517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:28.598371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:28.612603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:28.615276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:28.616124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:28.616966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:28.619665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:28.620321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:28.621871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f592aee-09da-423e-8f2a-f2f57407bf08 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1263 microversion: 2.1 time: 0.203448 Oct 15 20:50:28.622456 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 380/725] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:28 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1263 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:28.633337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.047903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:29.056521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:29.094147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.096126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.101028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.103233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.103621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.104064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.274310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:29.283993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:29.285748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:29.292544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f2aa4d1-16b8-4216-a4cf-552b54e0cfdc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.255971 Oct 15 20:50:29.305715 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 381/726] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:29 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:29.365598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:29.388038 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-03f42f71-936c-44d1-94c9-1e6ad4f12eee tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.793962 Oct 15 20:50:29.388762 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 346/727] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:28 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:29.442193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:29.443370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:29.456008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.456242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.456583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.457332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.457801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.458230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.614504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:29.616082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:29.617288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:29.625906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.633437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.634410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.635379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.635997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.637673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:29.943778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:29.952792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-947a65f0-7204-4f21-915a-10d0aecf4583 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.515799 Oct 15 20:50:29.953410 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 382/728] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:29 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:29.990738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:29.991557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:29.997764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:29.998409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:29.998954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:30.000623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:30.005988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:30.006455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:30.217190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:30.239087 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4cea3c5f-3b17-432e-8d15-68cf51a4d1f8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1269 microversion: 2.1 time: 0.263778 Oct 15 20:50:30.239535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:30.256114 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5c518782-00f4-4c52-ad37-80db0ad555a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1471 microversion: 2.1 time: 0.645625 Oct 15 20:50:30.265147 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 383/729] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:29 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1269 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:30.273209 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 347/730] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:29 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1471 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:30.325001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:30.326415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:30.335613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:30.336282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:30.336887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:30.337821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:30.338656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:30.339282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:30.414096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:30.414809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:30.443428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:30.451623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:30.452241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:30.453057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:30.454407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:30.455078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:30.550409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:30.563119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:30.567765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:30.583253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a2aaa22-41e7-4418-bd76-99d2f79485b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.264678 Oct 15 20:50:30.611170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 384/731] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:30 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:30.961963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:30.983029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:30.983514 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f1c7c380-1cc6-4f93-8ea8-919aed7916a2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1315 microversion: 2.1 time: 0.569877 Oct 15 20:50:30.983668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:30.984766 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 348/732] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:30 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1315 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:30.993224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.000315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.000788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.002791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.003280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.005209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.265640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:31.266269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:31.283923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.285712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.290113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.291940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.295643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.298005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.299436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:31.309550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22c0e2d4-2688-47e8-b628-1a631855d629 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.331508 Oct 15 20:50:31.328058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 385/733] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:30 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:31.346992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:31.352581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:31.384311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.387361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.390146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.391124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.395813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.396521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.598579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:31.613552 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-816ab6fe-25a2-4152-b5da-9f18fd3790d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1269 microversion: 2.1 time: 0.352966 Oct 15 20:50:31.614989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 349/734] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:31 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1269 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:31.639483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:31.642783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:31.652167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.653482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.654764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:31.656206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:31.682278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:31.683626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:32.121762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:32.142682 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5f0e360-3386-424b-ab1e-22333b3b2b1b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1471 microversion: 2.1 time: 0.802459 Oct 15 20:50:32.145544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 386/735] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:31 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1471 bytes in 817 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:32.161803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:32.162411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:32.171909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:32.176354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:32.177089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:32.178357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:32.178987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:32.188704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:32.504916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:32.517348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9515ae74-1352-491c-91fe-f2515b7edd36 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1503 microversion: 2.1 time: 0.359127 Oct 15 20:50:32.518263 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 387/736] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:32 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1503 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:32.538515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:32.541826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:32.550941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:32.552091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:32.552772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:32.553690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:32.554364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:32.555618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:32.675888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:32.696588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:32.701673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:32.718495 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69818a4b-ad4a-45c6-9cff-bdb80be04a7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 1.102054 Oct 15 20:50:32.721250 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 350/737] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:31 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 1105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:32.793964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:32.808597 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe4f03f6-86db-4af1-b829-763c3bbd248c tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.288348 Oct 15 20:50:32.812924 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 388/738] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:32 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:32.828096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-851721492"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:32.828930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:32.847513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:32.848946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:32.852403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:32.853993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:32.854892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:32.859669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:33.021088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1414049235", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:33.140910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.network.security_group_api [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Adding security group 818202c3-2675-4802-86fc-077c2a1d884e to port 54d23914-40e3-4e01-9162-9682f70a408f Oct 15 20:50:33.177773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:50:33.696158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e76a940-38b3-42b7-aad6-ad13744c8b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b/action" status: 202 len: 0 microversion: 2.1 time: 0.881845 Oct 15 20:50:33.698079 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 389/739] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:32 2022] POST /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b/action => generated 0 bytes in 884 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:33.704051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:33.704882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:33.721137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:33.721872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:33.722745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:33.723684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:33.726594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:33.730855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:33.834615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:33.835680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:33.836367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:33.836985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:33.837606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:33.858160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:33.858983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:33.859616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:33.865365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:33.869722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:33.871317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:33.928224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:50:33.946619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:50:33.960608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:33.971785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6108be18-ed69-4c56-85df-f5395c45bd85 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.272290 Oct 15 20:50:33.978183 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 390/740] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:33 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:33.988387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:33.990176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:33.997023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:50:34.010413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:50:34.011943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.014815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.020352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.021203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.024057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.024552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.144440 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5355ff89-ae07-4a57-be54-49f9bcfacca9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.421892 Oct 15 20:50:34.145994 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 351/741] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:50:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 1424 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:50:34.152352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1522083607"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:34.153472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:34.164267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.165718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.166341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.167268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.167890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.168479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.363805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:34.381014 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bcc70645-8a2b-45df-b86f-a08eacef1084 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.401438 Oct 15 20:50:34.381675 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 391/742] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:33 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:34.393249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d8e016ce-09c9-4075-9217-004a36d0f147 req-e84ea814-b802-4fdb-b65e-4773bf8a6f8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "72d34005-b40a-4e73-8f98-b0a6d2d0106b", "tag": "54d23914-40e3-4e01-9162-9682f70a408f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:34.403711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d8e016ce-09c9-4075-9217-004a36d0f147 req-e84ea814-b802-4fdb-b65e-4773bf8a6f8e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.404355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d8e016ce-09c9-4075-9217-004a36d0f147 req-e84ea814-b802-4fdb-b65e-4773bf8a6f8e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.404958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d8e016ce-09c9-4075-9217-004a36d0f147 req-e84ea814-b802-4fdb-b65e-4773bf8a6f8e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.422851 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d8e016ce-09c9-4075-9217-004a36d0f147 req-e84ea814-b802-4fdb-b65e-4773bf8a6f8e service nova] Creating event network-changed:54d23914-40e3-4e01-9162-9682f70a408f for instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b on nch2-master-1 Oct 15 20:50:34.428413 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d8e016ce-09c9-4075-9217-004a36d0f147 req-e84ea814-b802-4fdb-b65e-4773bf8a6f8e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043744 Oct 15 20:50:34.429049 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 392/743] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:34.436123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:34.447502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:34.455884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:UPDATE=4,nova_api:SELECT=93,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:34.458276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.458782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.459254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.460989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.461906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.463325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.538033 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.network.security_group_api [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Adding security group 35000ccc-1243-4d65-b5b9-450ce2755579 to port 54d23914-40e3-4e01-9162-9682f70a408f Oct 15 20:50:34.611903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:34.627025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:34.631870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:34.652634 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3eef04d7-0c4a-4f0c-baed-46b476aab112 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.218928 Oct 15 20:50:34.654982 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 393/744] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:34 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:34.672857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:34.674918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:34.704939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.711323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.715167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.716184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:34.721446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:34.722272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:34.801281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:35.133117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:35.155274 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df73333c-c0eb-445c-a596-6f0805e890cb tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.493772 Oct 15 20:50:35.155971 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 394/745] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:34 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:35.178019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:35.179336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:35.220557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:35.221367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:35.226979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:35.229936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:35.231848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:35.233809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:35.430806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:35.457252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f1f2e80-8a18-4eee-8899-e1f1cc747b8d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1221 microversion: 2.1 time: 0.298266 Oct 15 20:50:35.458249 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 395/746] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:35 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1221 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:35.626363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-142889", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "9c7544ec-ea93-44b4-996f-09fc127b83d3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:35.662823 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec90d05e-ee67-4bf2-9fd8-43edcad93c5e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b/action" status: 202 len: 0 microversion: 2.1 time: 1.515509 Oct 15 20:50:35.666538 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 352/747] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:34 2022] POST /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b/action => generated 0 bytes in 1520 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:35.672699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e9ecc80c-5ac2-45da-8759-24e7e5af863c req-99c28ef1-5f40-4c81-8554-48a628773fe8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d1ddd46-47f6-4948-a897-3b963780991e", "name": "network-vif-plugged", "status": "completed", "tag": "7d14c08a-67ff-4f01-a95d-0c19d04975e3"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:35.703831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e9ecc80c-5ac2-45da-8759-24e7e5af863c req-99c28ef1-5f40-4c81-8554-48a628773fe8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:35.708227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e9ecc80c-5ac2-45da-8759-24e7e5af863c req-99c28ef1-5f40-4c81-8554-48a628773fe8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:35.709166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e9ecc80c-5ac2-45da-8759-24e7e5af863c req-99c28ef1-5f40-4c81-8554-48a628773fe8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:35.746372 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e9ecc80c-5ac2-45da-8759-24e7e5af863c req-99c28ef1-5f40-4c81-8554-48a628773fe8 service nova] Creating event network-vif-plugged:7d14c08a-67ff-4f01-a95d-0c19d04975e3 for instance 5d1ddd46-47f6-4948-a897-3b963780991e on nch2-master-1 Oct 15 20:50:35.756800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] validate_networks() for [('9c7544ec-ea93-44b4-996f-09fc127b83d3', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:50:35.793771 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e9ecc80c-5ac2-45da-8759-24e7e5af863c req-99c28ef1-5f40-4c81-8554-48a628773fe8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.121226 Oct 15 20:50:35.794440 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 353/748] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:35.806729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:35.807825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:35.818484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:35.823363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:35.824237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:35.825447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:35.826864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:35.827516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:36.599110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:36.612171 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aea511f7-b318-400f-a146-eecf6ae702e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.816164 Oct 15 20:50:36.613184 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 354/749] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:35 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:36.626459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:36.627390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:36.637260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:36.641174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:36.641930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:36.642894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:36.645526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:36.646188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:36.654684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:36.986986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:36.988329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:36.989022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:36.989770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:36.990390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:37.007880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.008902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.009631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.010602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.013670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.014408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.062079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for project f400a79cb647438f968a895142c5410d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:50:37.077936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for user 180d9b7141ff4792ab7ab19e21a3d7ac and project f400a79cb647438f968a895142c5410d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:50:37.110553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:50:37.115497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:50:37.126553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:37.136952 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f902dd1-e27a-4895-8926-9658b17d9b18 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.522483 Oct 15 20:50:37.151846 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 355/750] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:36 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:37.170815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-984015a2-36cd-4505-981d-220e3c8a0799 req-7257a16a-f714-48b0-b6c0-c1343be5b768 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ee567153-18c3-4c9b-a3eb-a9c13b250fe3", "tag": "0ac06a96-2831-4954-a5e4-8d904f1d0eb6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:37.202256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-984015a2-36cd-4505-981d-220e3c8a0799 req-7257a16a-f714-48b0-b6c0-c1343be5b768 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.202798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-984015a2-36cd-4505-981d-220e3c8a0799 req-7257a16a-f714-48b0-b6c0-c1343be5b768 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.203549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-984015a2-36cd-4505-981d-220e3c8a0799 req-7257a16a-f714-48b0-b6c0-c1343be5b768 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.252044 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-984015a2-36cd-4505-981d-220e3c8a0799 req-7257a16a-f714-48b0-b6c0-c1343be5b768 service nova] Creating event network-changed:0ac06a96-2831-4954-a5e4-8d904f1d0eb6 for instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 on nch1-master-1 Oct 15 20:50:37.256695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ffeb771-804f-4caf-9e77-f16a1f303458 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 1.784852 Oct 15 20:50:37.263723 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 396/751] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:50:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 1800 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:50:37.266385 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-984015a2-36cd-4505-981d-220e3c8a0799 req-7257a16a-f714-48b0-b6c0-c1343be5b768 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106263 Oct 15 20:50:37.267410 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 356/752] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:37.271517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:37.272023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:37.276046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:37.277402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:37.286707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.287259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.288658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.289608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.290200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.291110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.310325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.310847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.311291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.312003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.312460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.312882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.408462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:37.417286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:37.418736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:37.438899 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d3bf59d-d19f-4f1f-82ae-d8ee2a94746a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.170182 Oct 15 20:50:37.448222 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 357/753] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:37 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:37.453540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:37.454154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:37.465668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.465668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.465963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.469229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.469601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.470044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.709232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:37.727173 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-becfa441-aa12-4963-9446-ccb4664830d4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1478 microversion: 2.1 time: 0.277508 Oct 15 20:50:37.737071 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 358/754] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:37 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1478 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:37.737413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:37.745413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:37.760178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.760805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.761375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.762197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:37.767409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:37.768846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:37.909256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:37.927898 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c0029df4-309c-4a95-92b8-dffad1902f97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1275 microversion: 2.1 time: 0.198543 Oct 15 20:50:37.929376 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 359/755] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:37 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1275 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:37.942922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d11c32d-5753-4b30-b3fe-8a2a4e6a5b37 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:37.943573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3d11c32d-5753-4b30-b3fe-8a2a4e6a5b37 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:38.046065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d11c32d-5753-4b30-b3fe-8a2a4e6a5b37 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:38.054617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d11c32d-5753-4b30-b3fe-8a2a4e6a5b37 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:38.057474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d11c32d-5753-4b30-b3fe-8a2a4e6a5b37 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:38.058783 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3d11c32d-5753-4b30-b3fe-8a2a4e6a5b37 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1258 microversion: 2.25 time: 0.128193 Oct 15 20:50:38.076464 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 360/756] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:37 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1258 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:38.088916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:38.089696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:38.098516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:38.099159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:38.099724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:38.100621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:38.101262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:38.102943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:38.352491 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-438d271d-a198-4a67-a9e1-991dcc8b6df5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b/os-security-groups" status: 200 len: 1156 microversion: 2.1 time: 1.089640 Oct 15 20:50:38.353375 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 397/757] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 20:50:37 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b/os-security-groups => generated 1156 bytes in 1094 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:38.369993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:38.373333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:38.400798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:38.403531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:38.406279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:38.408826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:38.411269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:38.411993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:38.731980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:38.748769 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1e971791-9773-4225-8268-d7fdb042aff9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.669565 Oct 15 20:50:38.749419 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 361/758] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:38 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 672 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:38.760749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8f6c6bcd-2e41-421a-9c29-74f93a41c273 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe74fb444c0>, body: {"security_group": {"name": " tempest-securitygroup -1194047773 ", "description": "tempest-description-858218579"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:38.918424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:38.931104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fbe1e69-cb05-4abb-9718-23f769191f5c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.574016 Oct 15 20:50:38.933978 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 398/759] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:38 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:38.942955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:38.945731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:38.960315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:38.962105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:38.962630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:38.965890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:38.968757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:38.969351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:39.140070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:39.152213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:39.155589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:39.169578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ff6ac830-3d69-4a6a-bba8-d860960f2c18 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.232077 Oct 15 20:50:39.171985 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 399/760] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:38 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:39.177820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:39.178721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:39.212937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:39.213459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:39.214299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:39.216070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:39.216748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:39.217436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:39.733329 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8f6c6bcd-2e41-421a-9c29-74f93a41c273 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 223 microversion: 2.1 time: 0.980447 Oct 15 20:50:39.745518 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 362/761] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:38 2022] POST /compute/v2.1/os-security-groups => generated 223 bytes in 993 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:39.754941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:39.755568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:39.794508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:39.796054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:39.797075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:39.805744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:39.809217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:39.809385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.240387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.262172 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aee65c72-568a-44a8-bce1-219a714af4b4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1463 microversion: 2.1 time: 1.078425 Oct 15 20:50:40.263557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 400/762] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:39 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1463 bytes in 1090 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:40.303276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:40.305558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:40.331651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.333423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.334463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.339119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.340031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.344015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.351334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.363333 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-46dfb7bf-0975-48df-a1d0-5da38d1bbd97 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.612888 Oct 15 20:50:40.364449 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 363/763] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:39 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:40.369257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8c481b3e-5a08-4ae5-b3e7-15269e45fae9 req-6ddc251f-d24a-4ad7-aadb-8dd9b1355c3f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "72d34005-b40a-4e73-8f98-b0a6d2d0106b", "tag": "54d23914-40e3-4e01-9162-9682f70a408f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:40.382632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8c481b3e-5a08-4ae5-b3e7-15269e45fae9 req-6ddc251f-d24a-4ad7-aadb-8dd9b1355c3f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.384322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8c481b3e-5a08-4ae5-b3e7-15269e45fae9 req-6ddc251f-d24a-4ad7-aadb-8dd9b1355c3f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.384909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8c481b3e-5a08-4ae5-b3e7-15269e45fae9 req-6ddc251f-d24a-4ad7-aadb-8dd9b1355c3f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.399815 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8c481b3e-5a08-4ae5-b3e7-15269e45fae9 req-6ddc251f-d24a-4ad7-aadb-8dd9b1355c3f service nova] Creating event network-changed:54d23914-40e3-4e01-9162-9682f70a408f for instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b on nch2-master-1 Oct 15 20:50:40.408996 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8c481b3e-5a08-4ae5-b3e7-15269e45fae9 req-6ddc251f-d24a-4ad7-aadb-8dd9b1355c3f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042914 Oct 15 20:50:40.411070 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 364/764] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:40.418939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e4723aea-a1cd-4a5e-80cd-d8c74399e7bd tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fac2dc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:40.498277 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e4723aea-a1cd-4a5e-80cd-d8c74399e7bd tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910" status: 200 len: 223 microversion: 2.1 time: 0.085550 Oct 15 20:50:40.498697 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 365/765] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:40 2022] GET /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 => generated 223 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:40.512260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:40.513920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:40.521813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.522607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.523287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.524498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.525232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.525919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.541340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.566486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.568408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.569790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c561a487-61bc-4522-ad25-cb92c7b7d6aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1316 microversion: 2.25 time: 0.299683 Oct 15 20:50:40.586396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 401/766] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:40 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1316 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:40.597322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:40.598127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:40.605135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.605964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.606676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.607678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.608348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.608966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.741392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.774117 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-21d10af2-141b-4feb-bf20-dcd69de0fd95 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.265596 Oct 15 20:50:40.774709 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 366/767] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:40 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:40.784500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:40.785486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:40.805187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.806038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.806725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.809134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.810708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.812453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.908763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.918272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.919960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:40.926880 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f53e082-c572-4c66-a333-14adceab89bc tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.149841 Oct 15 20:50:40.927469 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 367/768] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:40 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:40.932287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:40.935308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:40.942971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.943466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.944016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:40.944738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:40.945257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:40.945968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.045126 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1ff38e62-b6e7-433a-a4af-59c293cf6b97 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 0 microversion: 2.1 time: 0.115906 Oct 15 20:50:41.049003 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 368/769] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:40 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:41.056197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f43b0379-4e50-4724-8877-97ae8abbcfe2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fac2ca0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:41.082084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:41.092061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d242978c-e114-4065-ace1-1d3318ecc19d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.503996 Oct 15 20:50:41.107282 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 402/770] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:40 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:41.107282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:41.107282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:41.122107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.123257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.123706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.124483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.125201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.126479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.184873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.185428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.185883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.186351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lazy-loading 'fault' on Instance uuid 5d1ddd46-47f6-4948-a897-3b963780991e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:50:41.311201 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f43b0379-4e50-4724-8877-97ae8abbcfe2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910" status: 202 len: 0 microversion: 2.1 time: 0.262254 Oct 15 20:50:41.312547 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 369/771] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:50:41 2022] DELETE /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 => generated 0 bytes in 264 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:41.321389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fae00d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:41.358845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:50:41.359485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:50:41.360044 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.360044 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 HTTP/1.1 Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:50:41.372636 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] Oct 15 20:50:41.374450 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '0f6d06ff-1b65-4a54-a9d2-07e0c7d92910'} Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:41.375417 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:50:41.376927 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.378268 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] Oct 15 20:50:41.379814 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:41.380501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:50:41.380501 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-8d0ebf51-3d7c-4e24-8189-2c7a6ff2d770'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:50:41.381802 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e98308af-ed4e-4513-88b6-3dfabb170db9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910" status: 404 len: 194 microversion: 2.1 time: 0.064967 Oct 15 20:50:41.383147 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 370/772] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:41 2022] GET /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 => generated 194 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:41.389086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-527a69ba-bfd7-4197-83e5-3fa045b6967b req-a9804e6e-0db2-4b15-a8aa-d2430772c24a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0028073f-e418-41fb-830e-0c49db2dee33", "tag": "41a31112-fc5d-4e74-9274-0be89a46346e"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:41.401438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-527a69ba-bfd7-4197-83e5-3fa045b6967b req-a9804e6e-0db2-4b15-a8aa-d2430772c24a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.402063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-527a69ba-bfd7-4197-83e5-3fa045b6967b req-a9804e6e-0db2-4b15-a8aa-d2430772c24a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.402608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-527a69ba-bfd7-4197-83e5-3fa045b6967b req-a9804e6e-0db2-4b15-a8aa-d2430772c24a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.415571 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-527a69ba-bfd7-4197-83e5-3fa045b6967b req-a9804e6e-0db2-4b15-a8aa-d2430772c24a service nova] Creating event network-changed:41a31112-fc5d-4e74-9274-0be89a46346e for instance 0028073f-e418-41fb-830e-0c49db2dee33 on nch1-master-1 Oct 15 20:50:41.420784 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-527a69ba-bfd7-4197-83e5-3fa045b6967b req-a9804e6e-0db2-4b15-a8aa-d2430772c24a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036017 Oct 15 20:50:41.421639 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 371/773] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:41.427020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:41.428001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:41.436181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.436772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.437310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.438439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.438975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.439481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.596901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1b11369-484e-445a-bd9a-806a7241ef2f tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1597 microversion: 2.1 time: 0.503152 Oct 15 20:50:41.597679 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 403/774] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:41 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1597 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:41.602854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bd47fbb-c6f3-4a10-9d39-7fd45f2f110c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe754590dc0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-682968371", "description": "tempest-description-1720262236"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:41.610613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:41.618319 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d06765ee-2ac4-4a30-a38e-e27f96efc99c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.195218 Oct 15 20:50:41.619048 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 372/775] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:41 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:41.639254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:41.640218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:41.646389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.646913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.647356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.648121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.648615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.649151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.754918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:41.765144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:41.766942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:41.768348 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3fe985f9-1b2b-46c7-b9e5-1312defd4fa7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1316 microversion: 2.25 time: 0.134258 Oct 15 20:50:41.769507 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 373/776] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:41 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1316 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:41.774511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "active"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:41.777088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:41.785003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.785635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.786197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.787355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.788086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.788915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.877339 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c151d47-c5f7-4060-80f9-2f3de58ca203 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action" status: 202 len: 0 microversion: 2.1 time: 0.106307 Oct 15 20:50:41.878764 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 374/777] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:41 2022] POST /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e/action => generated 0 bytes in 108 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:41.884910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:41.886020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:41.897353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.898321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.899055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:41.900241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:41.901711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:41.902552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.341892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:42.344373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:42.352083 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a8720a9-1ef0-4556-9625-4340f48c8495 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.471661 Oct 15 20:50:42.352746 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 375/778] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:41 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:42.369053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:42.369053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:42.369053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.369053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.370278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.370278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.370278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.370278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.466377 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bd47fbb-c6f3-4a10-9d39-7fd45f2f110c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.867286 Oct 15 20:50:42.466978 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 404/779] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:41 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:42.472334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:42.472985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:42.479847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.480344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.480787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.481750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.482378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.482974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.516449 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c00e2818-fede-430f-916b-bd7b3ab55898 tempest-ServersAdminTestJSON-27969756 tempest-ServersAdminTestJSON-27969756-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1614 microversion: 2.1 time: 0.162472 Oct 15 20:50:42.517423 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 376/780] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:42 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1614 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:42.522105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:42.522766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:42.529183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.529682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.530403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.530865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.531326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.531758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.574107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:42.583359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:42.584853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:42.590014 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c920e909-8d46-4975-bc49-aabad6816521 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1537 microversion: 2.91 time: 0.121796 Oct 15 20:50:42.591341 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 405/781] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:42 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1537 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:42.596514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d8b917c-800e-4192-bf52-5608d0914b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe74fc093a0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-2001773522", "description": "tempest-description-103632136"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:42.681125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:42.696812 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4abf4e2c-3814-43e8-8d3b-f7b24c073cc9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.170635 Oct 15 20:50:42.696812 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 377/782] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:42 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:42.696812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:42.696812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:42.705419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.705927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.706405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.707113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.707677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.708243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.735866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:50:42.825493 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-79bf3ff3-2964-4c7d-94f5-9a12b8813314 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "DELETE /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 204 len: 0 microversion: 2.1 time: 0.133506 Oct 15 20:50:42.829386 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 378/783] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:50:42 2022] DELETE /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:50:42.835493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:42.836570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:42.843966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.844460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.844975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.845977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:42.846463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:42.846887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:42.999475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.004330 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d8b917c-800e-4192-bf52-5608d0914b3c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.412005 Oct 15 20:50:43.005923 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 406/784] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:42 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:43.012047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.012887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:43.018813 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e311aa54-ba18-4166-8788-090f885fd69b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.187908 Oct 15 20:50:43.019842 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 379/785] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:42 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:43.021535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.022036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.022477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.023190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.023649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.024078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.024550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.025212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:43.031862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.032360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.032807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.033560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.034029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.034617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.202740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.215441 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-22f9eb4b-c5b9-4094-be88-a29104f42ad8 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1470 microversion: 2.1 time: 0.194521 Oct 15 20:50:43.225561 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 380/786] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:43 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1470 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:43.225561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f3b855da-b3a1-47a8-89d0-6b3150860a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe74f9c5430>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-213365939", "description": "tempest-description-1277338237"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:43.267522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.276368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.277868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.279958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77e9f668-6c51-42fa-86e5-7e0407428911 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.272768 Oct 15 20:50:43.281057 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 407/787] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:43 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:43.291738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c3426770-50a9-4289-8244-381f5135ce5f req-3ab0a518-10d3-4325-b178-3a42a4e557c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ee567153-18c3-4c9b-a3eb-a9c13b250fe3", "name": "network-vif-plugged", "status": "completed", "tag": "0ac06a96-2831-4954-a5e4-8d904f1d0eb6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:43.315236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c3426770-50a9-4289-8244-381f5135ce5f req-3ab0a518-10d3-4325-b178-3a42a4e557c6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.315826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c3426770-50a9-4289-8244-381f5135ce5f req-3ab0a518-10d3-4325-b178-3a42a4e557c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.316715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c3426770-50a9-4289-8244-381f5135ce5f req-3ab0a518-10d3-4325-b178-3a42a4e557c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.330169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c3426770-50a9-4289-8244-381f5135ce5f req-3ab0a518-10d3-4325-b178-3a42a4e557c6 service nova] Creating event network-vif-plugged:0ac06a96-2831-4954-a5e4-8d904f1d0eb6 for instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 on nch1-master-1 Oct 15 20:50:43.335815 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c3426770-50a9-4289-8244-381f5135ce5f req-3ab0a518-10d3-4325-b178-3a42a4e557c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053236 Oct 15 20:50:43.336790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 408/788] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:43.381163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.381876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:43.389622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.390491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.391324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.391652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.392109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.392723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.398444 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f3b855da-b3a1-47a8-89d0-6b3150860a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.179899 Oct 15 20:50:43.399218 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 381/789] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:43 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:43.412508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0db3b75c-5f0c-42ca-8b88-3171debc0031 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fa2ddc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.773158 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0db3b75c-5f0c-42ca-8b88-3171debc0031 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups" status: 200 len: 1831 microversion: 2.1 time: 0.364523 Oct 15 20:50:43.773831 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 382/790] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:50:43 2022] GET /compute/v2.1/os-security-groups => generated 1831 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:43.783548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.784154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:43.791178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.791665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.792100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.792804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.793636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.794224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.796008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.804809 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f24b844e-7f9e-473e-bfa4-bd633274f420 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.428991 Oct 15 20:50:43.805595 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 409/791] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:43 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:43.811050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.811846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:43.818895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.819389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.819830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.820536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:43.821188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:43.821772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:43.887385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.896840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.898483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:43.903858 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4b7b9074-2da5-4bee-b2b6-23466aa26ee2 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1540 microversion: 2.91 time: 0.124555 Oct 15 20:50:43.904567 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 383/792] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:43 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1540 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:43.909417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f1ebb170-5bae-4f0c-b155-aa802ab2fa77 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9c99d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:43.914835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:44.016543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.026061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e251909-1091-4634-a29a-9cc4c10cc99a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.219180 Oct 15 20:50:44.038124 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 410/793] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:43 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:44.050033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d643961e-75d1-41cb-a2eb-b006942c8c6c req-b8918697-6bf2-4c9a-8739-9ef4b1eddb24 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "73402931-6f69-461a-84ae-e0e91c34b2e5", "tag": "998f23e5-d301-418e-aa99-242986c4578b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:44.061608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d643961e-75d1-41cb-a2eb-b006942c8c6c req-b8918697-6bf2-4c9a-8739-9ef4b1eddb24 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.062317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d643961e-75d1-41cb-a2eb-b006942c8c6c req-b8918697-6bf2-4c9a-8739-9ef4b1eddb24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.062933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d643961e-75d1-41cb-a2eb-b006942c8c6c req-b8918697-6bf2-4c9a-8739-9ef4b1eddb24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.075660 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d643961e-75d1-41cb-a2eb-b006942c8c6c req-b8918697-6bf2-4c9a-8739-9ef4b1eddb24 service nova] Creating event network-changed:998f23e5-d301-418e-aa99-242986c4578b for instance 73402931-6f69-461a-84ae-e0e91c34b2e5 on nch2-master-1 Oct 15 20:50:44.081053 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d643961e-75d1-41cb-a2eb-b006942c8c6c req-b8918697-6bf2-4c9a-8739-9ef4b1eddb24 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041100 Oct 15 20:50:44.082494 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 411/794] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:44.088393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Action: 'action', calling method: >, body: {"unshelve": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:44.091122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:44.097896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.098848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.099381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.100622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.101351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.102065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.115109 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f1ebb170-5bae-4f0c-b155-aa802ab2fa77 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e" status: 202 len: 0 microversion: 2.1 time: 0.209136 Oct 15 20:50:44.115841 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 384/795] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:50:43 2022] DELETE /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e => generated 0 bytes in 211 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:44.125078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.126041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:44.145629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.146712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.147709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.148586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.148899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.149513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.163805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "nch1-master-1". {{(pid=138688) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 20:50:44.236168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-082f2514-5020-4132-a7ae-3198b0727df1 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action" status: 202 len: 0 microversion: 2.91 time: 0.151640 Oct 15 20:50:44.236814 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 412/796] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:50:44 2022] POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action => generated 0 bytes in 154 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:50:44.241381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fc17ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.266822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:50:44.267449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 962c07bd-36a2-4a59-8da5-536db0b7d05e not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:50:44.268429 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.268429 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e HTTP/1.1 Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:50:44.273812 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] Oct 15 20:50:44.275332 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '962c07bd-36a2-4a59-8da5-536db0b7d05e'} Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.276185 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:50:44.277720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.279229 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-584b26be-9ec7-41f3-9d7a-b31bbe339125'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-616dee8b-3d9c-4098-8c36-1cd0c937dab2 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e" status: 404 len: 194 microversion: 2.1 time: 0.037200 Oct 15 20:50:44.280670 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 413/797] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e => generated 194 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:44.283177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.284050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:44.290419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.290911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.291351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.292059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.292514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.293152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.314371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.322656 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-07b60a51-a78b-4e31-b929-4f581e12a92f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.205539 Oct 15 20:50:44.326156 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 385/798] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:44.330939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.331757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:44.337878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.338754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.339372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.340249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.341077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.341522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.419305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.428455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.429945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.437070 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8d0e735a-b7f4-473a-a4ac-5225d33db5e3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1548 microversion: 2.91 time: 0.109701 Oct 15 20:50:44.437723 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 386/799] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1548 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:44.442283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2b90fe48-8abf-471b-b178-bb52ad2d6dad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9c5670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.467634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.471692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:44.477489 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8db9bf0f-f939-49c8-bb54-2194dac7c8c2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1470 microversion: 2.1 time: 0.200276 Oct 15 20:50:44.478136 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 414/800] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1470 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:44.483804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.484419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:44.496360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.496474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.497715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.498885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.499510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.499953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.697924 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2b90fe48-8abf-471b-b178-bb52ad2d6dad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93" status: 202 len: 0 microversion: 2.1 time: 0.258966 Oct 15 20:50:44.702447 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 387/801] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:50:44 2022] DELETE /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 => generated 0 bytes in 264 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:44.707563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fa2db80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.738673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.749076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.750643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:44.756086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a674cbf-ccd5-410a-a9e9-66972b5a85cb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.276839 Oct 15 20:50:44.759642 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 415/802] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:44.798673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:50:44.799482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:50:44.800258 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.800258 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 HTTP/1.1 Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:50:44.804151 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] Oct 15 20:50:44.805679 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93'} Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.806569 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:50:44.808062 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.810481 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-4ad54cbc-360b-4407-9e4c-44749f7ef347'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1ffb21b1-0d86-426f-8f68-d38aaa2e57cc tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93" status: 404 len: 194 microversion: 2.1 time: 0.102113 Oct 15 20:50:44.811957 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 388/803] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 => generated 194 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:44.819703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a720b93-0130-4282-9365-68576652bddd tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fb07af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.834766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:44.835749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:44.848117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.848795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.849530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:44.850536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:44.851204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:44.851804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.070944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.081605 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98495b9d-a58d-49bf-9feb-8bddafed12bd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1311 microversion: 2.1 time: 0.250888 Oct 15 20:50:45.098921 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 389/804] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:44 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1311 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:45.104936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.105906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:45.120335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:45.122853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.123617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.124126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.124958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.125630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.126542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.298671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.307262 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23f0c082-0abd-442d-89af-209bb639483e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.206960 Oct 15 20:50:45.308024 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 390/805] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:45.348775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.349474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:45.355810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.357124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.357841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.358733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.359205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.359642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.517411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.527221 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9f0c0557-e930-4750-a1bd-864dd415136a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.183301 Oct 15 20:50:45.528680 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 391/806] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:45.533787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.536735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:45.541784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.542486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.542922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.544341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.544955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.545611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.570426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a720b93-0130-4282-9365-68576652bddd tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98" status: 202 len: 0 microversion: 2.1 time: 0.756573 Oct 15 20:50:45.571364 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 416/807] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:50:44 2022] DELETE /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98 => generated 0 bytes in 760 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:45.579715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.580384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:45.597281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.598189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.598991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.601070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.602637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.603257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.658389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.676933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.681146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.686795 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c6293c88-2a6c-4604-870d-70ad65b7b99a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1548 microversion: 2.91 time: 0.156693 Oct 15 20:50:45.687364 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 392/808] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1548 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:45.692523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9c9310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.723871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:50:45.724470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 869ab600-7466-46ab-bac3-adb2eaf86a98 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:50:45.724996 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.724996 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98 HTTP/1.1 Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:50:45.727106 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] Oct 15 20:50:45.728374 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '869ab600-7466-46ab-bac3-adb2eaf86a98'} Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:45.729216 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:50:45.732132 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.733980 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-d8e23ca7-5409-491c-98e1-e518d97a6c01'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a54ac7c4-c188-4366-8170-dd8551a94078 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98" status: 404 len: 194 microversion: 2.1 time: 0.040333 Oct 15 20:50:45.735403 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 393/809] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98 => generated 194 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:45.740558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f3d52e0-2605-4537-ac71-4cabda7c7f47 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9d6c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.877555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:45.886709 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0522f8d1-9207-445c-a349-775b367e9fb7 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1470 microversion: 2.1 time: 0.314244 Oct 15 20:50:45.890914 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 417/810] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1470 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:45.911677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:45.916331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:45.929663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.930608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.931512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.932437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:45.933121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:45.936606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:45.964138 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f3d52e0-2605-4537-ac71-4cabda7c7f47 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups" status: 200 len: 1156 microversion: 2.1 time: 0.227506 Oct 15 20:50:45.973543 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 394/811] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/os-security-groups => generated 1156 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:45.985127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-82c1b603-fdb7-4a38-93ca-c6ed0cc320a3 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe74fd5c0d0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-35846877", "description": "tempest-description-1600937487"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:46.212471 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-82c1b603-fdb7-4a38-93ca-c6ed0cc320a3 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 242 microversion: 2.1 time: 0.231702 Oct 15 20:50:46.213151 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 395/812] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:45 2022] POST /compute/v2.1/os-security-groups => generated 242 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:46.214603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.221429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:46.221868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:46.228688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.230353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.232369 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9830b5cb-2888-4b3a-8841-5c541fe710e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.339680 Oct 15 20:50:46.233432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.233921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.234073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 418/813] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:45 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:46.234721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.235664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.236866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.237395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.256843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e5ac95aa-b6a7-4d22-8f73-6b967edd0870 req-08378154-3c3f-4b10-8f52-12bcc9f2025a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "683f8544-7601-473f-b81f-c5bf78f0c781", "name": "network-vif-deleted", "tag": "c833f037-3f48-49a5-9761-1581e6331daf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:46.267739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e5ac95aa-b6a7-4d22-8f73-6b967edd0870 req-08378154-3c3f-4b10-8f52-12bcc9f2025a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.268724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e5ac95aa-b6a7-4d22-8f73-6b967edd0870 req-08378154-3c3f-4b10-8f52-12bcc9f2025a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.269427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e5ac95aa-b6a7-4d22-8f73-6b967edd0870 req-08378154-3c3f-4b10-8f52-12bcc9f2025a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.289836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e5ac95aa-b6a7-4d22-8f73-6b967edd0870 req-08378154-3c3f-4b10-8f52-12bcc9f2025a service nova] Creating event network-vif-deleted:c833f037-3f48-49a5-9761-1581e6331daf for instance 683f8544-7601-473f-b81f-c5bf78f0c781 on nch2-master-1 Oct 15 20:50:46.299942 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e5ac95aa-b6a7-4d22-8f73-6b967edd0870 req-08378154-3c3f-4b10-8f52-12bcc9f2025a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064538 Oct 15 20:50:46.300933 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 419/814] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:46.312910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6809bda-e508-4f8b-a52b-42d74701e66e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe75416c940>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1042023238", "description": "tempest-description-909357287"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:46.409318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.418011 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-be982aca-4ee9-4d4a-832a-186804008c8f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1495 microversion: 2.1 time: 0.203851 Oct 15 20:50:46.419189 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 396/815] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:46 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1495 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:46.424291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:46.425137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:46.442172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.442737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.443185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.443905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.444394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.444860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.558512 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6809bda-e508-4f8b-a52b-42d74701e66e tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.255985 Oct 15 20:50:46.559516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 420/816] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:50:46 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:46.565156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:46.568831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:46.581349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.581920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.582427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.583264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.584008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.584549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.642682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.651879 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65dadf21-2be3-41f3-b072-bce9fc949788 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.231424 Oct 15 20:50:46.653595 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 397/817] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:46 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:46.659623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:46.660871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:46.669352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.670233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.670947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.672025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.672496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.672928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.733563 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-189ecfca-fdf7-407e-989f-1f10a5b499fe tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1646 microversion: 2.1 time: 0.172488 Oct 15 20:50:46.734230 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 421/818] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:46 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1646 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:46.738849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SecurityGroupsTestJSON-server-827027764", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "b3ffd293-16ae-49a6-a8ac-0b3025295760"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:46.794401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] validate_networks() for [('b3ffd293-16ae-49a6-a8ac-0b3025295760', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:50:46.809811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.813233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:46.818505 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a29343af-49db-4d20-9c2e-5ea7dc790929 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.163577 Oct 15 20:50:46.819008 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 398/819] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:46 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:46.824471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:46.825658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:46.832408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.832899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.833364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.834116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.834732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.835309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.913762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.923310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.925248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:46.930932 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c9158bb7-6f40-42c3-ba44-c94283586e68 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.110936 Oct 15 20:50:46.931858 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 399/820] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:46 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:46.937319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:46.938120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:46.944301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.945167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.945843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:46.946777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:46.947450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:46.948093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.034248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:47.095695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:47.097217 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a5e5019c-49a6-4027-8927-d640eb3685fd tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1646 microversion: 2.1 time: 0.163863 Oct 15 20:50:47.097750 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 400/821] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:46 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1646 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:47.105337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:47.106985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.114585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.115389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.116095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.117235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.117697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.118439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.191133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:47.205108 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-29ac032e-e3fe-4e69-a661-c92ea5099a40 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1428 microversion: 2.1 time: 0.105545 Oct 15 20:50:47.206215 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 401/822] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:47 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1428 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:47.210703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:47.214504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.215628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.216207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.217128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.217731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.218325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.244999 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a811f29c-612f-4e6f-a840-199901572164 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.037581 Oct 15 20:50:47.245617 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 402/823] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:50:47 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:47.258492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:47.259209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.266005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.266699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.267148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.267845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.268296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.268887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.343779 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ebaee81-d3af-411c-9946-3904bc0d8a7d tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3/action" status: 202 len: 0 microversion: 2.1 time: 0.088957 Oct 15 20:50:47.347257 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 403/824] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:47 2022] POST /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:50:47.350421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:47.351127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.357839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.358327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.358763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.359480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.360512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.360932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.503381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:47.513745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:47.515282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:47.516827 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-483a314c-33c6-4327-baab-491b77a47056 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.171425 Oct 15 20:50:47.533566 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 404/825] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:47 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:47.541657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:47.543238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.555405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.555909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.556348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.557107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.557579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.558012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.703381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:50:47.704963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:50:47.705485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:50:47.706123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:50:47.706764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:50:47.716816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.717752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.718483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.719311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.720049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.720797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.740687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Getting quotas for project 869f1ee4c91c4067a3e32e39155b30a6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:50:47.748091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Getting quotas for user 03bc9235fe1c4fc1b3097a913cbe1052 and project 869f1ee4c91c4067a3e32e39155b30a6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:50:47.754485 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bb51baa8-9de1-4231-829a-083f5d3ffb82 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1631 microversion: 2.1 time: 0.219000 Oct 15 20:50:47.755129 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 405/826] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:47 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1631 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:47.760407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:47.761346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.769354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.769905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.770471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.771181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.771780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.772437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.778441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:50:47.781452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:50:47.837211 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ac60ec2-5825-4bab-a120-42d3a7c13605 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.101814 Oct 15 20:50:47.837864 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 422/827] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:50:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 1103 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:50:47.842581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:47.845158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.852631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.853147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.853600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.854296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.854753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.855183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.932575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lazy-loading 'flavor' on Instance uuid ee567153-18c3-4c9b-a3eb-a9c13b250fe3 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:50:47.944236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:47.954075 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-80656367-32fa-450e-a712-5d5a0fe29809 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.197906 Oct 15 20:50:47.955465 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 406/828] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:47 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:47.960409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:47.962398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:47.978736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.979395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.979971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.980933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:47.981485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:47.981970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:47.994289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Going to try to live migrate instance to nch2-master-1 {{(pid=138688) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:50:48.053783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:48.361360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:48.376918 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a1b2a13a-175a-443f-bc5e-aee272e2181b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.420321 Oct 15 20:50:48.394330 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 407/829] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:47 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:48.404557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b469448e-ee1b-44e8-a643-1863f4e28087 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:48.405214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b469448e-ee1b-44e8-a643-1863f4e28087 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:48.545134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b469448e-ee1b-44e8-a643-1863f4e28087 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:48.556742 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b469448e-ee1b-44e8-a643-1863f4e28087 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1215 microversion: 2.1 time: 0.159747 Oct 15 20:50:48.560777 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 408/830] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:48 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1215 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:48.577283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:48.592656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:48.621230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.625156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.625905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:48.630131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.630674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.632708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:48.750640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:48.762660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:48.765255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:48.779537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f265e956-e2df-4421-874a-78be72d9c20a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.209608 Oct 15 20:50:48.780772 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 409/831] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:48 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:48.786320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:48.786919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:48.793889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.794650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.795329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:48.796221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.796829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.797445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:48.833558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.834049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.834481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:48.834933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lazy-loading 'fault' on Instance uuid 683f8544-7601-473f-b81f-c5bf78f0c781 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:50:48.910822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:48.922489 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-125eda68-b9f6-4e48-a0e5-173a4e434569 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 200 len: 1304 microversion: 2.1 time: 0.140383 Oct 15 20:50:48.923460 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 410/832] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:48 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 1304 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:48.930515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:48.942973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:48.964475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.966129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.971042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:48.975904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:48.976787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:48.979076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.296027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:49.308271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:49.310548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:49.327546 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a9b1ca07-4fb2-4baa-8cb1-52a9aa91778d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.391185 Oct 15 20:50:49.330220 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 411/833] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:48 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:49.339699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:49.343761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:49.355046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.355649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.357596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.358624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.359107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.359544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.550585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:49.560816 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0cbfb8c-ee29-4aa4-8983-8676e928a72a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.229129 Oct 15 20:50:49.561896 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 412/834] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:49 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:49.585969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-98de6d53-ea30-40d6-823b-3d7eb704e960 req-caa7e419-e203-4e49-a662-8c8ebc3f83ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0028073f-e418-41fb-830e-0c49db2dee33", "name": "network-vif-plugged", "status": "completed", "tag": "41a31112-fc5d-4e74-9274-0be89a46346e"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:49.597182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-98de6d53-ea30-40d6-823b-3d7eb704e960 req-caa7e419-e203-4e49-a662-8c8ebc3f83ad service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.597663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-98de6d53-ea30-40d6-823b-3d7eb704e960 req-caa7e419-e203-4e49-a662-8c8ebc3f83ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.598102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-98de6d53-ea30-40d6-823b-3d7eb704e960 req-caa7e419-e203-4e49-a662-8c8ebc3f83ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.610215 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-98de6d53-ea30-40d6-823b-3d7eb704e960 req-caa7e419-e203-4e49-a662-8c8ebc3f83ad service nova] Creating event network-vif-plugged:41a31112-fc5d-4e74-9274-0be89a46346e for instance 0028073f-e418-41fb-830e-0c49db2dee33 on nch1-master-1 Oct 15 20:50:49.615324 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-98de6d53-ea30-40d6-823b-3d7eb704e960 req-caa7e419-e203-4e49-a662-8c8ebc3f83ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034033 Oct 15 20:50:49.616283 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 413/835] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:49.621579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:49.622322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:49.635294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.637550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.638882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.640729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.641924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.643256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.843671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:49.855197 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-007f3c92-d98c-4f98-951e-ff5c0e3b303a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.237342 Oct 15 20:50:49.857112 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 414/836] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:49 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:49.866126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:49.866903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:49.873303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.873870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.874377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.875140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:49.875662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:49.876166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:49.964767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:49.980159 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bfaeafac-e0b8-4742-a1bd-6931070884ad tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1211 microversion: 2.1 time: 0.117564 Oct 15 20:50:49.994950 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 415/837] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:49 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1211 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:50.000838 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.001479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.007495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.007965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.008404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.009135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.009628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.010061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.029297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:50.096966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.122566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.124078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.130253 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a11dc8c1-0715-4e07-af90-b538e8e75406 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.133692 Oct 15 20:50:50.135248 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 416/838] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:49 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:50.136055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.136666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 683f8544-7601-473f-b81f-c5bf78f0c781] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.144281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.144800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.145300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.146008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.146473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.146905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.163969 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] HTTP exception thrown: Instance 683f8544-7601-473f-b81f-c5bf78f0c781 could not be found. Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Request method failure captured: Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 HTTP/1.1 Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyWsMyzwRupGYt7qzizpav-y2s_uX0AmAOgy8PF1ONi6ZGOJ1qHFeBY0u5HM11XNkNHrIM6z4fhjG5b4ekmwD55sfnmLlfDBnugvfkstJDqnMeQV5YrlIjFeHKzll04GJWGU0I-kii3dQZHhTJQqPg0g0ex0G7Ay62TNO8nY2eWG8A Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1f0b466076164a309233726db31b0efd Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersAdminTestJSON-806770791 Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:50:50.165712 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1f0b466076164a309233726db31b0efd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1f0b466076164a309233726db31b0efd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1f0b466076164a309233726db31b0efd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1f0b466076164a309233726db31b0efd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersAdminTestJSON-806770791 Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1f0b466076164a309233726db31b0efd Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersAdminTestJSON-806770791 Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersAdminTestJSON-806770791-project Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c84b11d2d54b43cbac87115e227ef50e Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersAdminTestJSON-806770791-project Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 683f8544-7601-473f-b81f-c5bf78f0c781 could not be found. Oct 15 20:50:50.166959 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '683f8544-7601-473f-b81f-c5bf78f0c781'} Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 683f8544-7601-473f-b81f-c5bf78f0c781 could not be found. Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:50:50.167699 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 683f8544-7601-473f-b81f-c5bf78f0c781 could not be found. Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 683f8544-7601-473f-b81f-c5bf78f0c781 could not be found. Oct 15 20:50:50.169122 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:50:50.170259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Returning 404 to user: Instance 683f8544-7601-473f-b81f-c5bf78f0c781 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:50:50.170259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e0f6e728-8fff-4cc0-b0d1-bfff588ecc2b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781" status: 404 len: 111 microversion: 2.1 time: 0.034705 Oct 15 20:50:50.170259 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 417/839] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:50 2022] GET /compute/v2.1/servers/683f8544-7601-473f-b81f-c5bf78f0c781 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:50:50.176080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.176677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.182915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.183404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.183910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.184649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.185210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.185645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.213798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:50:50.311882 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bf4b7ac6-117e-44b2-ad0d-47fa4f5488e4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 204 len: 0 microversion: 2.1 time: 0.139471 Oct 15 20:50:50.312627 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 418/840] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:50:50 2022] DELETE /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 20:50:50.321916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.322613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.328847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.329371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.329832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.330645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.331107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.331526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.478663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.487430 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ae5290d1-4836-4eca-932b-a5846ccc0a47 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.170151 Oct 15 20:50:50.488592 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 419/841] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:50 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:50.519976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.523164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.529692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.530872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.531854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.533345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.534755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.535683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.733754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.745595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.748379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.749674 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-13975399-b8fc-44c4-8e95-ff12109a4d35 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.247671 Oct 15 20:50:50.750566 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 420/842] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:50 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:50.761929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.761929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.767366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.767925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.768424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.769228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.769714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.770138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.918789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:50.927147 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-eadca79d-af5f-4008-8cea-b1e1dac977e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.171149 Oct 15 20:50:50.927727 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 421/843] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:50 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:50.932311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:50.932927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:50.939621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.940112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.940548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:50.941280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:50.941735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:50.942155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.084573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.093095 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8235997d-1807-49e4-abab-c66bf391d641 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.164316 Oct 15 20:50:51.093989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 422/844] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:50 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:51.098749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:51.099360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:51.112648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.113369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.113935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.114875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.115329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.115785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.184589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.193444 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-18a1856c-c523-4d01-8e72-494f128ad9ff tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1211 microversion: 2.1 time: 0.098343 Oct 15 20:50:51.194653 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 423/845] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:51 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1211 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:51.200742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:51.201667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:51.211749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.212434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.213017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.213910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.214437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.215003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.296094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.304368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.305833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.311000 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aef935eb-53da-4ebc-a3c9-e599b308415f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.115027 Oct 15 20:50:51.311779 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 424/846] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:51 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:51.521490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:51.522161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:51.530362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.530873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.531318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.532483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.532952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.533431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.684568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.692596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-79c7c586-ac3c-411d-b8a3-ada180579a2a tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.177339 Oct 15 20:50:51.694075 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 425/847] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:51 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:51.776148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:51.776829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:51.784782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.785396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.785898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.786674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:51.787196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:51.787689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:51.979182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.990195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.992069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:51.993470 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cc1df821-598a-4de0-bc48-a898d53b8ebe tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.221540 Oct 15 20:50:52.009398 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 426/848] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:51 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:50:52.019700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:52.020330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:52.026999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.027543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.027986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.029256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.030578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.030578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.185284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.194001 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-46f98034-aa14-49df-97dd-d4e663fc92c8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.183321 Oct 15 20:50:52.197088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 427/849] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:52 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:52.210183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4d797121-9b2e-4b08-a922-d46ee3a0915d req-cd514215-df71-4bbe-a9ec-ca0880ac14da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "73402931-6f69-461a-84ae-e0e91c34b2e5", "name": "network-vif-plugged", "status": "completed", "tag": "998f23e5-d301-418e-aa99-242986c4578b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:52.221892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4d797121-9b2e-4b08-a922-d46ee3a0915d req-cd514215-df71-4bbe-a9ec-ca0880ac14da service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.222390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4d797121-9b2e-4b08-a922-d46ee3a0915d req-cd514215-df71-4bbe-a9ec-ca0880ac14da service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.222828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4d797121-9b2e-4b08-a922-d46ee3a0915d req-cd514215-df71-4bbe-a9ec-ca0880ac14da service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.236160 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4d797121-9b2e-4b08-a922-d46ee3a0915d req-cd514215-df71-4bbe-a9ec-ca0880ac14da service nova] Creating event network-vif-plugged:998f23e5-d301-418e-aa99-242986c4578b for instance 73402931-6f69-461a-84ae-e0e91c34b2e5 on nch2-master-1 Oct 15 20:50:52.241213 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4d797121-9b2e-4b08-a922-d46ee3a0915d req-cd514215-df71-4bbe-a9ec-ca0880ac14da service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042899 Oct 15 20:50:52.241813 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 428/850] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:52.249179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:52.249965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:52.257013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.257629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.258135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.258918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.259452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.259966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.418627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.427185 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e31e07c1-80d2-4314-b2ca-95cfa4345f07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.182921 Oct 15 20:50:52.432732 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 429/851] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:52 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:52.438000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:52.438630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:52.440587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:52.446855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.447371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.447816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.448697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.449195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.450281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.522411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.531059 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d9c0dda3-b599-4a24-a469-9a4ee6301699 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1211 microversion: 2.1 time: 0.097159 Oct 15 20:50:52.531627 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 430/852] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:52 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1211 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:52.536650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:52.537551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:52.544332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.544825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.545287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.545990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.546444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.546867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.624093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.633113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.634582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.640525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f393ec06-67f4-4923-affa-b939e6587742 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.107777 Oct 15 20:50:52.641312 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 431/853] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:52 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:52.719249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:52.720688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:52.727980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.728692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.729359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.730530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:52.731178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:52.731753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:52.905351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:52.907605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:52.915838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c696720f-693b-4a09-ae55-0fbff4901191 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.201008 Oct 15 20:50:52.916479 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 432/854] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:52 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:53.024631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:53.025433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:53.032734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.033647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.034140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.034864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.035428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.035866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.176522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.186041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.187556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.188737 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e7f276bf-38e4-486e-87d0-345273b67cf2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.168645 Oct 15 20:50:53.189401 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 433/855] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:53 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:50:53.221732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:53.222353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:53.228364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.228857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.229337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.230031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.231009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.231473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.376617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.384753 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d80123f-232f-45a4-9c43-5e78be3abf95 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.167146 Oct 15 20:50:53.385554 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 434/856] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:53 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:53.450246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:53.450993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:53.457388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.458027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.458472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.461281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.461281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.461281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.610790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.619676 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d544a94f-82fb-495a-a697-bf13d34e1151 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.173949 Oct 15 20:50:53.620440 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 435/857] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:53 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:53.629134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:53.629689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:53.636297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.637468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.638049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.638770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.639235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.639674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.708282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.716619 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f9ebc9d7-14d0-4ef5-bb5c-2d6fdf75ddc8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1211 microversion: 2.1 time: 0.095166 Oct 15 20:50:53.719424 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 436/858] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:53 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1211 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:53.722402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:53.722848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:53.728894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.729412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.729850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.730695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.731217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.731643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.801639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.810113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.811548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:53.816769 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-653043a9-9f07-4ecd-9811-57dbf21d03e4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.098262 Oct 15 20:50:53.817366 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 437/859] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:53 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:53.939755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:53.940644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:53.947654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.947654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.947933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:53.948706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:53.949805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:53.950297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.102162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.110330 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-08753cc1-887e-45ea-918b-5eb8ee3f59f4 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.174351 Oct 15 20:50:54.111638 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 438/860] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:53 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:54.218106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:54.218729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:54.225728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.226211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.226646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.227379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.227841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.228266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.365170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.374416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.375881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.377065 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98f82983-dd91-485d-8a9c-52c2cdc2eaa8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1358 microversion: 2.25 time: 0.165967 Oct 15 20:50:54.378176 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 439/861] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:54 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:54.408462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:54.409387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:54.415760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.416334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.416879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.417692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.418290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.418994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.566393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.574648 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e67e3cfc-4d24-4ab3-ba87-2b2bb8c41c96 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1476 microversion: 2.1 time: 0.169919 Oct 15 20:50:54.575533 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 440/862] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:54 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1476 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:54.644174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:54.644928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:54.651317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.651816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.652408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.653602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.654084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.654561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.792071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.800735 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e133beb0-867f-4172-a08a-b93184646836 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.160620 Oct 15 20:50:54.801533 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 441/863] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:54 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:54.818799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:54.819518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:54.826016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.826595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.827112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.827821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.828280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.828707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.897576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:54.905741 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ed3a28c6-c04e-40e8-aa9a-a23b3c0bd2b6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1211 microversion: 2.1 time: 0.091581 Oct 15 20:50:54.906311 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 442/864] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:54 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1211 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:54.911970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:54.912809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:54.919749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.920497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.921080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.921793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:54.922250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:54.922677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:54.992391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.005390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.006964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.013099 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f2b7bd6-c5f7-45c0-8fe6-9ba4cf8a77a4 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.105578 Oct 15 20:50:55.013705 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 443/865] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:54 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:55.140084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:55.140084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:55.144809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.145507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.145950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.146662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.147283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.147709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.350986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.359749 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-294b9097-1103-43c5-bd76-361ecb83d979 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.228594 Oct 15 20:50:55.361085 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 444/866] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:55 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:50:55.403201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:55.403981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:55.410753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.411389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.411929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.412741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.413338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.413977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.567018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.576259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.582984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.584319 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e3c8659a-680e-4911-abed-a3555e0bf527 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1554 microversion: 2.25 time: 0.185367 Oct 15 20:50:55.585349 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 445/867] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:55 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1554 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:55.601255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:55.603216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:55.610540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.611012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.611457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.612169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.612620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.613077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.765282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:55.776092 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1a821277-1c06-49eb-90bd-78f888fa68ca tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1475 microversion: 2.1 time: 0.178944 Oct 15 20:50:55.781957 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 446/868] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:55 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1475 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:55.833224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:55.833868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:55.840707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.841228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.841661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.842965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:55.843434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:55.844286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:55.990770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid 73402931-6f69-461a-84ae-e0e91c34b2e5 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:50:56.006092 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-11c7628c-b921-4a58-992c-1a4d74920cc7 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 2043 microversion: 2.25 time: 0.227542 Oct 15 20:50:56.006793 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 447/869] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:55 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 2043 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:56.011403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:56.012282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:56.019248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.019750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.020197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.020911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.021407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.021839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.171233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:56.179805 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9186be00-9fdd-487b-b689-f05f2a202920 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.172011 Oct 15 20:50:56.180689 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 448/870] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:56 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:56.185935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:56.186531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:56.205692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.206504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.206504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.207245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.207797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.208225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.211212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:56.274548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:56.283679 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b68dd315-50bb-4624-b65e-16ca04a07944 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1211 microversion: 2.1 time: 0.101819 Oct 15 20:50:56.287038 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 449/871] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:56 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1211 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:56.288605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:56.289233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:56.295196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.295677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.296112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.296798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.297279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.297709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.426828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid 73402931-6f69-461a-84ae-e0e91c34b2e5 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:50:56.440212 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e1a9f94-82a6-4fc9-8d98-691437bcc233 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 2043 microversion: 2.25 time: 0.155006 Oct 15 20:50:56.440767 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 450/872] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:56 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 2043 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:56.445180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:56.445771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:56.451263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.451742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.452190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.452887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.453365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.453791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.522093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:56.530669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:56.532397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:56.537124 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a41c9d0a-fe92-4ac8-bcba-43350f8110b9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.095300 Oct 15 20:50:56.537634 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 451/873] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:56 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:56.541977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:56.542575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:56.548733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.549243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.549676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.550360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.550813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.551233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.682643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:56.691361 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-01395dca-c1a5-4e1a-bb3f-385c45a6edaf tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.152754 Oct 15 20:50:56.691924 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 452/874] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:56 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:56.696121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:56.700931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.701443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.701880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.702616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.703071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.703503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.727045 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-909e7940-d972-4ee3-9215-5a78eb5317c5 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.034141 Oct 15 20:50:56.727618 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 453/875] 10.222.0.16 () {60 vars in 1217 bytes} [Sat Oct 15 20:50:56 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:50:56.739764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:56.742085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:56.748711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.749221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.749658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.750345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:56.750799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:56.751223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:56.775756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'flavor' on Instance uuid 73402931-6f69-461a-84ae-e0e91c34b2e5 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:50:56.787597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Going to try to live migrate instance to nch1-master-1 {{(pid=138689) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:50:57.729909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:57.756995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:58.011001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:58.082742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=43,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:58.448293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21725a43-8ff4-408f-83cb-7634a74fd931 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3/action" status: 202 len: 0 microversion: 2.1 time: 10.609068 Oct 15 20:50:58.449108 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 423/876] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:50:47 2022] POST /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3/action => generated 0 bytes in 10610 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:50:58.454666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:58.455270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:58.462020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.462493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.462927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.463612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.464066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.464487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.550194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:50:58.595760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:58.604064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30f29314-7c71-4225-aec4-e89ce756ac2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1475 microversion: 2.1 time: 0.153830 Oct 15 20:50:58.604627 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 424/877] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:58 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1475 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:58.609102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:58.609711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:58.615372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.615853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.616286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.616971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.617464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.617885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.749158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:58.756802 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-41fa90bc-87bc-44a9-83fd-dfd8c4fb078d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1317 microversion: 2.1 time: 0.151196 Oct 15 20:50:58.757396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 425/878] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:58 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1317 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:58.761711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:58.762312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:58.769054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.769542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.769982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.770672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.771134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.771576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.833961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:58.841517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a43977d7-0a00-4930-9b91-cfadb317b8cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1267 microversion: 2.1 time: 0.083173 Oct 15 20:50:58.842272 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 426/879] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:58 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1267 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:58.846786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:58.847380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:58.853191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.853674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.854136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.854828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.855340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.855694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.919248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:58.927354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:58.928824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:58.934097 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f860440-7d1b-470e-977b-205d58b423d3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.090867 Oct 15 20:50:58.934710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 427/880] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:50:58 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:50:58.940423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:58.941070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:58.959188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.959438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.959819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:58.961290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:58.961290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:58.961476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:59.233690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:59.243734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a26cafb-26e5-4cfa-87c4-a3be8465649b tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1468 microversion: 2.1 time: 0.307836 Oct 15 20:50:59.257669 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 428/881] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:58 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1468 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:59.284826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:59.285476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:59.302079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:59.302565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:59.303001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:59.304069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:59.304233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:59.304557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:59.640747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:50:59.649958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b18c3033-a79b-4471-bc50-e32da34266d6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.391048 Oct 15 20:50:59.661060 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 429/882] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:59 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:50:59.661060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e84ed5b-8560-4a2c-a144-382839509e8f", "name": "network-vif-unplugged", "status": "completed", "tag": "5531cb3d-9d57-432f-9e05-8b2e5b9f4831"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:50:59.666362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:59.666706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:59.667135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:59.680041 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] Creating event network-vif-unplugged:5531cb3d-9d57-432f-9e05-8b2e5b9f4831 for instance 0e84ed5b-8560-4a2c-a144-382839509e8f on nch1-master-1 Oct 15 20:50:59.686614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] Instance 0e84ed5b-8560-4a2c-a144-382839509e8f is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:50:59.713246 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4a4477b7-b3fb-4064-89de-f30f2a8ee890 req-f433abf0-9636-4c2b-89b9-8aac1c23a24b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.059994 Oct 15 20:50:59.716084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 430/883] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:50:59 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:50:59.756122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:50:59.756840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:50:59.783836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:59.784751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:59.785405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:50:59.786305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:50:59.786939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:50:59.789657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.164464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.172875 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ad616d7-9dbd-4a7b-8324-18a0715045f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1475 microversion: 2.1 time: 0.455230 Oct 15 20:51:00.173754 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 431/884] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:50:59 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1475 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:00.179362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.180217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.195144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.195980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.197360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.198851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.199481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.200157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.347537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a69ac71d-8890-455c-b229-fe53ab9c6af6 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5/action" status: 202 len: 0 microversion: 2.25 time: 3.611005 Oct 15 20:51:00.350866 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 454/885] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:50:56 2022] POST /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5/action => generated 0 bytes in 3615 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 15 20:51:00.358377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.359471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.381138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.381895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.382503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.383663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.390652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.391286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.465935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.478687 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a4c6eca7-bcb5-4729-aad9-fb2558219e39 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1267 microversion: 2.1 time: 0.124516 Oct 15 20:51:00.494164 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 455/886] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1267 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:00.503695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-28ec6a81-1894-4b17-a953-32360ab2e646 req-fc00504f-bb38-445c-bb1b-092054f69642 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "63f6e4c4-f541-4646-955c-0d10645390ee", "name": "network-vif-deleted", "tag": "36b4370d-6280-4bfa-ae65-44586b1902e8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:00.514839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-28ec6a81-1894-4b17-a953-32360ab2e646 req-fc00504f-bb38-445c-bb1b-092054f69642 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.515412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-28ec6a81-1894-4b17-a953-32360ab2e646 req-fc00504f-bb38-445c-bb1b-092054f69642 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.515850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-28ec6a81-1894-4b17-a953-32360ab2e646 req-fc00504f-bb38-445c-bb1b-092054f69642 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.529141 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-28ec6a81-1894-4b17-a953-32360ab2e646 req-fc00504f-bb38-445c-bb1b-092054f69642 service nova] Creating event network-vif-deleted:36b4370d-6280-4bfa-ae65-44586b1902e8 for instance 63f6e4c4-f541-4646-955c-0d10645390ee on nch1-master-1 Oct 15 20:51:00.534105 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-28ec6a81-1894-4b17-a953-32360ab2e646 req-fc00504f-bb38-445c-bb1b-092054f69642 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037825 Oct 15 20:51:00.534735 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 456/887] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:00.540275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.540974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.544177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.548268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.548754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.549223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.550125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.550762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.551340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.554902 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-653ead61-0463-444c-9418-a56fcb507e57 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1467 microversion: 2.1 time: 0.379642 Oct 15 20:51:00.556514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 432/888] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1467 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:00.562792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.563515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.569866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.570478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.571014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.571815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.572375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.572902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.606817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.608103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.609188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.609872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lazy-loading 'fault' on Instance uuid 63f6e4c4-f541-4646-955c-0d10645390ee {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:00.624988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.633996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.635454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.641579 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-44573be5-0d16-4978-9ea2-83a859fc6d13 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.105326 Oct 15 20:51:00.642158 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 457/889] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:00.646695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.647517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.656356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.656855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.657364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.658076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.658534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.658962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.671996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.680255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-701468ea-f36a-470a-9349-4ce29b280f17 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 200 len: 1303 microversion: 2.1 time: 0.122305 Oct 15 20:51:00.681084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 433/890] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 1303 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:00.847255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.848628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.855787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.856587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.857247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.858692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.859472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.860083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.943727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.952411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.954036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:00.955270 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b8d6769c-1b16-4d6c-a60a-540c05661fe4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.312183 Oct 15 20:51:00.955839 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 458/891] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:00.960265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:00.961120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:00.969359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.969859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.970555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:00.972053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:00.972577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:00.973286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.266794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:01.289251 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-587d40e4-98af-4934-994e-cb7a148c7f99 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.332286 Oct 15 20:51:01.289885 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 459/892] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:01.296109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:01.306490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:01.324753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.325581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.326855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.328375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.335331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.336208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.556596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:01.566280 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7f7118a1-0829-4431-811e-830ee159ae72 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1475 microversion: 2.1 time: 0.275022 Oct 15 20:51:01.567136 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 460/893] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:01 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1475 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:01.573455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:01.574360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:01.582681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.583545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.584247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.585346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.586081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.586553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.714769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:01.728442 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e868552e-f3d2-4723-ae61-69ce3a1491a6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1273 microversion: 2.1 time: 0.159572 Oct 15 20:51:01.729539 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 461/894] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:01 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1273 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:01.735489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:01.737117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:01.747260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.747770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.749178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.750221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.750773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.751314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.862606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:01.877420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:01.880041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:01.887263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa474f17-6b1c-45dc-a536-21087cfceec9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.156124 Oct 15 20:51:01.888184 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 462/895] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:01 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:01.896495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:01.897549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 63f6e4c4-f541-4646-955c-0d10645390ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:01.906087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.907967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.908653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.910047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.910755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.911505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.930065 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] HTTP exception thrown: Instance 63f6e4c4-f541-4646-955c-0d10645390ee could not be found. Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Request method failure captured: Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee HTTP/1.1 Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyWsMyzwRupGYt7qzizpav-y2s_uX0AmAOgy8PF1ONi6ZGOJ1qHFeBY0u5HM11XNkNHrIM6z4fhjG5b4ekmwD55sfnmLlfDBnugvfkstJDqnMeQV5YrlIjFeHKzll04GJWGU0I-kii3dQZHhTJQqPg0g0ex0G7Ay62TNO8nY2eWG8A Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1f0b466076164a309233726db31b0efd Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersAdminTestJSON-806770791 Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:01.932885 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1f0b466076164a309233726db31b0efd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1f0b466076164a309233726db31b0efd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1f0b466076164a309233726db31b0efd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1f0b466076164a309233726db31b0efd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersAdminTestJSON-806770791 Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1f0b466076164a309233726db31b0efd Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersAdminTestJSON-806770791 Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersAdminTestJSON-806770791-project Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c84b11d2d54b43cbac87115e227ef50e Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersAdminTestJSON-806770791-project Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 63f6e4c4-f541-4646-955c-0d10645390ee could not be found. Oct 15 20:51:01.934445 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '63f6e4c4-f541-4646-955c-0d10645390ee'} Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 63f6e4c4-f541-4646-955c-0d10645390ee could not be found. Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:01.935278 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 63f6e4c4-f541-4646-955c-0d10645390ee could not be found. Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 63f6e4c4-f541-4646-955c-0d10645390ee could not be found. Oct 15 20:51:01.937285 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:01.939614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Returning 404 to user: Instance 63f6e4c4-f541-4646-955c-0d10645390ee could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:01.941590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bdb16e35-e36d-4a8d-9e42-fb21d69f9d61 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee" status: 404 len: 111 microversion: 2.1 time: 0.049951 Oct 15 20:51:01.942794 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 463/896] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:01 2022] GET /compute/v2.1/servers/63f6e4c4-f541-4646-955c-0d10645390ee => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:01.950761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e84ed5b-8560-4a2c-a144-382839509e8f", "tag": "5531cb3d-9d57-432f-9e05-8b2e5b9f4831"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:01.962511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:01.963306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:01.963960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:01.978444 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] Creating event network-changed:5531cb3d-9d57-432f-9e05-8b2e5b9f4831 for instance 0e84ed5b-8560-4a2c-a144-382839509e8f on nch1-master-1 Oct 15 20:51:01.984728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] Instance 0e84ed5b-8560-4a2c-a144-382839509e8f is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:51:01.995825 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e3a41033-7930-4df2-bf04-79dbd00dd489 req-e1bcf385-57bd-48c8-9549-b5277a114f8c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051216 Oct 15 20:51:01.997097 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 464/897] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:02.003314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.004170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.011756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.012625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.013271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.014541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.015190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.015842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.060454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:51:02.066325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:02.162348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c51feb59-1587-4aa1-9f08-2d4162e3c1d3 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1618 microversion: 2.1 time: 1.479852 Oct 15 20:51:02.166909 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 434/898] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:00 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1618 bytes in 1485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:02.173675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:02.179834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.181207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.182065 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65da8a91-c09f-4c31-b9dd-fcdb2d4f97aa tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 204 len: 0 microversion: 2.1 time: 0.182549 Oct 15 20:51:02.183134 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 465/899] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:01 2022] DELETE /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:02.202563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.202979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.204272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.205112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.207926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.210284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.210810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.214906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.222710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.223832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.224677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.226647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.228016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.229028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.370715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.379556 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e2663332-e109-4ca0-9d9c-21608b1686ad tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1469 microversion: 2.1 time: 0.191292 Oct 15 20:51:02.380239 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 466/900] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:02 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1469 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:02.385285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d284bb2-097b-41e5-9bca-747c31481a51 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: .version_select at 0x7fe74f9945e0>, body: {"server_group": {"name": "tempest-ServersOnMultiNodesTest-Server-Group-1162117926", "policies": ["affinity"]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:02.402736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-4d284bb2-097b-41e5-9bca-747c31481a51 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:02.408318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-4d284bb2-097b-41e5-9bca-747c31481a51 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:02.458808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-4d284bb2-097b-41e5-9bca-747c31481a51 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:02.463769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-4d284bb2-097b-41e5-9bca-747c31481a51 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:02.477402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d284bb2-097b-41e5-9bca-747c31481a51 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 188 microversion: 2.1 time: 0.096052 Oct 15 20:51:02.478170 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 467/901] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 20:51:02 2022] POST /compute/v2.1/os-server-groups => generated 188 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:02.483050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.483641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.489585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.489977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.490520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.491234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.491681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.492546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.558608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.566939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.568709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.569846 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3554b60f-9e70-4884-90d8-dab9107d7f13 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.401345 Oct 15 20:51:02.570430 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 435/902] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:02 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:02.575598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1269109183", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 2, "return_reservation_id": true}, "os:scheduler_hints": {"group": "98fcc0af-9b5b-4ce5-ab02-53a563fa5866"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:02.630770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.639556 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aced198c-2db3-4037-bcf5-478c9d3a3373 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.159911 Oct 15 20:51:02.640527 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 468/903] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:02 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:02.645264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.645886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.648344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:02.651932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.652429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.652980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.653771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.654775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.655365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.795727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.803667 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e138bf89-4c7e-4aba-b953-320d5cb427ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1473 microversion: 2.1 time: 0.162097 Oct 15 20:51:02.804350 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 469/904] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:02 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1473 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:02.809170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.809737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.815628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.816194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.816705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.817530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.818064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.818564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.882875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:02.891133 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6623ea3d-d42c-4f3d-b3ff-5cec08a89255 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1273 microversion: 2.1 time: 0.085651 Oct 15 20:51:02.891789 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 470/905] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:02 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1273 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:02.920739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:02.921696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:02.928905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.929437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.930440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:02.931362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:02.931834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:02.932272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.009886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:03.019396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:03.021217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:03.026810 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce86049a-e7e4-4b9d-8bbd-c543b9ddde9d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1533 microversion: 2.91 time: 0.110573 Oct 15 20:51:03.042215 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 471/906] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:02 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1533 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:03.057404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:03.059856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:03.069326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.070010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.070695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.071781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.072505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.073683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.145582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:03.146040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:03.146558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:03.146969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:03.147610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:03.213850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lazy-loading 'services' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:03.246047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.250034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.250203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.251028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.251565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.255028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.264399 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5695a2c0-4eab-431a-bf2e-89d0f1bcc46f tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1938 microversion: 2.91 time: 0.210397 Oct 15 20:51:03.273624 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 472/907] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:03 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1938 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:03.276756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:03.283126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:03.293414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:03.293949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:03.297612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Going to run 2 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:03.300207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:03.300400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.300891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.301375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.302070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.302522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.302944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.327375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lazy-loading 'flavor' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:03.343197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Checking whether user_id and queued_for_delete are populated for all projects {{(pid=138688) is_qfd_populated /opt/stack/nova/nova/quota.py:1268}} Oct 15 20:51:03.373589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:03.381435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:03.473878 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [383acc1f-300f-4ec6-84a1-9b7307c5fc94] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 20:51:03.665509 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-854979ab-d29e-41ae-8016-75c2ded04104 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action" status: 202 len: 0 microversion: 2.91 time: 0.375729 Oct 15 20:51:03.666296 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 473/908] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:51:03 2022] POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action => generated 0 bytes in 378 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 15 20:51:03.672583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:03.673340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:03.681356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.681861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.682307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.683027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.683497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.684118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.897631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:03.906546 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9ed1a353-518b-4692-99bb-4231ab96ca27 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1469 microversion: 2.1 time: 0.239047 Oct 15 20:51:03.907125 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 474/909] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:03 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1469 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:03.915160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:03.915783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:03.922016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.922535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.922984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:03.923697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:03.924160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:03.924590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.092381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.101650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.103408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.104721 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d7b24fff-ac38-492b-80c0-92b37b80b8fd tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.193465 Oct 15 20:51:04.105617 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 475/910] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:03 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:04.124791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:04.125746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:04.132641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.133204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.133648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.134450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.134917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.135684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.370967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.380441 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a8455ad1-b618-4bc6-afc6-c624de00a955 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.262113 Oct 15 20:51:04.396367 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 476/911] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:04 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:04.396367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:04.396367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:04.396367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.396819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.396819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.396819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.396819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.397405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.471234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.480585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.482825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.485271 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [383acc1f-300f-4ec6-84a1-9b7307c5fc94] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 46766. Oct 15 20:51:04.489327 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aedb8043-22c5-4dfd-a455-edd3a6eb2c63 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.107122 Oct 15 20:51:04.490170 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 477/912] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:04 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:04.505397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:04.511952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:04.516859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:04.517645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:04.524250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.524799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.525206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.525997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.526462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.526883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.528581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=76141eca-81bf-4771-87e9-fcd4007ff701,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:04.584758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:04.591702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:04.637345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:04.640889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:04.646402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:04.677629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.689154 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7969e64-8220-462c-95c3-673a83a258af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 2.117447 Oct 15 20:51:04.690092 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 436/913] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:51:02 2022] POST /compute/v2.1/servers => generated 32 bytes in 2119 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:51:04.691598 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-907e1e1f-2a77-4d6b-9091-69826cf3427e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1470 microversion: 2.1 time: 0.184467 Oct 15 20:51:04.692457 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 478/914] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:04 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1470 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:04.695212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:04.695841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:04.699493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ee567153-18c3-4c9b-a3eb-a9c13b250fe3", "name": "network-vif-unplugged", "status": "completed", "tag": "0ac06a96-2831-4954-a5e4-8d904f1d0eb6"}, {"name": "network-changed", "server_uuid": "ee567153-18c3-4c9b-a3eb-a9c13b250fe3", "tag": "0ac06a96-2831-4954-a5e4-8d904f1d0eb6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:04.702871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.703359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.704568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.705758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.705758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.706180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.723068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.724274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.724944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.740698 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] Creating event network-vif-unplugged:0ac06a96-2831-4954-a5e4-8d904f1d0eb6 for instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 on nch1-master-1 Oct 15 20:51:04.741563 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] Creating event network-changed:0ac06a96-2831-4954-a5e4-8d904f1d0eb6 for instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 on nch1-master-1 Oct 15 20:51:04.749302 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-7184abf5-e76d-4d6c-a66e-0338ce0b9dff req-ac8ee992-06f8-4e10-aa2a-659ca6ea8c17 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 352 microversion: 2.1 time: 0.054881 Oct 15 20:51:04.750520 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 479/915] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:04 2022] POST /compute/v2.1/os-server-external-events => generated 352 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:04.756243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:04.762213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:04.763188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Searching by: {'deleted': False, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe'} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:51:04.786344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.787610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.788069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.788996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.789512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.790005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.878224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:04.899510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.multi_cell_list [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Listed batch of 1 results from cell out of 998 limit. Returned 1 total so far. {{(pid=138689) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:51:04.972739 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ed04db93-4d8e-448f-96e6-036d6b2ac11b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1004 microversion: 2.1 time: 0.220621 Oct 15 20:51:04.974322 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 480/916] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:51:04 2022] GET /compute/v2.1/servers => generated 1004 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:04.981001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:04.981965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:04.991875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.992559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.993300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:04.994365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:04.994930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:04.995581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.099719 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dd7d9354-07ed-4f51-bf78-41d03c1087e7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f/action" status: 204 len: 0 microversion: 2.1 time: 0.123819 Oct 15 20:51:05.104470 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 481/917] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:51:04 2022] POST /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f/action => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:05.112159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:05.113361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:05.121456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.122375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.123953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.125230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.126028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.126679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.235868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.243778 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d2aa9ac-bfed-4200-820c-8d98424d3b37 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.552614 Oct 15 20:51:05.260220 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 437/918] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:04 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:05.265238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c296e40e-84c1-4097-99e7-64a98a14e6c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:05.265857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c296e40e-84c1-4097-99e7-64a98a14e6c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:05.371993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c296e40e-84c1-4097-99e7-64a98a14e6c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.395547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c296e40e-84c1-4097-99e7-64a98a14e6c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1219 microversion: 2.1 time: 0.134104 Oct 15 20:51:05.414608 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 438/919] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:05 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1219 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:05.421232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:05.422693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:05.429007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.429889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.430643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.431579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.432801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.433489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.471059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.489571 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78ab7e48-e074-4286-8057-1bc0c8ff2668 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1469 microversion: 2.1 time: 0.377049 Oct 15 20:51:05.490506 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 482/920] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:05 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1469 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:05.513649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:05.514625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:05.523924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.524456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.524904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.525681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.526152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.526665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.749518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.772435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.775848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.778202 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-50e4924c-dd67-4738-81ce-7c0c16a6aef6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.270511 Oct 15 20:51:05.790301 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 483/921] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:05 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:05.806089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:05.806867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:05.813107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.813742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.814181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.814973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:05.815432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:05.815856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:05.939582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:05.948784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fed3ece8-4f8c-441e-a030-2edc9864864f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1470 microversion: 2.1 time: 0.532535 Oct 15 20:51:05.965435 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 439/922] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:05 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1470 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:05.977381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:05.979663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:06.010123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.015654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.016576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.017668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.018420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.019151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.051167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.062978 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6463fbdd-ff4a-4c57-a737-13c8eddab24a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.271329 Oct 15 20:51:06.063836 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 484/923] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:05 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:06.227541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.247328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.250910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.266375 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c1ff93de-7521-4695-9e4d-3f059ef5afe3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.298677 Oct 15 20:51:06.268259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:06.269353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:06.278618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.280471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.283855 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 440/924] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:05 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:06.284563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.285810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.286189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.286648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.419566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:06.420197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:06.429104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.429608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.430047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.430752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.434494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.435663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.496066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.505881 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1dbac341-210b-4bc0-98b6-326929ce6fdf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.242246 Oct 15 20:51:06.506456 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 485/925] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:06 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:06.524653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:06.525515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:06.533429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.534180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.534694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.535471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.535974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.536451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.571562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.572054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.572486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.572932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lazy-loading 'fault' on Instance uuid 5d1ddd46-47f6-4948-a897-3b963780991e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:06.605303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.614254 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2cd6812f-a46f-48b4-8398-d04eeea4578c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1215 microversion: 2.1 time: 0.197798 Oct 15 20:51:06.615284 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 441/926] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:06 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1215 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:06.630571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.639599 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f724092-0bfd-40a3-8bcb-628097fdea64 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1303 microversion: 2.1 time: 0.121250 Oct 15 20:51:06.640161 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 486/927] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:06 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1303 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:06.805460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "73402931-6f69-461a-84ae-e0e91c34b2e5", "name": "network-vif-unplugged", "status": "completed", "tag": "998f23e5-d301-418e-aa99-242986c4578b"}, {"server_uuid": "5d1ddd46-47f6-4948-a897-3b963780991e", "name": "network-vif-deleted", "tag": "7d14c08a-67ff-4f01-a95d-0c19d04975e3"}, {"name": "network-changed", "server_uuid": "f0ff5b12-729e-4102-8502-c7d39895c72a", "tag": "4e7695c0-414a-4d18-aa08-e1bf39f76d1a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:06.816621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:06.817290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:06.817608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.818119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.818537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.823214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:06.823439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.824294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.824928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.826206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.826772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.827266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.838170 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Creating event network-vif-unplugged:998f23e5-d301-418e-aa99-242986c4578b for instance 73402931-6f69-461a-84ae-e0e91c34b2e5 on nch2-master-1 Oct 15 20:51:06.838687 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Creating event network-vif-deleted:7d14c08a-67ff-4f01-a95d-0c19d04975e3 for instance 5d1ddd46-47f6-4948-a897-3b963780991e on nch2-master-1 Oct 15 20:51:06.839194 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] Creating event network-changed:4e7695c0-414a-4d18-aa08-e1bf39f76d1a for instance f0ff5b12-729e-4102-8502-c7d39895c72a on nch1-master-1 Oct 15 20:51:06.850752 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f8afa754-009f-44e5-ab83-0ee8de1618a2 req-5a8feb1e-2c87-4456-aec7-cb1996c40a1f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 523 microversion: 2.1 time: 0.049187 Oct 15 20:51:06.851356 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 442/928] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:06 2022] POST /compute/v2.1/os-server-external-events => generated 523 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:06.971694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.988419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:06.989313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:06.992916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:06.996643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.997369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.998073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:06.998976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:06.999602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:06.999878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.000439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.002086 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-371acf0a-0579-45a5-b3d6-b795871461c8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.188298 Oct 15 20:51:07.003243 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 487/929] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:06 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:07.091257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:07.092517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:07.099755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.100408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.100855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.101675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.102325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.102754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.273481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.283212 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-55982f32-2f27-4855-9020-0017058b3724 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.196185 Oct 15 20:51:07.283997 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 488/930] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:07 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:07.305147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:07.306022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:07.312565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.313183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.313696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.314476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.315005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.315507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.389891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.398653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.400871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.408966 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c027e758-4e7d-4c97-9b36-af582c41a152 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.114787 Oct 15 20:51:07.410655 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 489/931] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:07 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:07.492138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e84ed5b-8560-4a2c-a144-382839509e8f", "name": "network-vif-plugged", "status": "completed", "tag": "5531cb3d-9d57-432f-9e05-8b2e5b9f4831"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:07.504662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.505728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.507555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.525306 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] Creating event network-vif-plugged:5531cb3d-9d57-432f-9e05-8b2e5b9f4831 for instance 0e84ed5b-8560-4a2c-a144-382839509e8f on nch1-master-1 Oct 15 20:51:07.532454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] Instance 0e84ed5b-8560-4a2c-a144-382839509e8f is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:51:07.544590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-3da23651-3994-47aa-b31c-02ab32bc98b4 req-a3368de1-5285-489e-bf89-46db21065e98 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056890 Oct 15 20:51:07.545594 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 490/932] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:51:07.550770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:07.551658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:07.558197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.559281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.559281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.560518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.561478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.561956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.705076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.713148 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8e1a5224-ab8e-4594-b6fa-4f2d84b908cf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.166427 Oct 15 20:51:07.713734 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 491/933] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:07 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:07.734947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:07.735567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:07.742268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.742421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.742767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.743290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.744007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.744471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.745139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.752739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-532bfd52-6fac-4dd1-8bc3-6fddce15aeb5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1470 microversion: 2.1 time: 0.769067 Oct 15 20:51:07.753438 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 443/934] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:06 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1470 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:07.758507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:07.759194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:07.766248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.767180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.767823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.769030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.770925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.771581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.810301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:07.810848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:07.811512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:07.812138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lazy-loading 'fault' on Instance uuid 5d1ddd46-47f6-4948-a897-3b963780991e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:07.831066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.838928 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-601cd01d-b897-4feb-9e93-a6ab1dd86559 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1271 microversion: 2.1 time: 0.108722 Oct 15 20:51:07.839724 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 492/935] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:07 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1271 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:07.856272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:07.865119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16e5a9c5-fc1e-4f1f-85c3-0577a5971da2 tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1303 microversion: 2.1 time: 0.110300 Oct 15 20:51:07.865927 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 444/936] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:07 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1303 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:08.027824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:08.028546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:08.035488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.035987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.036427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.037377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.037844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.038276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.185648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.196112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.197615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.198806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8c071430-da43-4e4f-a9b1-cf183dd8c13e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.174972 Oct 15 20:51:08.199786 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 493/937] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:08 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:08.307903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:08.308511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:08.315082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.315784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.316223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.316920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.317414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.317839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.437522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:08.438258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:08.444355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.444841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.445312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.446013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.446472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.446894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.512764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.521826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.523266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.528000 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7aabef2a-2416-4908-a03a-4ba5b62bb590 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.095005 Oct 15 20:51:08.528567 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 494/938] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:08 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:08.532688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:08.540935 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d460bfef-4e89-45e7-a878-413116e15af7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.236910 Oct 15 20:51:08.541842 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 445/939] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:08 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:08.741514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:08.752124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:08.768310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.770421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.772078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.773856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.782609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:08.783468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:08.788328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.790389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.798890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.799386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.799828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:08.800538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:08.800993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:08.804698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:08.805544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.026138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.034190 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d503d87-5670-47ba-a1f3-47280158dc2d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.296880 Oct 15 20:51:09.034766 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 495/940] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:08 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:09.041081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:09.041974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:09.048154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:09.049983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.050971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.051992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.053204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.053935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.054581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.128866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.136542 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea70ebf7-e99b-49b9-8ec3-76c776eaca93 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1289 microversion: 2.1 time: 0.099607 Oct 15 20:51:09.137369 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 496/941] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:09 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1289 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:09.142659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:09.143267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:09.150913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.151682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.152347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.153262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.154341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.155048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.188396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.188891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.189365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.189826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lazy-loading 'fault' on Instance uuid 5d1ddd46-47f6-4948-a897-3b963780991e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:09.232805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c84b11d2d54b43cbac87115e227ef50e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0b466076164a309233726db31b0efd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.240764 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a78e7888-589d-4863-979e-799f613dd8ae tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 200 len: 1303 microversion: 2.1 time: 0.101805 Oct 15 20:51:09.241359 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 497/942] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:09 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 1303 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:09.245902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-56c1a7ca-2497-47fc-8d10-9946c97f1743 req-cc542b63-f518-4c85-a40b-378102411f64 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "73402931-6f69-461a-84ae-e0e91c34b2e5", "tag": "998f23e5-d301-418e-aa99-242986c4578b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:09.255684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-56c1a7ca-2497-47fc-8d10-9946c97f1743 req-cc542b63-f518-4c85-a40b-378102411f64 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.256160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-56c1a7ca-2497-47fc-8d10-9946c97f1743 req-cc542b63-f518-4c85-a40b-378102411f64 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.256588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-56c1a7ca-2497-47fc-8d10-9946c97f1743 req-cc542b63-f518-4c85-a40b-378102411f64 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.268112 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-56c1a7ca-2497-47fc-8d10-9946c97f1743 req-cc542b63-f518-4c85-a40b-378102411f64 service nova] Creating event network-changed:998f23e5-d301-418e-aa99-242986c4578b for instance 73402931-6f69-461a-84ae-e0e91c34b2e5 on nch2-master-1 Oct 15 20:51:09.273023 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-56c1a7ca-2497-47fc-8d10-9946c97f1743 req-cc542b63-f518-4c85-a40b-378102411f64 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030560 Oct 15 20:51:09.273641 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 498/943] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:09.278549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:09.280053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:09.286072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.286550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.286978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.287672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.288125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.288548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.429894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.442936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.444755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.448540 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-05be7367-92e9-405e-bf25-af08ecc2922b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.171444 Oct 15 20:51:09.448540 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 499/944] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:09 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:09.546107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.556025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:09.557212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:09.567151 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-341a8279-12e1-40e4-8b26-558a822bde54 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1470 microversion: 2.1 time: 0.789032 Oct 15 20:51:09.575247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.576247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.576689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.577439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.577908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.579827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.585191 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 446/945] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:08 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1470 bytes in 808 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:09.598120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:09.598858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:09.607943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.609184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.609903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.611110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:09.611776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:09.612919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:09.663963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.672819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.674419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:09.680038 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c188d3f-dc77-46cd-8e48-53b78f550147 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.128230 Oct 15 20:51:09.680437 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 500/946] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:09 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:10.062465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:10.063314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:10.070186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.070681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.071117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.072012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.072462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.072916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.229816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.240102 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c539e7a8-208e-46cf-bf92-eee0ccd694ce tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.182748 Oct 15 20:51:10.252645 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 501/947] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:10 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:10.252645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:10.252645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:10.257128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.257625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.258058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.258770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.259221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.259638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.333592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.342862 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e3f33e1-48c3-450e-9a1d-b5214205431a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1277 microversion: 2.1 time: 0.101062 Oct 15 20:51:10.356173 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 502/948] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:10 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1277 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:10.356173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:10.356173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] [instance: 5d1ddd46-47f6-4948-a897-3b963780991e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:10.356173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.356173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.356746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.356910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.357393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.358175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.376949 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] HTTP exception thrown: Instance 5d1ddd46-47f6-4948-a897-3b963780991e could not be found. Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Request method failure captured: Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e HTTP/1.1 Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyWsMyzwRupGYt7qzizpav-y2s_uX0AmAOgy8PF1ONi6ZGOJ1qHFeBY0u5HM11XNkNHrIM6z4fhjG5b4ekmwD55sfnmLlfDBnugvfkstJDqnMeQV5YrlIjFeHKzll04GJWGU0I-kii3dQZHhTJQqPg0g0ex0G7Ay62TNO8nY2eWG8A Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1f0b466076164a309233726db31b0efd Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersAdminTestJSON-806770791 Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:10.378662 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1f0b466076164a309233726db31b0efd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1f0b466076164a309233726db31b0efd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1f0b466076164a309233726db31b0efd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1f0b466076164a309233726db31b0efd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersAdminTestJSON-806770791 Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1f0b466076164a309233726db31b0efd Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersAdminTestJSON-806770791 Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersAdminTestJSON-806770791-project Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c84b11d2d54b43cbac87115e227ef50e Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersAdminTestJSON-806770791-project Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 5d1ddd46-47f6-4948-a897-3b963780991e could not be found. Oct 15 20:51:10.380203 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '5d1ddd46-47f6-4948-a897-3b963780991e'} Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d1ddd46-47f6-4948-a897-3b963780991e could not be found. Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:10.381415 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d1ddd46-47f6-4948-a897-3b963780991e could not be found. Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d1ddd46-47f6-4948-a897-3b963780991e could not be found. Oct 15 20:51:10.383406 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:10.384531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] Returning 404 to user: Instance 5d1ddd46-47f6-4948-a897-3b963780991e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:10.384531 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-95bfb3d4-cdb2-407e-8d9e-2530c5e314df tempest-ServersAdminTestJSON-806770791 tempest-ServersAdminTestJSON-806770791-project] 10.222.0.16 "GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e" status: 404 len: 111 microversion: 2.1 time: 0.035909 Oct 15 20:51:10.385109 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 503/949] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:10 2022] GET /compute/v2.1/servers/5d1ddd46-47f6-4948-a897-3b963780991e => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:10.477472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:10.478607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:10.483606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.486736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.488324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.488966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.489861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.490597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.491178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.491623 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2685e857-0f9d-4d37-98ac-9d80151a3a81 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.905235 Oct 15 20:51:10.493428 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 447/950] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:09 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 908 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:10.603005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:10.605665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:10.614622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.615463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.615935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.616649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.617137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.617755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.678407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.698003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.699722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.703328 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-53dfc3d7-877e-490b-95a0-e35583f11224 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.230786 Oct 15 20:51:10.703900 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 504/951] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:10 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:10.709026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:10.709870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:10.716448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.717565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.718180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.719087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:10.720490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:10.721127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:10.804002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.815094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.816988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:10.823288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-22a3bb53-304b-4755-be04-36dada675c4c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.118109 Oct 15 20:51:10.824156 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 505/952] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:10 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:11.248364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:11.257298 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fdbb4d2-35d9-4d10-8431-7d7e2d450154 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1462 microversion: 2.1 time: 0.659422 Oct 15 20:51:11.264345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:11.265987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:11.275530 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 448/953] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:10 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1462 bytes in 678 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:11.279946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.280562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.281397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.282404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.283094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.285291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.304251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:11.306151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:11.317511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.318342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.319041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.320724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.321553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.322294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.527027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:11.535508 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d423d989-9cf0-45cd-ae20-ec3bad8e4530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.275461 Oct 15 20:51:11.549993 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 506/954] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:11 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:11.557972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:11.558848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:11.566443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.567360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.568026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.568964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.569672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.570322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.664291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:11.675229 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a35763d-4c07-419d-9154-a87eeee7c05f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1277 microversion: 2.1 time: 0.123668 Oct 15 20:51:11.677086 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 507/955] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:11 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1277 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:11.687162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:11.688003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:11.695017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.695655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.696244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.697266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.697971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.698495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.884469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:11.897015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9c7e2d0-0e64-4cc0-9764-9de568b669e3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1462 microversion: 2.1 time: 0.597792 Oct 15 20:51:11.900225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:11.912361 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-77020b99-de19-4d1b-bd20-50c33c726dbd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.234578 Oct 15 20:51:11.914953 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 449/956] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:11 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1462 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:11.917643 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 508/957] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:11 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:11.933200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:11.936500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:11.939376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:11.940735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:11.951697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.953240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.953955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.964792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.965411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.965997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.966905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.967452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:11.967572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.968090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:11.971706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:11.973389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.181389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.191856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.193750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.201665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66716d19-ab86-4dc4-8654-b0bcc23041a0 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.278117 Oct 15 20:51:12.202380 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 450/958] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:11 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:12.207485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4e19c521-8a79-4ede-9bf6-cdb527c9c1e4 req-7135f9ff-6be8-4091-9b26-c74bf9878c68 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "76141eca-81bf-4771-87e9-fcd4007ff701", "tag": "9585a427-e471-4df6-a4bc-2e935e877931"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:12.230404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4e19c521-8a79-4ede-9bf6-cdb527c9c1e4 req-7135f9ff-6be8-4091-9b26-c74bf9878c68 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.231184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4e19c521-8a79-4ede-9bf6-cdb527c9c1e4 req-7135f9ff-6be8-4091-9b26-c74bf9878c68 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.232656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4e19c521-8a79-4ede-9bf6-cdb527c9c1e4 req-7135f9ff-6be8-4091-9b26-c74bf9878c68 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.245731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.252031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4e19c521-8a79-4ede-9bf6-cdb527c9c1e4 req-7135f9ff-6be8-4091-9b26-c74bf9878c68 service nova] Creating event network-changed:9585a427-e471-4df6-a4bc-2e935e877931 for instance 76141eca-81bf-4771-87e9-fcd4007ff701 on nch2-master-1 Oct 15 20:51:12.257572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.259844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4e19c521-8a79-4ede-9bf6-cdb527c9c1e4 req-7135f9ff-6be8-4091-9b26-c74bf9878c68 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055404 Oct 15 20:51:12.260387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.261428 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 451/959] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:12.261580 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b484f683-a725-43e3-b4c5-2314ef95a615 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.342599 Oct 15 20:51:12.262616 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 509/960] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:11 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:12.269195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0ece2f5-af00-43cd-a157-26c8897ddb33 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'create', calling method: .version_select at 0x7fe74f9d8160>, body: {"keypair": {"name": "tempest-keypair-253147738", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGwx4k5XfTuf/kbKSwbvtvFCocy/P/rCfqfJlKEXg2gLomS9em1uMkmMjFHPMjHy2OA88KUgkg7SLCj5V5KpmZqFp0BoYHtXS5JdOObYvIitmSpXAoakYuSWVQ2zVm/7sw=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:12.289217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d0ece2f5-af00-43cd-a157-26c8897ddb33 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for project 94e961feee714587bb2ffe545d286c5c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:12.298720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d0ece2f5-af00-43cd-a157-26c8897ddb33 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for user e32a1c53b4be44188b07c768405961ed and project 94e961feee714587bb2ffe545d286c5c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:12.331114 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0ece2f5-af00-43cd-a157-26c8897ddb33 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.067983 Oct 15 20:51:12.331832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 452/961] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:51:12 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:12.560202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:12.564944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:12.565597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:12.572733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.573236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.573671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.574365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.574816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.575236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.701525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:12.702441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:12.711006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.711871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.712698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.713747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.714422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.715045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.803266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:12.824827 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5e891d2-175b-4a32-a268-37903c4200c5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.264816 Oct 15 20:51:12.828746 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 510/962] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:12 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:12.940714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:12.941374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:12.958298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.960807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.961437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:12.962975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:12.964878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:12.965376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.268664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:13.336451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.347974 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7a7fb598-521d-438b-a648-f6e135e6e8f6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.413608 Oct 15 20:51:13.349174 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 511/963] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:12 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:13.355862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:13.357345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:13.387178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.391842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.393893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.397542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.398539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.400760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.522553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.526014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:13.532342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.540927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.555607 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-97903019-e764-407b-9ef3-4c98619dea64 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.205140 Oct 15 20:51:13.561965 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 512/964] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:13 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:13.571929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:13.572918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:13.584969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.589705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.590384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.591899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.592517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.593596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.831118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.839783 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-357b9f93-4248-4fff-ab86-1be6bf2662fc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 1.144374 Oct 15 20:51:13.849532 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 453/965] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:12 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 1153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:13.854263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fc5127a9-4637-472a-bdb6-c8156eb1dd0e req-75e71bb0-a9a9-4774-b636-d984215f058d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0ff5b12-729e-4102-8502-c7d39895c72a", "name": "network-vif-plugged", "status": "completed", "tag": "4e7695c0-414a-4d18-aa08-e1bf39f76d1a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:13.876037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc5127a9-4637-472a-bdb6-c8156eb1dd0e req-75e71bb0-a9a9-4774-b636-d984215f058d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.876773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc5127a9-4637-472a-bdb6-c8156eb1dd0e req-75e71bb0-a9a9-4774-b636-d984215f058d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.877379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc5127a9-4637-472a-bdb6-c8156eb1dd0e req-75e71bb0-a9a9-4774-b636-d984215f058d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.889244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.895423 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fc5127a9-4637-472a-bdb6-c8156eb1dd0e req-75e71bb0-a9a9-4774-b636-d984215f058d service nova] Creating event network-vif-plugged:4e7695c0-414a-4d18-aa08-e1bf39f76d1a for instance f0ff5b12-729e-4102-8502-c7d39895c72a on nch1-master-1 Oct 15 20:51:13.901739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fc5127a9-4637-472a-bdb6-c8156eb1dd0e req-75e71bb0-a9a9-4774-b636-d984215f058d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050957 Oct 15 20:51:13.902817 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 454/966] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:13.904626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.907357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:13.909799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:13.910678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:13.920190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.920693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.921177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:13.921879 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1a2dd170-3bb7-44fa-9776-a27eb8722c47 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.345793 Oct 15 20:51:13.929207 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 513/967] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:13 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:13.929680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:13.930268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:13.930720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.090783 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d9a007f8-e835-4833-9a44-8cdf40241a91 tempest-TenantUsagesNegativeTestJSON-68710078 tempest-TenantUsagesNegativeTestJSON-68710078-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/?start=2022-10-14T20%3A51%3A13.896815&end=2022-10-16T20%3A51%3A13.896815" status: 404 len: 112 microversion: - time: 0.160223 Oct 15 20:51:14.091346 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 514/968] 10.222.0.16 () {58 vars in 1316 bytes} [Sat Oct 15 20:51:13 2022] GET /compute/v2.1/os-simple-tenant-usage/?start=2022-10-14T20%3A51%3A13.896815&end=2022-10-16T20%3A51%3A13.896815 => generated 112 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 15 20:51:14.101440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c8456b8-fc6e-4336-8bc5-c5fcaa7a1432 tempest-TenantUsagesNegativeTestJSON-68710078 tempest-TenantUsagesNegativeTestJSON-68710078-project-admin] Calling method '.version_select at 0x7fe7585695e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:14.108606 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-2c8456b8-fc6e-4336-8bc5-c5fcaa7a1432 tempest-TenantUsagesNegativeTestJSON-68710078 tempest-TenantUsagesNegativeTestJSON-68710078-project-admin] HTTP exception thrown: Invalid start time. The start time cannot occur after the end time. Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c8456b8-fc6e-4336-8bc5-c5fcaa7a1432 tempest-TenantUsagesNegativeTestJSON-68710078 tempest-TenantUsagesNegativeTestJSON-68710078-project-admin] Request method failure captured: Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-simple-tenant-usage/5545306632c2429a98f479af041c9db9?start=2022-10-16T20%3A51%3A13.896815&end=2022-10-14T20%3A51%3A13.896815 HTTP/1.1 Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx1BMvWVV6kok12pOzJz5-nbEHs4H5xbD53INyKAPqi3LkHbxZOEeMF33020xEk91zq2C1ovOSPU_8vwbKSxxcWcInZooiP4v7srhMbcfz20yWTrdfVYNaQAQKoyEUcOF8xs3R2SkFvAYTnNOZ9FU-cImwG_wgJ-4G72_cQRGbGqCSQ Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 354cd0c497e74e66a0d7284157067a9c Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TenantUsagesNegativeTestJSON-68710078 Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:51:14.110289 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/354cd0c497e74e66a0d7284157067a9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_354cd0c497e74e66a0d7284157067a9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/354cd0c497e74e66a0d7284157067a9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/354cd0c497e74e66a0d7284157067a9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TenantUsagesNegativeTestJSON-68710078 Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 354cd0c497e74e66a0d7284157067a9c Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TenantUsagesNegativeTestJSON-68710078 Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TenantUsagesNegativeTestJSON-68710078-project-admin Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: e64b5f7e10054320a67b6147553391cd Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TenantUsagesNegativeTestJSON-68710078-project-admin Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid start time. The start time cannot occur after the end time. Oct 15 20:51:14.111542 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '5545306632c2429a98f479af041c9db9'} Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 292, in show Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._show(req, id) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 338, in _show Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi (period_start, period_stop, ignore) = self._get_datetime_range( Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 259, in _get_datetime_range Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Oct 15 20:51:14.112331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid start time. The start time cannot occur after the end time. Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid start time. The start time cannot occur after the end time. Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c8456b8-fc6e-4336-8bc5-c5fcaa7a1432 tempest-TenantUsagesNegativeTestJSON-68710078 tempest-TenantUsagesNegativeTestJSON-68710078-project-admin] Returning 400 to user: Invalid start time. The start time cannot occur after the end time. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:14.113620 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2c8456b8-fc6e-4336-8bc5-c5fcaa7a1432 tempest-TenantUsagesNegativeTestJSON-68710078 tempest-TenantUsagesNegativeTestJSON-68710078-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/5545306632c2429a98f479af041c9db9?start=2022-10-16T20%3A51%3A13.896815&end=2022-10-14T20%3A51%3A13.896815" status: 400 len: 111 microversion: 2.1 time: 0.013914 Oct 15 20:51:14.114316 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 515/969] 10.222.0.16 () {58 vars in 1412 bytes} [Sat Oct 15 20:51:14 2022] GET /compute/v2.1/os-simple-tenant-usage/5545306632c2429a98f479af041c9db9?start=2022-10-16T20%3A51%3A13.896815&end=2022-10-14T20%3A51%3A13.896815 => generated 111 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 15 20:51:14.223635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1cd39665-3e8e-4e0a-8d60-648517128989 tempest-TenantUsagesNegativeTestJSON-480974475 tempest-TenantUsagesNegativeTestJSON-480974475-project] Calling method '.version_select at 0x7fe7584a0f70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:14.227126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1cd39665-3e8e-4e0a-8d60-648517128989 tempest-TenantUsagesNegativeTestJSON-480974475 tempest-TenantUsagesNegativeTestJSON-480974475-project] Policy check for os_compute_api:os-simple-tenant-usage:list failed with credentials {'is_admin': False, 'user_id': 'b90d5845757e4dc58e884ca9f59a71f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5545306632c2429a98f479af041c9db9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1cd39665-3e8e-4e0a-8d60-648517128989 tempest-TenantUsagesNegativeTestJSON-480974475 tempest-TenantUsagesNegativeTestJSON-480974475-project] Request method failure captured: Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-simple-tenant-usage?start=2022-10-14T20%3A51%3A13.896815&end=2022-10-16T20%3A51%3A13.896815&detailed=1 HTTP/1.1 Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx1A5aDnv4-MpRUvIM-KJsAmab_l59--WC-yPBaMwOsDk_OAxtQ46skV-siI4Si36RgXpsGrWFLhXiMBeOf8DL_UI-CYazZhMMLRgI4IhATNYHq4kxCqNTAA9aePvPQNOS_NndIuuR0oq1suM8qxsUNx_vSlHUwuRcOpYwnqRm238gE Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 5545306632c2429a98f479af041c9db9 Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TenantUsagesNegativeTestJSON-480974475 Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:14.228733 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5545306632c2429a98f479af041c9db9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5545306632c2429a98f479af041c9db9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5545306632c2429a98f479af041c9db9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5545306632c2429a98f479af041c9db9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TenantUsagesNegativeTestJSON-480974475 Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 5545306632c2429a98f479af041c9db9 Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TenantUsagesNegativeTestJSON-480974475 Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TenantUsagesNegativeTestJSON-480974475-project Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: b90d5845757e4dc58e884ca9f59a71f0 Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TenantUsagesNegativeTestJSON-480974475-project Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:51:14.230317 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 277, in index Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._index(req) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 297, in _index Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(stu_policies.POLICY_ROOT % 'list') Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:51:14.231322 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:51:14.233329 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:51:14.234458 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Oct 15 20:51:14.234458 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:14.234458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1cd39665-3e8e-4e0a-8d60-648517128989 tempest-TenantUsagesNegativeTestJSON-480974475 tempest-TenantUsagesNegativeTestJSON-480974475-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:14.234458 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1cd39665-3e8e-4e0a-8d60-648517128989 tempest-TenantUsagesNegativeTestJSON-480974475 tempest-TenantUsagesNegativeTestJSON-480974475-project] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage?start=2022-10-14T20%3A51%3A13.896815&end=2022-10-16T20%3A51%3A13.896815&detailed=1" status: 403 len: 123 microversion: 2.1 time: 0.111700 Oct 15 20:51:14.234458 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 516/970] 10.222.0.16 () {58 vars in 1335 bytes} [Sat Oct 15 20:51:14 2022] GET /compute/v2.1/os-simple-tenant-usage?start=2022-10-14T20%3A51%3A13.896815&end=2022-10-16T20%3A51%3A13.896815&detailed=1 => generated 123 bytes in 113 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:51:14.338633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:14.347834 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97c8ff38-e628-4939-9675-28633acf48f5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.443085 Oct 15 20:51:14.348638 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 455/971] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:13 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:14.384563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:14.385595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:14.392907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.393483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.393924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.394638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.395099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.395526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.558925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:14.574961 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70726181-f6c3-49af-8fd4-d3e2f12c48e7 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.196967 Oct 15 20:51:14.576136 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 517/972] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:14 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:14.586830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3603d7e4-113c-4a76-ae51-5c0cbefa02c9 req-c244243f-dda8-4864-8c55-d065bc9c6df6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "22368920-f970-409f-a301-afe70a4ee4c8", "tag": "113c86a4-f077-45e8-aafc-023b1a21c6ef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:14.599236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:14.600054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:14.603481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3603d7e4-113c-4a76-ae51-5c0cbefa02c9 req-c244243f-dda8-4864-8c55-d065bc9c6df6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.604278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3603d7e4-113c-4a76-ae51-5c0cbefa02c9 req-c244243f-dda8-4864-8c55-d065bc9c6df6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.605156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3603d7e4-113c-4a76-ae51-5c0cbefa02c9 req-c244243f-dda8-4864-8c55-d065bc9c6df6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.609298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.610952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.611629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.613077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.613904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.614583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.623477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3603d7e4-113c-4a76-ae51-5c0cbefa02c9 req-c244243f-dda8-4864-8c55-d065bc9c6df6 service nova] Creating event network-changed:113c86a4-f077-45e8-aafc-023b1a21c6ef for instance 22368920-f970-409f-a301-afe70a4ee4c8 on nch2-master-1 Oct 15 20:51:14.631388 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3603d7e4-113c-4a76-ae51-5c0cbefa02c9 req-c244243f-dda8-4864-8c55-d065bc9c6df6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048688 Oct 15 20:51:14.638300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 456/973] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:14.707762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:14.718923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:14.721231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:14.727545 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-354c24ec-f12a-44b7-9e39-56c30ce1bfcb tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.134340 Oct 15 20:51:14.735986 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 518/974] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:14 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:14.873379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:14.874239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:14.881054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.881574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.882007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.882706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.883206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.883788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.948124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:14.948946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:14.952793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:14.958172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.958752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.959276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:14.960117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:14.960703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:14.961236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.096049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.109441 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b89f56bd-06bc-4943-9596-6f697152d4a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.241138 Oct 15 20:51:15.110865 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 457/975] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:14 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:15.125771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-825c1aa1-59c9-4b79-90ee-6ec141b599c9 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '.version_select at 0x7fe74fa9f670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.129888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c1aa1-59c9-4b79-90ee-6ec141b599c9 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.130600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c1aa1-59c9-4b79-90ee-6ec141b599c9 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.131182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c1aa1-59c9-4b79-90ee-6ec141b599c9 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.142396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-825c1aa1-59c9-4b79-90ee-6ec141b599c9 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 1950 microversion: 2.1 time: 0.030173 Oct 15 20:51:15.143809 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 458/976] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/os-migrations => generated 1950 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:15.164287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57b75dd9-5ddb-4b39-b8c9-c2ddc52dfc60 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.166941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.173357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57b75dd9-5ddb-4b39-b8c9-c2ddc52dfc60 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.013124 Oct 15 20:51:15.174516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 459/977] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:15.177022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.178564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.179870 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff9e769a-eaa1-4b7c-9efd-cd49b2659e3c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.235939 Oct 15 20:51:15.180410 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 519/978] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:14 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:15.195156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9c26839-d0b1-492a-af91-4b8e453d5de4 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_resize_flavor_-408468231", "ram": 1024, "disk": 1, "vcpus": 1}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:15.218356 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9c26839-d0b1-492a-af91-4b8e453d5de4 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 498 microversion: 2.1 time: 0.027281 Oct 15 20:51:15.218949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 460/979] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Oct 15 20:51:15 2022] POST /compute/v2.1/flavors => generated 498 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:15.230902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d440c4f0-810a-49fe-a535-cd4b8b8e7520 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.239088 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d440c4f0-810a-49fe-a535-cd4b8b8e7520 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.011738 Oct 15 20:51:15.239943 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 520/980] 10.222.0.16 () {58 vars in 1182 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:51:15.340388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-1582621596", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "d658aa56-cb28-45ff-aa29-056466f85584"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:15.372139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.372978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:15.379531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.380021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.380674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.381815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.382214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.382852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.402726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:15.540351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.548884 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-77ab154b-093e-4286-a476-3faab96e6fe5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1315 microversion: 2.1 time: 0.180862 Oct 15 20:51:15.549874 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 521/981] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1315 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:15.601114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.602366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:15.610711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.611473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.612475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.614230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.614896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.615333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.713453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:15.713942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:15.714422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:15.714837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:15.715252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:15.726057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.726537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.726981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.727713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.728301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.728742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.751459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:15.758314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for user e32a1c53b4be44188b07c768405961ed and project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:15.773728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:15.776455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:15.810167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.820077 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3d8ac7f5-d842-41cb-9850-2acb000ec0e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.222780 Oct 15 20:51:15.833511 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 522/982] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:15.852110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.852843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:15.861101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.861722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.862141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3505e138-f1f1-4239-a40a-f59635e20c6a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.525700 Oct 15 20:51:15.862258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.862797 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 461/983] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:51:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 527 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:51:15.868738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:15.868738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:15.868738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:15.876427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f48770b6-0482-44be-a491-cb09b4ffb771 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:15.878220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f48770b6-0482-44be-a491-cb09b4ffb771 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:15.947661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f48770b6-0482-44be-a491-cb09b4ffb771 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.949645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.957259 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f48770b6-0482-44be-a491-cb09b4ffb771 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1279 microversion: 2.1 time: 0.085854 Oct 15 20:51:15.957920 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 462/984] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1279 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:15.959885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.961466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:15.967211 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-392df684-1a5f-4a66-bbb0-b0a6d9f678c5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.120806 Oct 15 20:51:15.967779 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 523/985] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:15 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:16.134246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:16.134927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:16.154330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.158424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.158950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.159802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.161454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.162551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.208916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:16.210488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:16.224968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.227426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.228010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.234964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.235637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.236100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.536389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:16.550845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:16.561284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:16.565929 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-99134e7a-f987-4df0-9382-605ca1bfac2c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.361603 Oct 15 20:51:16.569018 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 524/986] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:16 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:16.581923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:16.584683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:16.600747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.602844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.605929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.609279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.609906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.610434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.919291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:16.942337 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7659215d-9581-4cf1-a860-96002ebb3caa tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1502 microversion: 2.1 time: 0.371535 Oct 15 20:51:16.949637 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 525/987] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:16 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1502 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:16.962528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:16.966705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:16.976766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.977577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.979001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.987822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:16.988815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:16.992118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:16.995739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.026558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c116cf6-5c97-4bb5-8358-6b02f123a8a9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.895319 Oct 15 20:51:17.031333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 463/988] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:16 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 901 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:17.043044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-35846877"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:17.045675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:17.069623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.071447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.072344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.073341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.076419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.077187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.289779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.298557 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c3a9929-ba61-4833-b58b-dfa63fe71243 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1505 microversion: 2.1 time: 0.347103 Oct 15 20:51:17.299373 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 526/989] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:16 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1505 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:17.320872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:17.321774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:17.328017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.328744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.329448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.330362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.330984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.331573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.410120 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.network.security_group_api [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Adding security group f5689f29-f1fc-4b2b-a301-5dc187e08213 to port 4e7695c0-414a-4d18-aa08-e1bf39f76d1a Oct 15 20:51:17.411907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.421256 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-95d834de-ad3a-40be-b0b3-4b334905a8a5 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1331 microversion: 2.1 time: 0.105952 Oct 15 20:51:17.422080 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 527/990] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:17 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1331 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:17.426856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:17.427614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:17.434185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.436011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.436521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.437454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.437924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.438412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.518269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.527270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.529655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.536510 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e2c15d3-5199-4763-8a28-46467c140377 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.113534 Oct 15 20:51:17.538116 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 528/991] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:17 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:17.599218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:17.601104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:17.611060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.611797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.612592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.613704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:17.614677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:17.615271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:17.776552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.785613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.787092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:17.788271 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-53059f6f-d1d4-4fec-b891-174569c70105 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1564 microversion: 2.25 time: 0.197778 Oct 15 20:51:17.807407 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 529/992] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:17 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1564 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:18.055321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:18.056040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:18.063653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.064209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.064673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.065553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.066016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.066446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.215102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e49e3494-ed68-43d0-82b4-712fd805604d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a/action" status: 202 len: 0 microversion: 2.1 time: 1.182545 Oct 15 20:51:18.220028 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 464/993] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:51:17 2022] POST /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a/action => generated 0 bytes in 1188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:18.225851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3a89e8b-b2bf-4e61-825d-f29697b6d74a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74fa3ce50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:18.228806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:18.240906 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7e6863b8-b5da-4b60-b629-54a39840f0d9 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.189562 Oct 15 20:51:18.241817 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 530/994] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:18 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:18.248083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-898f2746-9671-49bf-bdce-95f88804e7a0 req-512affc9-bf6c-4bdc-b29b-32a2f4a9db15 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0ff5b12-729e-4102-8502-c7d39895c72a", "tag": "4e7695c0-414a-4d18-aa08-e1bf39f76d1a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:18.273011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-898f2746-9671-49bf-bdce-95f88804e7a0 req-512affc9-bf6c-4bdc-b29b-32a2f4a9db15 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.274058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-898f2746-9671-49bf-bdce-95f88804e7a0 req-512affc9-bf6c-4bdc-b29b-32a2f4a9db15 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.275054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-898f2746-9671-49bf-bdce-95f88804e7a0 req-512affc9-bf6c-4bdc-b29b-32a2f4a9db15 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.291600 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-898f2746-9671-49bf-bdce-95f88804e7a0 req-512affc9-bf6c-4bdc-b29b-32a2f4a9db15 service nova] Creating event network-changed:4e7695c0-414a-4d18-aa08-e1bf39f76d1a for instance f0ff5b12-729e-4102-8502-c7d39895c72a on nch1-master-1 Oct 15 20:51:18.298678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-898f2746-9671-49bf-bdce-95f88804e7a0 req-512affc9-bf6c-4bdc-b29b-32a2f4a9db15 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055069 Oct 15 20:51:18.303963 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 531/995] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:18.328010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:18.328837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:18.348636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.349527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.350196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.351133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.351764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.352613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.642971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-b3a89e8b-b2bf-4e61-825d-f29697b6d74a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupInUse", "message": "Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use.", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:18.643911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b3a89e8b-b2bf-4e61-825d-f29697b6d74a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.643911 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3a89e8b-b2bf-4e61-825d-f29697b6d74a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213 HTTP/1.1 Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:18.646265 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: exception: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] Oct 15 20:51:18.647789 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f5689f29-f1fc-4b2b-a301-5dc187e08213'} Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1073, in delete_security_group Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:18.648573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.649736 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:18.650860 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:18.652371 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:18.652371 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:18.652371 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:18.652371 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.652371 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] Oct 15 20:51:18.652371 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:18.655271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3a89e8b-b2bf-4e61-825d-f29697b6d74a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 400 to user: Security Group f5689f29-f1fc-4b2b-a301-5dc187e08213 in use. Oct 15 20:51:18.655271 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e7130584-a1a2-4bbc-addf-4fa3ae99f8ec'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:18.656724 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3a89e8b-b2bf-4e61-825d-f29697b6d74a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213" status: 400 len: 185 microversion: 2.1 time: 0.435425 Oct 15 20:51:18.658657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 465/996] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:18 2022] DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213 => generated 185 bytes in 438 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:51:18.669919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:18.674445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:18.690751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.691597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.693656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.695235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.695914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.696599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.811924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:18.833346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c20442b-b319-4b0e-9832-c6605cf17fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1337 microversion: 2.1 time: 0.173346 Oct 15 20:51:18.836875 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 466/997] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:18 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1337 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:18.851912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:18.854536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:18.879871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.881445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.884576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:18.889890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:18.892719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:18.894983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.051134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.053958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.055153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:19.069539 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-58036d3f-de88-4cc3-9cef-c3406d71b2fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1480 microversion: 2.1 time: 0.745079 Oct 15 20:51:19.071668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.073243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.083635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:19.087062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16b95400-4674-4798-b114-75eeaed50d0f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.248010 Oct 15 20:51:19.088151 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 467/998] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:18 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:19.091259 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 532/999] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:18 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1480 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:19.102401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.103340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.109530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:19.110218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.111766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.112412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.113426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.114080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.114508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.117674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.135817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.136818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.141970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.144743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.145427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.147779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.306150 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24b76036-0d1d-419f-8a56-4f280e2ab0f9 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a/action" status: 202 len: 0 microversion: 2.1 time: 0.216403 Oct 15 20:51:19.307517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 468/1000] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:51:19 2022] POST /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a/action => generated 0 bytes in 219 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:19.313747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9463d878-ac09-4579-aae3-399f0dbf50ac tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '.version_select at 0x7fe74fa60e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.318620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9463d878-ac09-4579-aae3-399f0dbf50ac tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.319329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9463d878-ac09-4579-aae3-399f0dbf50ac tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.319913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9463d878-ac09-4579-aae3-399f0dbf50ac tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.344187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9463d878-ac09-4579-aae3-399f0dbf50ac tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 1950 microversion: 2.1 time: 0.034158 Oct 15 20:51:19.347832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 469/1001] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/os-migrations => generated 1950 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:19.354015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.354874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.371410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.374931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.376238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.377189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.377816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.381575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.428522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:51:19.471452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.487632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.489100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.490341 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a57d9b57-76d0-49b7-b5f4-72c3ca2248f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 1554 microversion: 2.25 time: 0.397771 Oct 15 20:51:19.491235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 533/1002] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 1554 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:19.519436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3db3331a-03f2-41b8-b962-70fcfcdaa570 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe758404310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.549966 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-64f1a33b-09eb-4db2-9151-0c33dbb29f3c tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 204 len: 0 microversion: 2.1 time: 0.200146 Oct 15 20:51:19.551075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 470/1003] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:19 2022] DELETE /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:19.558537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.559420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.571661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.572205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.572730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.573574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.574186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.574725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.587170 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3db3331a-03f2-41b8-b962-70fcfcdaa570 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/84a484d1-2c91-4a62-b8fd-3470464affd5" status: 200 len: 660 microversion: 2.1 time: 0.080618 Oct 15 20:51:19.597703 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 534/1004] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/images/84a484d1-2c91-4a62-b8fd-3470464affd5 => generated 660 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:19.601797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.604679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.617537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.618036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.618559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.619802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.620110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.620632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.627558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:19.816980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.826944 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2484eca6-0100-4b74-a1ca-3f69d9ad1778 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1571 microversion: 2.1 time: 0.234817 Oct 15 20:51:19.827890 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 535/1005] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1571 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:19.832774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.833006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.839068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.839672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.840208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.841424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.841986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.842513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.875533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:19.883493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a2f48d8-3c6c-4a15-a8a9-9c99c87011af tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.330664 Oct 15 20:51:19.884065 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 471/1006] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:19.905587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ad8723e-0831-408d-a8f1-accf1847409a tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '.version_select at 0x7fe74f9e4b80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.909159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ad8723e-0831-408d-a8f1-accf1847409a tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.909839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ad8723e-0831-408d-a8f1-accf1847409a tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.910281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ad8723e-0831-408d-a8f1-accf1847409a tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.920685 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ad8723e-0831-408d-a8f1-accf1847409a tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 2609 microversion: 2.25 time: 0.021960 Oct 15 20:51:19.921425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 472/1007] 10.222.0.16 () {60 vars in 1184 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/os-migrations => generated 2609 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:19.926026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:19.926634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:19.932944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.933468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.933909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:19.935333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:19.935962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:19.936390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.091866 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-301a5a2c-9258-4d4e-87be-fd3481e1c80f tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 200 len: 1631 microversion: 2.1 time: 0.262812 Oct 15 20:51:20.092445 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 536/1008] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 1631 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:20.115660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e07f50d0-a6cc-4f7c-baeb-92d69d1526e0 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9943a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:20.168031 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e07f50d0-a6cc-4f7c-baeb-92d69d1526e0 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/84a484d1-2c91-4a62-b8fd-3470464affd5" status: 200 len: 660 microversion: 2.1 time: 0.059878 Oct 15 20:51:20.170754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:20.172151 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 537/1009] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:20 2022] GET /compute/v2.1/images/84a484d1-2c91-4a62-b8fd-3470464affd5 => generated 660 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:20.178807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:20.180051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:20.195752 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1854d7d4-47e4-4161-a6b6-a50cff8cdfef tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1713 microversion: 2.1 time: 0.261798 Oct 15 20:51:20.195752 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 473/1010] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:19 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1713 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:20.208450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.208913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:20.209098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.209794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:20.214837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.217397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.217945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.218395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.225028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.225680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.226239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.228915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.229662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.233219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.532017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:20.539577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid 73402931-6f69-461a-84ae-e0e91c34b2e5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:20.540759 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-816e967d-31b2-48d3-8e95-cf7f4637e3f6 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.367590 Oct 15 20:51:20.541843 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 538/1011] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:20 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:20.546910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:20.547723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:20.568539 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d8142f5-5796-4887-a563-a4c6d108cc29 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 200 len: 2043 microversion: 2.25 time: 0.373101 Oct 15 20:51:20.568869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.573240 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 474/1012] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:20 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 2043 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:20.573524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.573524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.573524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.573524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.573847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.574098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a6b29fe-c56e-4867-a362-90f6c4af3f51 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Action: 'create', calling method: .version_select at 0x7fe74fa12310>, body: {"keypair": {"name": "tempest-keypair-603506732", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGURKtdqBpFWPBesBcblyYAv3+nFahNBQNq/qq7eMvsQEmRYKBik99ETvUCjqEpfyOQ9dauRdtwq7yunKUIQVJEIHQPTzN9s6lvqviuF0W1HnK1ro4njY19Cd21oJVTxlQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:20.593825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4a6b29fe-c56e-4867-a362-90f6c4af3f51 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:20.599372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4a6b29fe-c56e-4867-a362-90f6c4af3f51 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for user c613201f55b94a528c6c36e5b5772799 and project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:20.630346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a6b29fe-c56e-4867-a362-90f6c4af3f51 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.059648 Oct 15 20:51:20.631263 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 475/1013] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:51:20 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:20.643160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f2e748c4-299f-4aca-ab0c-dae71ea3e188 req-c20d3904-a1f5-42ce-b5cb-f6039efd3ea7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:20.655591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f2e748c4-299f-4aca-ab0c-dae71ea3e188 req-c20d3904-a1f5-42ce-b5cb-f6039efd3ea7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.656365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f2e748c4-299f-4aca-ab0c-dae71ea3e188 req-c20d3904-a1f5-42ce-b5cb-f6039efd3ea7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.656960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f2e748c4-299f-4aca-ab0c-dae71ea3e188 req-c20d3904-a1f5-42ce-b5cb-f6039efd3ea7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.670492 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f2e748c4-299f-4aca-ab0c-dae71ea3e188 req-c20d3904-a1f5-42ce-b5cb-f6039efd3ea7 service nova] Creating event network-changed:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:51:20.677399 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f2e748c4-299f-4aca-ab0c-dae71ea3e188 req-c20d3904-a1f5-42ce-b5cb-f6039efd3ea7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044213 Oct 15 20:51:20.678550 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 476/1014] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:20.684815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:20.686027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ebb0f3c9-66c5-4239-8822-12e2440099f6 req-227bc38c-4d02-4f2b-b503-bbe67bd250c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76141eca-81bf-4771-87e9-fcd4007ff701", "name": "network-vif-plugged", "status": "completed", "tag": "9585a427-e471-4df6-a4bc-2e935e877931"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:20.696732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:20.699555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ebb0f3c9-66c5-4239-8822-12e2440099f6 req-227bc38c-4d02-4f2b-b503-bbe67bd250c9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.700892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ebb0f3c9-66c5-4239-8822-12e2440099f6 req-227bc38c-4d02-4f2b-b503-bbe67bd250c9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.701711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:20.702035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ebb0f3c9-66c5-4239-8822-12e2440099f6 req-227bc38c-4d02-4f2b-b503-bbe67bd250c9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.716600 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0f17730-c002-4b24-8898-a4869359228f tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.170194 Oct 15 20:51:20.717474 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 539/1015] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:20 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:20.719047 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ebb0f3c9-66c5-4239-8822-12e2440099f6 req-227bc38c-4d02-4f2b-b503-bbe67bd250c9 service nova] Creating event network-vif-plugged:9585a427-e471-4df6-a4bc-2e935e877931 for instance 76141eca-81bf-4771-87e9-fcd4007ff701 on nch2-master-1 Oct 15 20:51:20.724649 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ebb0f3c9-66c5-4239-8822-12e2440099f6 req-227bc38c-4d02-4f2b-b503-bbe67bd250c9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044229 Oct 15 20:51:20.725845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 477/1016] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:20.783731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-233543120", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "9c7544ec-ea93-44b4-996f-09fc127b83d3"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:20.852279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:20.853232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:20.861159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.861739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.862261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:20.863210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:20.863761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:20.864272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.060691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] validate_networks() for [('9c7544ec-ea93-44b4-996f-09fc127b83d3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:21.165696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:21.178557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80eaa0b4-1925-495c-b24c-74b4b94bae1b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1571 microversion: 2.1 time: 0.330796 Oct 15 20:51:21.195032 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 478/1017] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:20 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1571 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:21.210570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:21.214225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:21.228111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.228612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.229083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.230029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.230492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.230917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.413648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:21.422451 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d1c58663-523c-418b-b4a3-cb35da38b895 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.225980 Oct 15 20:51:21.423213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 479/1018] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:21 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:21.428339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:21.429537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:21.435280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.435752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.436193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.437029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.437513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.437932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.683763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:21.685013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:21.685677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:21.686187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:21.686652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:21.701924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.702421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.702897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.703716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.704211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.704713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.729894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for project f400a79cb647438f968a895142c5410d. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:21.735862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:21.737453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for user 180d9b7141ff4792ab7ab19e21a3d7ac and project f400a79cb647438f968a895142c5410d. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:21.743613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:21.757203 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1dbb7214-9b71-4963-b573-f6b5bfab445f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1713 microversion: 2.1 time: 0.329363 Oct 15 20:51:21.758141 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 480/1019] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:21 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1713 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:21.766424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:21.767652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:21.776030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:21.779970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:21.783580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.784141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.784683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.785416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.785915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.797319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.854302 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f757a1a-db83-48da-8284-ff11cda65348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 1.076615 Oct 15 20:51:21.862774 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 540/1020] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:51:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 1086 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:51:21.867636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:21.868333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:21.875202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.875691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.876385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.877211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:21.877671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:21.878350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:21.947627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:21.956550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:21.958127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:21.964281 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e6426dbb-4260-441a-917a-3fb3138900f3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.100379 Oct 15 20:51:21.966127 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 541/1021] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:21 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:21.972045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bdfb81c6-601a-42dc-b533-993b14b45c9c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75847eaf0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:22.023582 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bdfb81c6-601a-42dc-b533-993b14b45c9c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/1bf895d1-f7b4-488c-a907-15e88a11c1a4" status: 200 len: 662 microversion: 2.1 time: 0.056044 Oct 15 20:51:22.024867 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 542/1022] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:21 2022] GET /compute/v2.1/images/1bf895d1-f7b4-488c-a907-15e88a11c1a4 => generated 662 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:22.031237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5723e078-fd70-48e1-811c-2a3db3fad914 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:22.032050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5723e078-fd70-48e1-811c-2a3db3fad914 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:22.095562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5723e078-fd70-48e1-811c-2a3db3fad914 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:22.104899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5723e078-fd70-48e1-811c-2a3db3fad914 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:22.106621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5723e078-fd70-48e1-811c-2a3db3fad914 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:22.108615 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5723e078-fd70-48e1-811c-2a3db3fad914 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1261 microversion: 2.25 time: 0.081846 Oct 15 20:51:22.109941 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 543/1023] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:22 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1261 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:22.114676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d4d5f6dd-00b8-4817-a945-07afad2344f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7584a0310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:22.173718 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d4d5f6dd-00b8-4817-a945-07afad2344f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/1bf895d1-f7b4-488c-a907-15e88a11c1a4" status: 200 len: 662 microversion: 2.1 time: 0.062414 Oct 15 20:51:22.174984 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 544/1024] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:22 2022] GET /compute/v2.1/images/1bf895d1-f7b4-488c-a907-15e88a11c1a4 => generated 662 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:22.210914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:22.211984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:22.226188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.227968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.229538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.232561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.234034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.235594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.266176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:22.279702 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-169e232d-d91f-4d20-bc8b-062104c6237a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.516565 Oct 15 20:51:22.283567 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 481/1025] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:21 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:22.446442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:22.447097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:22.457394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.457990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.461395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.462156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.462615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.463218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.494141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:22.512184 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fed9dc02-70f4-4643-8243-ac3aa2f4dae7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1569 microversion: 2.1 time: 0.306507 Oct 15 20:51:22.513667 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 545/1026] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:22 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1569 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:22.536209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1042023238"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:22.536965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:22.544222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.544767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.545436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.546161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.546929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.547373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.694615 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.network.security_group_api [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Adding security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d to port 4e7695c0-414a-4d18-aa08-e1bf39f76d1a Oct 15 20:51:22.790921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:22.801073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-726a2105-1d3a-4a39-b53d-232f13cc7c47 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.358546 Oct 15 20:51:22.802085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 482/1027] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:22 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:22.807100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:22.807951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:22.814346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.815107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.816029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:22.817559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:22.818292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:22.818995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.070654 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9b7a0f2d-86a4-4cf5-b9ac-90f95158075d tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a/action" status: 202 len: 0 microversion: 2.1 time: 0.538214 Oct 15 20:51:23.076342 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 546/1028] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:51:22 2022] POST /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a/action => generated 0 bytes in 545 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:51:23.082687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:23.083586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:23.093535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.094651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.095323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.097414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.099365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.100150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.181703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:23.193731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:23.196365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:23.202668 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b756e6f6-938c-4a42-9580-d677a7fdcbe6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.124827 Oct 15 20:51:23.214835 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 547/1029] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:23 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:23.221465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c859b0f8-ba3a-4144-9874-0567d8a97a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe7584a0310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:23.424095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:23.426241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-c859b0f8-ba3a-4144-9874-0567d8a97a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupInUse", "message": "Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use.", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:23.426967 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c859b0f8-ba3a-4144-9874-0567d8a97a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.426967 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c859b0f8-ba3a-4144-9874-0567d8a97a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d HTTP/1.1 Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:23.429568 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: exception: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] Oct 15 20:51:23.431205 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '22cb2357-e1e2-4d82-b7e4-f6e5b469d66d'} Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1073, in delete_security_group Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:23.431976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.433497 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:23.435318 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c859b0f8-ba3a-4144-9874-0567d8a97a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 400 to user: Security Group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d in use. Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-912a32e7-268e-489c-91eb-aad5c69f7cb2'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c859b0f8-ba3a-4144-9874-0567d8a97a12 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d" status: 400 len: 185 microversion: 2.1 time: 0.214882 Oct 15 20:51:23.438508 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 548/1030] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:23 2022] DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d => generated 185 bytes in 216 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:51:23.441598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-9b60985a-2c47-4f76-9d96-0cdb5f3c6164 req-2461096c-aaa7-426b-afee-87b65ad6e896 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0ff5b12-729e-4102-8502-c7d39895c72a", "tag": "4e7695c0-414a-4d18-aa08-e1bf39f76d1a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:23.445826 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9bbc925-c52d-45d1-811c-24e89d8148c3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1713 microversion: 2.1 time: 0.642587 Oct 15 20:51:23.457844 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 483/1031] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:22 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1713 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:23.468686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9b60985a-2c47-4f76-9d96-0cdb5f3c6164 req-2461096c-aaa7-426b-afee-87b65ad6e896 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.469204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9b60985a-2c47-4f76-9d96-0cdb5f3c6164 req-2461096c-aaa7-426b-afee-87b65ad6e896 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.469785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:23.469944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9b60985a-2c47-4f76-9d96-0cdb5f3c6164 req-2461096c-aaa7-426b-afee-87b65ad6e896 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.470770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:23.484065 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-9b60985a-2c47-4f76-9d96-0cdb5f3c6164 req-2461096c-aaa7-426b-afee-87b65ad6e896 service nova] Creating event network-changed:4e7695c0-414a-4d18-aa08-e1bf39f76d1a for instance f0ff5b12-729e-4102-8502-c7d39895c72a on nch1-master-1 Oct 15 20:51:23.484509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.485408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.486854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.489850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.490546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.493260 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-9b60985a-2c47-4f76-9d96-0cdb5f3c6164 req-2461096c-aaa7-426b-afee-87b65ad6e896 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058556 Oct 15 20:51:23.493653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.494548 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 549/1032] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:23.519113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:23.535968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:23.558467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.562766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.565986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.571117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.574429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.576650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.926875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:23.935289 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6897a766-8655-43c5-9e06-5f4f223c9536 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.439247 Oct 15 20:51:23.936570 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 550/1033] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:23 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:23.941227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:23.941894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:23.949225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.950776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.951388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:23.952442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:23.953102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:23.954034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.004169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:51:24.084850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.093992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.095720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.097192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90ee3e94-6996-4d18-b751-d6b1605f7eb9 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1257 microversion: 2.25 time: 0.638162 Oct 15 20:51:24.097956 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 484/1034] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:23 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1257 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:24.102802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:24.103654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:24.110305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.111057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.111697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.112686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.113375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.114000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.119977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:24.128249 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0933f255-dc18-407f-8a49-db9aa5d5c9d7 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 204 len: 0 microversion: 2.1 time: 0.190119 Oct 15 20:51:24.128833 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 551/1035] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:23 2022] DELETE /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:24.135993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f4937c5-1ceb-4f0f-9b37-cd4e1a56d0a7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75847d160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:24.206662 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f4937c5-1ceb-4f0f-9b37-cd4e1a56d0a7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/50aa02fa-a46c-4e0d-acdb-ee5346eb1a86" status: 200 len: 662 microversion: 2.1 time: 0.076376 Oct 15 20:51:24.207849 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 552/1036] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:24 2022] GET /compute/v2.1/images/50aa02fa-a46c-4e0d-acdb-ee5346eb1a86 => generated 662 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:24.218958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-1475523315", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "6bc761ff-f597-43f8-85e3-57edd2825dc3"}], "security_groups": [{"name": "tempest-securitygroup--79896765"}], "key_name": "tempest-keypair-603506732", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:24.329837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] validate_networks() for [('6bc761ff-f597-43f8-85e3-57edd2825dc3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:24.441746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.452460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-41d238a3-ee53-4ae3-a731-fc3e92331606 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.353230 Oct 15 20:51:24.453425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 485/1037] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:24 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:24.472048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:24.472755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:24.479163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.479737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.480399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.481904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.482571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.483418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.786870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.795907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-320e97b2-87cf-4cd6-a7d0-b2ddfc9eeba4 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1644 microversion: 2.1 time: 0.329090 Oct 15 20:51:24.796493 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 486/1038] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:24 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1644 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:24.801460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a032c45-39f0-4e06-acf0-e97a8f7792bd tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9e4e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:24.815125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:24.815545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:24.816017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:24.816989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:24.817755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:24.828904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.830109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.830740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.832056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.832733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.833364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.847828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a032c45-39f0-4e06-acf0-e97a8f7792bd tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/50aa02fa-a46c-4e0d-acdb-ee5346eb1a86" status: 200 len: 662 microversion: 2.1 time: 0.050087 Oct 15 20:51:24.848498 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 487/1039] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:24 2022] GET /compute/v2.1/images/50aa02fa-a46c-4e0d-acdb-ee5346eb1a86 => generated 662 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:24.853276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:24.854120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:24.855513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:24.862413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for user c613201f55b94a528c6c36e5b5772799 and project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:24.871299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.874284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.874857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.876322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.876445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:24.876882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.877972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.879015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:24.931662 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d3575cc9-3e03-4086-83fe-f2c8a202e4f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.723140 Oct 15 20:51:24.933191 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 553/1040] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:51:24 2022] POST /compute/v2.1/servers => generated 404 bytes in 725 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:51:24.941723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:24.942957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:24.951199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.952073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.952646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.955233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:24.955890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:24.956580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:24.981421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.997188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:24.998530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.008788 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8ee4a514-746a-4880-9404-e483063d0e47 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.158648 Oct 15 20:51:25.010627 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 488/1041] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:24 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:25.033846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0ec5a1a2-c33d-48d9-b679-c3144ff8ccd8 req-a67fbf86-5407-4236-98eb-82cac2de3b13 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "22368920-f970-409f-a301-afe70a4ee4c8", "name": "network-vif-plugged", "status": "completed", "tag": "113c86a4-f077-45e8-aafc-023b1a21c6ef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:25.049174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ec5a1a2-c33d-48d9-b679-c3144ff8ccd8 req-a67fbf86-5407-4236-98eb-82cac2de3b13 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.050225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ec5a1a2-c33d-48d9-b679-c3144ff8ccd8 req-a67fbf86-5407-4236-98eb-82cac2de3b13 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.050978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ec5a1a2-c33d-48d9-b679-c3144ff8ccd8 req-a67fbf86-5407-4236-98eb-82cac2de3b13 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.052450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.061854 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-43147aec-5eb4-4a53-8f32-3258f93b295f tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1645 microversion: 2.1 time: 0.126642 Oct 15 20:51:25.062441 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 554/1042] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:24 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1645 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:25.073966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListImageFiltersTestJSON-server-833669725", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:25.087781 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0ec5a1a2-c33d-48d9-b679-c3144ff8ccd8 req-a67fbf86-5407-4236-98eb-82cac2de3b13 service nova] Creating event network-vif-plugged:113c86a4-f077-45e8-aafc-023b1a21c6ef for instance 22368920-f970-409f-a301-afe70a4ee4c8 on nch2-master-1 Oct 15 20:51:25.096673 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0ec5a1a2-c33d-48d9-b679-c3144ff8ccd8 req-a67fbf86-5407-4236-98eb-82cac2de3b13 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084369 Oct 15 20:51:25.098012 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 489/1043] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:25.112871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e2cd042d-5e72-4311-8603-1b5edccb159b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:25.120108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e2cd042d-5e72-4311-8603-1b5edccb159b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:25.164802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:25.223530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e2cd042d-5e72-4311-8603-1b5edccb159b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.234538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e2cd042d-5e72-4311-8603-1b5edccb159b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1234 microversion: 2.1 time: 0.133441 Oct 15 20:51:25.235615 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 490/1044] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:25 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1234 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:25.250606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:25.255344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:25.270281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.270787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.271224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.276547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.277136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.278168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.466837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:25.467299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:25.467963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:25.468728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:25.471552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:25.487702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.488552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.489156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.489982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.490988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.492409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.511266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Getting quotas for project e5c50f30f706437ea737a3d5d20ea8a1. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:25.518248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Getting quotas for user 3e76b25814474b9db035954b3a437c96 and project e5c50f30f706437ea737a3d5d20ea8a1. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:25.532600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:25.534969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:25.552120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.566066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4133d0f-c93f-4e57-91ad-f0910d03caed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.327953 Oct 15 20:51:25.567798 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 491/1045] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:25 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:25.576141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-544d1559-69ad-447f-80da-2bcd8a123655 req-b848e83d-60c7-43df-9992-770f6f633cc1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1dd90a8-ba2b-462d-b891-c244f6a1722b", "name": "network-vif-deleted", "tag": "110098ee-818d-4570-838f-f57f60ecb527"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:25.586833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-544d1559-69ad-447f-80da-2bcd8a123655 req-b848e83d-60c7-43df-9992-770f6f633cc1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.587577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-544d1559-69ad-447f-80da-2bcd8a123655 req-b848e83d-60c7-43df-9992-770f6f633cc1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.588100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-544d1559-69ad-447f-80da-2bcd8a123655 req-b848e83d-60c7-43df-9992-770f6f633cc1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.603675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-544d1559-69ad-447f-80da-2bcd8a123655 req-b848e83d-60c7-43df-9992-770f6f633cc1 service nova] Creating event network-vif-deleted:110098ee-818d-4570-838f-f57f60ecb527 for instance d1dd90a8-ba2b-462d-b891-c244f6a1722b on nch2-master-1 Oct 15 20:51:25.604298 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-302a53e1-1dc9-441c-adf1-eb8feae8132e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.534622 Oct 15 20:51:25.604955 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 555/1046] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:51:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 536 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:51:25.609588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-544d1559-69ad-447f-80da-2bcd8a123655 req-b848e83d-60c7-43df-9992-770f6f633cc1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039958 Oct 15 20:51:25.610572 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 492/1047] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:25.616970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:25.618406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:25.624408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:25.625025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:25.627973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.633304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.633768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.633990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.634388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.634511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.634826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.635028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.635530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:25.635644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.635980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:25.636403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:25.740602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:25.824575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.842402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.846576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.849588 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f467fa53-c91d-4e35-a0f1-05c4fb795baa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1313 microversion: 2.25 time: 0.231093 Oct 15 20:51:25.851362 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 556/1048] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:25 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1313 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:25.859205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListImageFiltersTestJSON-server-1446736699", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:25.964256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:25.989655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:25.999863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dec5affc-de9a-4325-af4d-983ef3759550 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1319 microversion: 2.1 time: 0.388017 Oct 15 20:51:26.013592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 493/1049] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:25 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1319 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:26.021160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:26.022502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:26.032014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.032768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.035026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.035994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.036686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.037369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.299018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:26.299507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:26.300012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:26.300433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:26.309851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:26.317913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.319007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.319878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.320868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.321792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.323011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.348424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Getting quotas for project e5c50f30f706437ea737a3d5d20ea8a1. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:26.358704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Getting quotas for user 3e76b25814474b9db035954b3a437c96 and project e5c50f30f706437ea737a3d5d20ea8a1. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:26.374674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:26.377890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:26.450155 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3bcd0f4c-6a85-43c6-94e4-7f61d4db4bb8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.596358 Oct 15 20:51:26.458374 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 557/1050] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:51:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 606 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:51:26.478121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:26.478852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:26.494027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.496163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.498237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.499201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.502878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.503583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.641215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:26.657090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:26.662155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:26.690721 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9fe23687-b156-4444-9b20-6171a26f167a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.230891 Oct 15 20:51:26.692915 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 558/1051] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:26 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:26.702916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:26.704938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:26.706502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:26.722311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.725850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.727418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c29cbc7-4959-42c0-9727-1cbe49b3b732 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1644 microversion: 2.1 time: 0.710369 Oct 15 20:51:26.728084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.730273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 494/1052] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:26 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1644 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:26.731368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.736988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.737595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.748869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:26.753098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:26.784463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.785566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.789706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.815440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.818838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.821402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.862811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:26.867659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:26.868127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:26.872284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lazy-loading 'fault' on Instance uuid d1dd90a8-ba2b-462d-b891-c244f6a1722b {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:26.875702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:27.037707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14bd717b05ce4e40a1cd54c2db8474a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f856d315b3b84bdcac94b6d300ef8b2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.058025 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-327655bc-ff4c-4a5c-b970-2735799b895d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 200 len: 1355 microversion: 2.1 time: 0.362616 Oct 15 20:51:27.060654 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 559/1053] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:26 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 1355 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:27.071952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0675ffc7-e690-44f8-909e-e997d9a9e6a6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.076559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0675ffc7-e690-44f8-909e-e997d9a9e6a6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.091451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=12,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:27.284126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0675ffc7-e690-44f8-909e-e997d9a9e6a6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.299409 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0675ffc7-e690-44f8-909e-e997d9a9e6a6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1218 microversion: 2.1 time: 0.234101 Oct 15 20:51:27.302555 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 560/1054] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1218 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:27.310773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.329471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18aa63aa-d27b-40a3-9eeb-098bc6562146 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1230 microversion: 2.1 time: 0.593588 Oct 15 20:51:27.333985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.335064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.336040 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 495/1055] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:26 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1230 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:27.345265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.346249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.346828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.346995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.347444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.348895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.350271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.351524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.372885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.375089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.375543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.376273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.376737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.377205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.592087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.609120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.610951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.614770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22791ee4-271b-4e60-82a3-5df833e45327 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1319 microversion: 2.25 time: 0.276990 Oct 15 20:51:27.621941 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 496/1056] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1319 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:27.627740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.630499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.647234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.647734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.648178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.649703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.657978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.658705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.692433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.702317 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-36177729-cba5-4dfa-9103-5acf37115019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.380723 Oct 15 20:51:27.702989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 561/1057] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:27.728502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.729227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.737326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.739129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.739831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.741423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.742094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.742651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.848641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.853309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.859288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b54ae558-403a-4dbb-9a93-f1b05e0eb808 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1469 microversion: 2.1 time: 0.239962 Oct 15 20:51:27.860572 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 497/1058] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1469 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:27.865532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.867808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.868685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.878206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.878966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.879617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.880816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.881921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.883203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:27.884318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.893242 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8c9a72d5-7831-4676-bdba-1e04d162b13b tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.169508 Oct 15 20:51:27.895178 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 562/1059] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:27.917823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:27.922784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:27.942185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.943171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.944650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:27.945781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:27.947443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:27.948116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.342671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:28.370490 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d75e7d3-bb95-4595-b624-6d8e49c2b7e1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 200 len: 1468 microversion: 2.1 time: 0.473732 Oct 15 20:51:28.371032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:28.371236 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 563/1060] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 1468 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:28.378371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:28.378998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:28.394854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.398411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1e9c8261-3e33-462d-987d-b31d5108492c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 200 len: 1644 microversion: 2.1 time: 0.535926 Oct 15 20:51:28.398997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 498/1061] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:27 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 1644 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:28.401119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.401929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.405562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.406226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.406840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.410028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e2ccd41b-53b3-4c45-859b-e92f63735db7 req-3b22c5e0-9a3c-45ce-8d94-ff966d781bdf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0ff5b12-729e-4102-8502-c7d39895c72a", "name": "network-vif-deleted", "tag": "4e7695c0-414a-4d18-aa08-e1bf39f76d1a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:28.447370 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:28.447848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2ccd41b-53b3-4c45-859b-e92f63735db7 req-3b22c5e0-9a3c-45ce-8d94-ff966d781bdf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.448355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2ccd41b-53b3-4c45-859b-e92f63735db7 req-3b22c5e0-9a3c-45ce-8d94-ff966d781bdf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.448792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2ccd41b-53b3-4c45-859b-e92f63735db7 req-3b22c5e0-9a3c-45ce-8d94-ff966d781bdf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b HTTP/1.1 Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:28.449065 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:28.451237 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'd1dd90a8-ba2b-462d-b891-c244f6a1722b'} Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:28.452045 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:28.454762 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.457874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:28.457874 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-884c2cba-1c4c-4cdd-898a-44c5fe693947 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 404 len: 111 microversion: 2.1 time: 0.079175 Oct 15 20:51:28.457874 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 564/1062] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:28 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:28.461207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:28.462022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:28.469892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.470836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.471486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.472386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.472848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.473425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.485547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e2ccd41b-53b3-4c45-859b-e92f63735db7 req-3b22c5e0-9a3c-45ce-8d94-ff966d781bdf service nova] Creating event network-vif-deleted:4e7695c0-414a-4d18-aa08-e1bf39f76d1a for instance f0ff5b12-729e-4102-8502-c7d39895c72a on nch1-master-1 Oct 15 20:51:28.511218 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e2ccd41b-53b3-4c45-859b-e92f63735db7 req-3b22c5e0-9a3c-45ce-8d94-ff966d781bdf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105976 Oct 15 20:51:28.517138 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 499/1063] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:28.525368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:28.531427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:28.553542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.554037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.554583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.555305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.555773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.566925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.708147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:28.720517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:28.725466 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-afd8eabc-e8be-4c9d-ab61-172465e7270f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1292 microversion: 2.1 time: 0.204352 Oct 15 20:51:28.744465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 500/1064] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:28 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1292 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:28.745030 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-77c00ccd-86e5-4922-b5a2-e35156bf5af6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1214 microversion: 2.1 time: 0.273445 Oct 15 20:51:28.745030 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 565/1065] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:28 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1214 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:28.746746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:28.750055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:28.750676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:28.763139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:28.763368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.763368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.764271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.764615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.765178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.766621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Searching by: {'reservation_id': 'r-mp0gjt31', 'deleted': False, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:51:28.767686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.780578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.781522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.782031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.782792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.783259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.783696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.793111 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 HTTP/1.1 Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:28.796070 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:28.797635 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '56f00c4c-5642-40b7-b0a7-62c3ec8b3b56'} Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:28.798628 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:28.800053 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:28.801995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:28.801995 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7fa8a370-66ce-4745-aaeb-4798f1aade23 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 404 len: 111 microversion: 2.1 time: 0.062758 Oct 15 20:51:28.801995 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 566/1066] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:28 2022] DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:28.806451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:28.807087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:28.813614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.814308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.814915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.815808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:28.816616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:28.817259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:28.946728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:51:29.078152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=43 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:29.082284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.090578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.091374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.091982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.387417 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5838a525-754a-4de8-938d-89287fb80833 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?reservation_id=r-mp0gjt31" status: 200 len: 2930 microversion: 2.1 time: 0.657026 Oct 15 20:51:29.388470 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 501/1067] 10.222.0.16 () {58 vars in 1197 bytes} [Sat Oct 15 20:51:28 2022] GET /compute/v2.1/servers/detail?reservation_id=r-mp0gjt31 => generated 2930 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:29.393839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.394451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:29.411027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.411721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.412343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.413108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.413731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.414329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.519001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.535282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.538085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.539859 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-38a849f5-fbbc-4dea-9f61-fa2b4d4d408a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 0.737350 Oct 15 20:51:29.561466 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 567/1068] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:28 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:29.567603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.568795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:29.580718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.584229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.585547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.585706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.588206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.588918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.622857 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 HTTP/1.1 Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:29.628718 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:29.630240 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '56f00c4c-5642-40b7-b0a7-62c3ec8b3b56'} Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:29.631146 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:29.632514 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.635750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:29.635750 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2213f979-746e-4bd6-be50-085040671bbf tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 404 len: 111 microversion: 2.1 time: 0.067339 Oct 15 20:51:29.635750 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 568/1069] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:29.641904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.642523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:29.649937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.651027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.651941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.652763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.653385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.653776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.679836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.689013 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8bcd247-f979-46b6-bfb0-d16af809b0ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.298931 Oct 15 20:51:29.689838 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 502/1070] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:29.701420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ca3fe51-1ed1-490b-9bad-76490c830754 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.760272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_groups [None req-9ca3fe51-1ed1-490b-9bad-76490c830754 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Querying cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) for 2 instances {{(pid=138688) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Oct 15 20:51:29.761029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ca3fe51-1ed1-490b-9bad-76490c830754 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.761550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ca3fe51-1ed1-490b-9bad-76490c830754 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.762554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ca3fe51-1ed1-490b-9bad-76490c830754 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.832394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.835133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ca3fe51-1ed1-490b-9bad-76490c830754 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/98fcc0af-9b5b-4ce5-ab02-53a563fa5866" status: 200 len: 266 microversion: 2.1 time: 0.143264 Oct 15 20:51:29.841594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 503/1071] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/os-server-groups/98fcc0af-9b5b-4ce5-ab02-53a563fa5866 => generated 266 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:29.842059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.842692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:29.844401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.847309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:29.853432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.853935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.854388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.855179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.856356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.856897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.860395 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-12eacc24-a557-4530-8157-e97652be7970 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.225918 Oct 15 20:51:29.868319 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 569/1072] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:29.874979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-15fb81a8-4cbe-46e7-a2c9-8bce28acc31b req-065327cc-00a6-48f7-9846-07944777751a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "name": "network-vif-plugged", "status": "completed", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:29.878792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a HTTP/1.1 Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:29.881739 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:29.883086 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f0ff5b12-729e-4102-8502-c7d39895c72a'} Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:29.883850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:29.885712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:29.888467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:29.890518 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98d81458-112e-468b-a0ab-9cc1fa7007a0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 404 len: 111 microversion: 2.1 time: 0.052779 Oct 15 20:51:29.894626 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 504/1073] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:29.895757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-15fb81a8-4cbe-46e7-a2c9-8bce28acc31b req-065327cc-00a6-48f7-9846-07944777751a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.902139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f48ecbf7-0051-4e19-a028-3fb696110e2d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '.version_select at 0x7fe74f9a3a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.904601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-15fb81a8-4cbe-46e7-a2c9-8bce28acc31b req-065327cc-00a6-48f7-9846-07944777751a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.905582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-15fb81a8-4cbe-46e7-a2c9-8bce28acc31b req-065327cc-00a6-48f7-9846-07944777751a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.919976 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-15fb81a8-4cbe-46e7-a2c9-8bce28acc31b req-065327cc-00a6-48f7-9846-07944777751a service nova] Creating event network-vif-plugged:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:51:29.926694 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-15fb81a8-4cbe-46e7-a2c9-8bce28acc31b req-065327cc-00a6-48f7-9846-07944777751a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056519 Oct 15 20:51:29.932799 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 570/1074] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:29.939895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.940148 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f48ecbf7-0051-4e19-a028-3fb696110e2d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1690905189" status: 202 len: 0 microversion: 2.1 time: 0.043727 Oct 15 20:51:29.940857 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 505/1075] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:51:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1690905189 => generated 0 bytes in 45 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:29.945191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:29.951120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:29.953653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.954978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.955915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.956936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.957653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.958343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.958602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:29.977862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.979685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.981226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:29.989207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:29.990013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:29.992338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:30.272560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:30.284545 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-afbbbf53-27b8-4224-bcf5-50d8a76a91bc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1278 microversion: 2.1 time: 0.338960 Oct 15 20:51:30.285853 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 506/1076] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1278 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:30.292326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:30.294030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:30.303020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:30.310239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:30.311159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:30.312247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:30.313232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:30.313872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:30.314534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:30.321102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:30.359060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:30.369091 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7f175081-f8b9-4fdb-b19f-717dff611e5d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.433644 Oct 15 20:51:30.369799 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 571/1077] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:29 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:30.377138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-83f55cef-6507-4956-ad92-c818e15f9bf6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe758582280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:30.774051 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1759497-d546-4da4-8ce0-35a24b202886 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1620 microversion: 2.1 time: 0.485397 Oct 15 20:51:30.797977 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 507/1078] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:30 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1620 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:30.807913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:30.809879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:30.819359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:30.820089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:30.820803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:30.821892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:30.822733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:30.823963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.140213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.149614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.154286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.156167 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbb2cb31-a5d5-484d-a724-5d90e29d4586 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 0.357399 Oct 15 20:51:31.162095 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 508/1079] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:30 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:31.169493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:31.170463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:31.179953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.180746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.181545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.182702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.183396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.184283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.233966 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-83f55cef-6507-4956-ad92-c818e15f9bf6 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213" status: 202 len: 0 microversion: 2.1 time: 0.862667 Oct 15 20:51:31.234856 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 572/1080] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:30 2022] DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213 => generated 0 bytes in 864 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:31.240095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:31.240849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:31.247980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.248555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.249120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.250053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.250730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.251235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.412273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.423819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4adbaa0-776d-470d-83d3-33ffce07fb7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1379 microversion: 2.1 time: 0.259592 Oct 15 20:51:31.425277 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 509/1081] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:31 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1379 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:31.430045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:31.431010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:31.440594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.441431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.442033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.442755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.443215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.443638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.566047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.575697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.577190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.582937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5f44c9d-d8cc-48f0-b61e-dc9f2af5665c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.156465 Oct 15 20:51:31.596388 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 510/1082] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:31 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:31.596388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ae01f781-09d7-40e2-aa2c-466f9281ebb1 req-0aa4cfa9-b319-4720-b3e3-ea780b7d1b2d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d16a441b-8807-4eb9-932e-3c2db14bc720", "tag": "c3cd8e3f-84a3-431a-99bb-a4efc4f21579"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:31.600683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ae01f781-09d7-40e2-aa2c-466f9281ebb1 req-0aa4cfa9-b319-4720-b3e3-ea780b7d1b2d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.601325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ae01f781-09d7-40e2-aa2c-466f9281ebb1 req-0aa4cfa9-b319-4720-b3e3-ea780b7d1b2d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.601759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ae01f781-09d7-40e2-aa2c-466f9281ebb1 req-0aa4cfa9-b319-4720-b3e3-ea780b7d1b2d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.617161 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ae01f781-09d7-40e2-aa2c-466f9281ebb1 req-0aa4cfa9-b319-4720-b3e3-ea780b7d1b2d service nova] Creating event network-changed:c3cd8e3f-84a3-431a-99bb-a4efc4f21579 for instance d16a441b-8807-4eb9-932e-3c2db14bc720 on nch1-master-1 Oct 15 20:51:31.626893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ae01f781-09d7-40e2-aa2c-466f9281ebb1 req-0aa4cfa9-b319-4720-b3e3-ea780b7d1b2d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042120 Oct 15 20:51:31.628958 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 511/1083] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:31.632396 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e137d036-6c32-456a-a0bb-00b1919fad79 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 200 len: 1619 microversion: 2.1 time: 0.396080 Oct 15 20:51:31.633514 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 573/1084] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:31 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 1619 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:31.638653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d90e3ff-5152-4542-885a-fa8e6a9758c3 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9e4940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:31.649166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:31.650187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:31.663843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.664570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.665102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.666023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.666577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.667107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.806658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:31.816360 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d077ae31-fa24-4804-aeb9-039655290256 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1276 microversion: 2.1 time: 0.181486 Oct 15 20:51:31.829058 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 574/1085] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:31 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1276 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:31.829058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:31.829058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:31.832925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.833673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.834281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:31.835159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:31.835800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:31.836236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:32.177070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:32.188742 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1a668779-7286-46f5-8f5e-f49bbf2ba17c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.370118 Oct 15 20:51:32.218855 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 575/1086] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:31 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:32.226278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-befaef60-5107-454e-8991-1fe23cc7e1d3 req-0bf28955-6785-46c3-a062-850ca9c22c5c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bec91401-323d-4a36-88b7-88fe24aa813f", "tag": "e75505e8-a2ad-4084-9f6f-4d72bd372f6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:32.238837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-befaef60-5107-454e-8991-1fe23cc7e1d3 req-0bf28955-6785-46c3-a062-850ca9c22c5c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:32.239533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-befaef60-5107-454e-8991-1fe23cc7e1d3 req-0bf28955-6785-46c3-a062-850ca9c22c5c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:32.239755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-befaef60-5107-454e-8991-1fe23cc7e1d3 req-0bf28955-6785-46c3-a062-850ca9c22c5c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:32.253984 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-befaef60-5107-454e-8991-1fe23cc7e1d3 req-0bf28955-6785-46c3-a062-850ca9c22c5c service nova] Creating event network-changed:e75505e8-a2ad-4084-9f6f-4d72bd372f6f for instance bec91401-323d-4a36-88b7-88fe24aa813f on nch2-master-1 Oct 15 20:51:32.256019 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d90e3ff-5152-4542-885a-fa8e6a9758c3 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d" status: 202 len: 0 microversion: 2.1 time: 0.623959 Oct 15 20:51:32.256596 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 512/1087] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:31 2022] DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d => generated 0 bytes in 626 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:32.269219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c2a1e66-2de0-4f08-936c-cd81e7b4dec2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: .version_select at 0x7fe74f8b1280>, body: {"server_group": {"name": "tempest-ServersOnMultiNodesTest-Server-Group-955332158", "policies": ["anti-affinity"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:32.271314 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-befaef60-5107-454e-8991-1fe23cc7e1d3 req-0bf28955-6785-46c3-a062-850ca9c22c5c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050633 Oct 15 20:51:32.273989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 576/1088] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:32.280358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e5847f37-6aa6-4478-a5f1-84eb37730247 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:32.288280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3c2a1e66-2de0-4f08-936c-cd81e7b4dec2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:32.296335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3c2a1e66-2de0-4f08-936c-cd81e7b4dec2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:32.319778 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e5847f37-6aa6-4478-a5f1-84eb37730247 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/668428536" status: 202 len: 0 microversion: 2.1 time: 0.041929 Oct 15 20:51:32.326841 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 577/1089] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:51:32 2022] DELETE /compute/v2.1/flavors/668428536 => generated 0 bytes in 52 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:32.340883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:32.341951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:32.350000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:32.350900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:32.351390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:32.367748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3c2a1e66-2de0-4f08-936c-cd81e7b4dec2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:32.376649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3c2a1e66-2de0-4f08-936c-cd81e7b4dec2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:32.378070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:32.381402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.026s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:32.382057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:32.397511 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c2a1e66-2de0-4f08-936c-cd81e7b4dec2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 192 microversion: 2.1 time: 0.139447 Oct 15 20:51:32.398923 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 513/1090] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 20:51:32 2022] POST /compute/v2.1/os-server-groups => generated 192 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:32.405600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e1f12c9-a601-43d6-a902-677d4b0e95b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'create', calling method: .version_select at 0x7fe74f8b1af0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1155384372", "description": "tempest-description-1175351268"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:32.662474 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e1f12c9-a601-43d6-a902-677d4b0e95b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 0.261675 Oct 15 20:51:32.667684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 514/1091] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:51:32 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:32.680005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ddc52b6-e60f-4b56-97ac-3f22914e92f4 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:32.710563 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ddc52b6-e60f-4b56-97ac-3f22914e92f4 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5892 microversion: 2.1 time: 0.041253 Oct 15 20:51:32.711525 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 515/1092] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:51:32 2022] GET /compute/v2.1/flavors/detail => generated 5892 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:32.716678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:32.717441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:32.724271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:32.724769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:32.725242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:32.725956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:32.726416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:32.726845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:32.881645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:32.891717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-595a9549-9b9d-42f3-aaf6-dfe13acab7a2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1529 microversion: 2.1 time: 0.178950 Oct 15 20:51:32.905247 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 516/1093] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:32 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1529 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:32.905247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1160069315", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 2, "return_reservation_id": true}, "os:scheduler_hints": {"group": "dfac27d3-d366-4ce8-a977-90fbd824648a"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:32.955814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:51:32.980937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:32.990379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:32.992184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:32.993420 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-794e9a4e-9bcf-45ed-ac6a-b374d2a71f22 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 0.663271 Oct 15 20:51:32.994079 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 578/1094] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:32 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:32.999582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.000359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:33.006810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.007383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.007902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.008683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.009265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.009796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.083859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:33.092505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:33.094419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:33.099575 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2dde1899-afef-4635-a3bc-1d5dbc09ce92 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.104308 Oct 15 20:51:33.100620 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 579/1095] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:32 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:33.105835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65012d13-8f3c-4ecd-831c-5b44cb706f1a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Action: 'update', calling method: .version_select at 0x7fe74f994820>, body: {"security_group": {"name": "tempest-sg-hth-1067285659", "description": "tempest-description-hth-582065637"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:33.199294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:33.199740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:33.200449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:33.201242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:33.202033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:33.219355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.219837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.220275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.221113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.221567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.222023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.240896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:33.247539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:33.260354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Going to run 2 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:51:33.263133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:33.313883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:33.319686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:33.357329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:33.362516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:33.376948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=c683117d-d7b3-4e84-beb1-c7ff7c809805,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:51:33.427374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:33.429968 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65012d13-8f3c-4ecd-831c-5b44cb706f1a tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "PUT /compute/v2.1/os-security-groups/884c4004-a391-4f83-a4af-b3cbc510c22f" status: 200 len: 217 microversion: 2.1 time: 0.327581 Oct 15 20:51:33.441151 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 580/1096] 10.222.0.16 () {60 vars in 1290 bytes} [Sat Oct 15 20:51:33 2022] PUT /compute/v2.1/os-security-groups/884c4004-a391-4f83-a4af-b3cbc510c22f => generated 217 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:33.441375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:33.441480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8017f3d7-5892-473b-8fdf-b53ed8dd7f53 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.452586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:33.455482 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8017f3d7-5892-473b-8fdf-b53ed8dd7f53 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1723496106" status: 202 len: 0 microversion: 2.1 time: 0.023134 Oct 15 20:51:33.456992 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 581/1097] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:51:33 2022] DELETE /compute/v2.1/flavors/1723496106 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:33.462049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.462800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:33.469955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.470676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.471190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.472255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.472788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.473479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.478221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:51:33.484356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:51:33.522484 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2ec1697-e9cb-4425-bf9f-951417ccd040 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 0.628733 Oct 15 20:51:33.523840 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 517/1098] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:51:32 2022] POST /compute/v2.1/servers => generated 32 bytes in 631 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:51:33.533695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.534420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:33.541375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.542133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.542763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.544464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.545062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.546301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.696606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:33.706313 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ddfa0fcd-9dfb-41bb-bc2b-8953e1dca6ff tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.248056 Oct 15 20:51:33.707570 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 582/1099] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:33 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:33.714097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.715134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:33.724625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.726244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.726993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.728676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.729428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.730084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.737107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:33.746805 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc8f50a0-af49-46e0-9451-65564efe9e9d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1529 microversion: 2.1 time: 0.219294 Oct 15 20:51:33.748081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 518/1100] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:33 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1529 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:33.753156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03af95b5-fc32-45ac-999f-56b056b672ab tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9e4940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.822343 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03af95b5-fc32-45ac-999f-56b056b672ab tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/884c4004-a391-4f83-a4af-b3cbc510c22f" status: 200 len: 217 microversion: 2.1 time: 0.072923 Oct 15 20:51:33.822952 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 519/1101] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:51:33 2022] GET /compute/v2.1/os-security-groups/884c4004-a391-4f83-a4af-b3cbc510c22f => generated 217 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:33.828221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fddad151-3394-4245-a36c-dcc581862b88 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.851164 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fddad151-3394-4245-a36c-dcc581862b88 tempest-ServersWithSpecificFlavorTestJSON-282582200 tempest-ServersWithSpecificFlavorTestJSON-282582200-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5479 microversion: 2.1 time: 0.026670 Oct 15 20:51:33.852451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 520/1102] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:51:33 2022] GET /compute/v2.1/flavors/detail => generated 5479 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:33.862997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:33.868477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:33.869444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Searching by: {'deleted': False, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:51:33.895756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.896404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.896770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.897854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:33.898315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:33.898736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:33.990432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Listed batch of 3 results from cell out of 998 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:51:34.030059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.039545 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec676ce4-8efd-4ff8-adb4-5e3a418ffc2a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.330503 Oct 15 20:51:34.040436 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 583/1103] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:33 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:34.046479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c78f5c89-5738-460c-b0fd-0a43eba47101 req-83d1e1a4-a80a-4932-b6b2-8f2f8a8a756d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:34.058848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c78f5c89-5738-460c-b0fd-0a43eba47101 req-83d1e1a4-a80a-4932-b6b2-8f2f8a8a756d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.059579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c78f5c89-5738-460c-b0fd-0a43eba47101 req-83d1e1a4-a80a-4932-b6b2-8f2f8a8a756d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.060333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c78f5c89-5738-460c-b0fd-0a43eba47101 req-83d1e1a4-a80a-4932-b6b2-8f2f8a8a756d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.073531 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0dcafe47-0dd7-4263-b86a-f4be8dd10976 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1666 microversion: 2.1 time: 0.219647 Oct 15 20:51:34.082505 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 521/1104] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:51:33 2022] GET /compute/v2.1/servers => generated 1666 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:34.082505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1f14af7-53e5-4184-a003-6474aa609e4e tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.083793 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c78f5c89-5738-460c-b0fd-0a43eba47101 req-83d1e1a4-a80a-4932-b6b2-8f2f8a8a756d service nova] Creating event network-changed:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:51:34.090133 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c78f5c89-5738-460c-b0fd-0a43eba47101 req-83d1e1a4-a80a-4932-b6b2-8f2f8a8a756d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048163 Oct 15 20:51:34.091154 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 584/1105] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:34.097655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-717e067b-a7a3-4e8c-bfd9-50ca77bf1703 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe758531af0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.098946 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1f14af7-53e5-4184-a003-6474aa609e4e tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/d658aa56-cb28-45ff-aa29-056466f85584" status: 202 len: 0 microversion: 2.1 time: 0.022717 Oct 15 20:51:34.099725 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 522/1106] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/flavors/d658aa56-cb28-45ff-aa29-056466f85584 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:34.109305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.109854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.117076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.117644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.118082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.118956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.119375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.119781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.135888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b HTTP/1.1 Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.139103 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.140748 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd1dd90a8-ba2b-462d-b891-c244f6a1722b'} Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.141656 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.143209 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.144621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.144621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c6652c9-5569-4348-bc49-f02325ff8b87 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 404 len: 111 microversion: 2.1 time: 0.040370 Oct 15 20:51:34.144621 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 523/1107] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.150516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.151371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.159682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.160419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.160860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.161592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.162048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.162471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.334621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.347180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.349230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.351584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-125708ce-1d06-4291-8a8a-65969e6c3f56 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 0.205444 Oct 15 20:51:34.352521 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 524/1108] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:34.358631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.359243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.365294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.365785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.366218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.367083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.367692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.368271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.398424 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-717e067b-a7a3-4e8c-bfd9-50ca77bf1703 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/884c4004-a391-4f83-a4af-b3cbc510c22f" status: 202 len: 0 microversion: 2.1 time: 0.305840 Oct 15 20:51:34.399624 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 585/1109] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/os-security-groups/884c4004-a391-4f83-a4af-b3cbc510c22f => generated 0 bytes in 308 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:34.405144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64f2fc33-fe75-4a79-8d87-3ff07b59744a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.429902 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64f2fc33-fe75-4a79-8d87-3ff07b59744a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.028944 Oct 15 20:51:34.445388 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 586/1110] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:34.453148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.454429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.463242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.463443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.463927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.465390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.465919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.466406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.467454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.471664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:34.500404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6770ba4-6762-49d9-ba89-cac2ce7bd55b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1223 microversion: 2.1 time: 0.143461 Oct 15 20:51:34.504110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 525/1111] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1223 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:34.518118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.519251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: d1dd90a8-ba2b-462d-b891-c244f6a1722b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.529744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.530591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.531155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.532030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.532538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.533094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.548346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b HTTP/1.1 Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.549944 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.551340 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd1dd90a8-ba2b-462d-b891-c244f6a1722b'} Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.552040 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. Oct 15 20:51:34.553210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.554365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance d1dd90a8-ba2b-462d-b891-c244f6a1722b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.554447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.554561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02183e72-406f-4caa-b2b6-19d46679c49d tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b" status: 404 len: 111 microversion: 2.1 time: 0.037238 Oct 15 20:51:34.554561 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 526/1112] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/d1dd90a8-ba2b-462d-b891-c244f6a1722b => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.557304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.557805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.563202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.564765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.570081 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-94610a63-d10c-4341-aeaa-8d895e654b11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1560 microversion: 2.91 time: 0.123546 Oct 15 20:51:34.570658 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 587/1113] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1560 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:34.576431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:34.579088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.585283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.585770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.586211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.586921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.587380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.587809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.589479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.590976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.591778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.592781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.593480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.594151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.612223 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a HTTP/1.1 Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.613810 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.615131 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f0ff5b12-729e-4102-8502-c7d39895c72a'} Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.615861 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.617318 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.619838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.619838 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8111f039-0f30-4af4-801d-874224061b61 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 404 len: 111 microversion: 2.1 time: 0.062034 Oct 15 20:51:34.619838 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 527/1114] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.624423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.624921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.631496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.632117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.632610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.633354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.633813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.635473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.649980 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 HTTP/1.1 Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.651772 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.654184 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '56f00c4c-5642-40b7-b0a7-62c3ec8b3b56'} Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.655033 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.656422 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.657921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.657921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6c5d17c-8b30-431d-9332-0548992339a3 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 404 len: 111 microversion: 2.1 time: 0.034141 Oct 15 20:51:34.657921 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 528/1115] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.662570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.663306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: f0ff5b12-729e-4102-8502-c7d39895c72a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.670124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.670609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.671042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.671738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.672191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.672612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.689136 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a HTTP/1.1 Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.690688 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.691887 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f0ff5b12-729e-4102-8502-c7d39895c72a'} Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.692645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. Oct 15 20:51:34.693859 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.694981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Instance f0ff5b12-729e-4102-8502-c7d39895c72a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.694981 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a22b3928-53fd-4378-b862-7a108f837924 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a" status: 404 len: 111 microversion: 2.1 time: 0.033210 Oct 15 20:51:34.694981 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 529/1116] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/f0ff5b12-729e-4102-8502-c7d39895c72a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.697245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.698226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] [instance: 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.703879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.704365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.704804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.705524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:34.705976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:34.706394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:34.711340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:34.718786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Old instance type tempest-test_resize_flavor_-408468231, new instance type m1.heat {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 20:51:34.723613 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] HTTP exception thrown: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Request method failure captured: Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 HTTP/1.1 Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyCc6rfuabEFLtPNOHe3RNd2U1NOQBZFFC0prUXwC9OolWM_z2N_bYa0kpjKqU3H3KNdbH68IqBP7FT-SJDcYGcFc9gAnL5lkqYCDQ9pfI-0fP_b57S-akZgMmwn3jfSu5IaHdozriQS7yBTvvgTSxwMDCE1jVSB7VW0g9nmHVZzuY Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.725443 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f856d315b3b84bdcac94b6d300ef8b2b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f856d315b3b84bdcac94b6d300ef8b2b Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272 Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 14bd717b05ce4e40a1cd54c2db8474a6 Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1712769272-project Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.728430 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '56f00c4c-5642-40b7-b0a7-62c3ec8b3b56'} Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.729190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. Oct 15 20:51:34.730918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.732339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] Returning 404 to user: Instance 56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.732975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:51:34.732975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:51:34.733691 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6e48bc9-a77a-4495-989e-c20c27f030f4 tempest-ServersWithSpecificFlavorTestJSON-1712769272 tempest-ServersWithSpecificFlavorTestJSON-1712769272-project] 10.222.0.16 "GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56" status: 404 len: 111 microversion: 2.1 time: 0.033284 Oct 15 20:51:34.733788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:51:34.733881 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 530/1117] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/56f00c4c-5642-40b7-b0a7-62c3ec8b3b56 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.733985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:51:34.733985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:51:34.737070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9e4a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.816408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:34.817178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:51:34.818031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.818031 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d HTTP/1.1 Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:34.820202 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] Oct 15 20:51:34.821805 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '22cb2357-e1e2-4d82-b7e4-f6e5b469d66d'} Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.822854 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:51:34.824191 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.825618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 22cb2357-e1e2-4d82-b7e4-f6e5b469d66d does not exist Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e55707e2-3667-42ba-822d-b7308cf3b2d5'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.827200 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c0b4f475-12aa-4731-bfa3-ed6170c28530 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d" status: 404 len: 194 microversion: 2.1 time: 0.092802 Oct 15 20:51:34.831549 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 531/1118] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/os-security-groups/22cb2357-e1e2-4d82-b7e4-f6e5b469d66d => generated 194 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:34.840290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.841233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:34.920663 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc0b4d3d-bd14-4126-88f2-ff29167e53d4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3/action" status: 202 len: 0 microversion: 2.1 time: 0.348389 Oct 15 20:51:34.921831 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 588/1119] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:51:34 2022] POST /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3/action => generated 0 bytes in 350 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:34.927457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe758531670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:34.972293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:34.973233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group f5689f29-f1fc-4b2b-a301-5dc187e08213 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:51:34.973792 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.973792 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213 HTTP/1.1 Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:34.975964 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] Oct 15 20:51:34.977575 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'f5689f29-f1fc-4b2b-a301-5dc187e08213'} Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.978517 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:51:34.979758 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.982401 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group f5689f29-f1fc-4b2b-a301-5dc187e08213 does not exist Oct 15 20:51:34.984955 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-95877f02-6e68-46e5-83a5-6cd6fe96759d'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:34.988799 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-74747f85-954e-4e71-ad81-5955f7028faf tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213" status: 404 len: 194 microversion: 2.1 time: 0.061469 Oct 15 20:51:34.992735 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 589/1120] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:34 2022] DELETE /compute/v2.1/os-security-groups/f5689f29-f1fc-4b2b-a301-5dc187e08213 => generated 194 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:35.019515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:35.019771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:35.037832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.038362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.038818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.039697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.040387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.041027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.171648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.172300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.172816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.174130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.174762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.175359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.342538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:35.356230 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cdc5b518-789d-4f93-8c99-d03ae17edcda tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1538 microversion: 2.1 time: 0.361733 Oct 15 20:51:35.357706 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 590/1121] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1538 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:35.367446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe75844b700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:35.453077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:35.454556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 869ab600-7466-46ab-bac3-adb2eaf86a98 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:51:35.455352 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.455352 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98 HTTP/1.1 Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:35.457518 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] Oct 15 20:51:35.458896 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '869ab600-7466-46ab-bac3-adb2eaf86a98'} Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:35.459688 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:51:35.460838 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.461924 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] Oct 15 20:51:35.463136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:35.466567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 869ab600-7466-46ab-bac3-adb2eaf86a98 does not exist Oct 15 20:51:35.466567 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-5f0e7a68-29bc-4e70-986c-d019453134b3'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:35.468000 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0512c7a0-648d-419a-a97c-715cebf05886 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98" status: 404 len: 194 microversion: 2.1 time: 0.109138 Oct 15 20:51:35.472970 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 591/1122] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:35 2022] DELETE /compute/v2.1/os-security-groups/869ab600-7466-46ab-bac3-adb2eaf86a98 => generated 194 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:35.478361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:35.479260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:35.496849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.497481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.498043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.499826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.500527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.501207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.644054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:35.651908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10fb3a4c-b42c-4c47-9b9d-d06e010f595f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.822456 Oct 15 20:51:35.662986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 532/1123] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:34 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:35.672259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:35.673055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:35.688939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.690620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.691521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.692937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:35.696935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:35.697616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:35.891476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:35.899694 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bad7e758-efa9-4ba7-a6e1-5faac1c1d080 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.425338 Oct 15 20:51:35.902817 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 592/1124] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:35 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:35.913702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe758531c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:35.996136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:35.996901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:51:35.997474 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:35.997474 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 HTTP/1.1 Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:36.000418 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] Oct 15 20:51:36.001830 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93'} Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.002629 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:51:36.003788 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.004866 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 does not exist Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-54f9705d-22ca-400e-93a8-2edb642e6f6d'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5ebe8080-fe6c-4f71-bb56-1285d81b1393 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93" status: 404 len: 194 microversion: 2.1 time: 0.097431 Oct 15 20:51:36.008801 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 593/1125] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:35 2022] DELETE /compute/v2.1/os-security-groups/eacc3e8d-8302-4b4f-bcb6-ae42d9a5ed93 => generated 194 bytes in 103 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:36.018406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:36.019165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:36.030480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.031467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.031940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.032804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.033824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.034428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.131660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.180360 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b6e615d7-0414-49e8-bd6a-d9771626f5e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1271 microversion: 2.1 time: 0.171806 Oct 15 20:51:36.185357 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 594/1126] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:36 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1271 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:36.219299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:36.220228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:36.229315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.230616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.231863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.233483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.234163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.235414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.336298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:36.599036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.608780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.616708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.628414 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-83122f3d-8022-43f7-80a5-638f47b71f3c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.440119 Oct 15 20:51:36.654972 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 595/1127] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:36 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:36.664570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9ffee0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:36.667282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.686692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.688147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:36.689507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3555493b-8561-4c45-9176-68522bb1e512 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 1.025227 Oct 15 20:51:36.690251 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 533/1128] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:35 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 1026 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:36.719589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:36.721363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:36.748445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.750532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.751216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.754030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.754800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.756164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.804429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:36.805632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 962c07bd-36a2-4a59-8da5-536db0b7d05e not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:51:36.805632 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.805632 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e HTTP/1.1 Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:36.807436 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] Oct 15 20:51:36.809467 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '962c07bd-36a2-4a59-8da5-536db0b7d05e'} Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.810689 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:51:36.812041 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.813439 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 962c07bd-36a2-4a59-8da5-536db0b7d05e does not exist Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-1e268437-998e-4e31-b829-4481fd85dd25'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f6afe4a-9da5-4e4c-8905-328bfd3deb35 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e" status: 404 len: 194 microversion: 2.1 time: 0.152310 Oct 15 20:51:36.815011 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 596/1129] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:36 2022] DELETE /compute/v2.1/os-security-groups/962c07bd-36a2-4a59-8da5-536db0b7d05e => generated 194 bytes in 154 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:36.819009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:36.819695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:36.864798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.865388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.865839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:36.866560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:36.867028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:36.867460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.291251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=28 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:37.367926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:37.382403 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2bc96685-742d-47e4-8879-0d6a9dca7a49 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.570786 Oct 15 20:51:37.382986 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 597/1130] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:36 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:37.407069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9ffd30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:37.438056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:37.453756 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-765b81be-c60b-47f9-bfba-83d58ed20524 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1538 microversion: 2.1 time: 0.748888 Oct 15 20:51:37.466727 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 534/1131] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:36 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1538 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:37.488222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:37.493451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:37.507074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:37.512739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:37.513532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.514536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:37.515232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:37.515888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.532035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist", "detail": ""}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:51:37.532674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.security_group_api [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Neutron security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 not found {{(pid=138689) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:51:37.534018 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.534018 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 HTTP/1.1 Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:51:37.536126 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: exception: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] Oct 15 20:51:37.537929 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '0f6d06ff-1b65-4a54-a9d2-07e0c7d92910'} Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:37.538889 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:51:37.540207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.541560 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] Oct 15 20:51:37.543399 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:51:37.545068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Security group 0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 does not exist Oct 15 20:51:37.545068 ncdv-master-1 devstack@n-api.service[138689]: Neutron server returns request_ids: ['req-1385afbb-def9-4423-9c3e-d3a6167d5721'] {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:37.545971 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f57c13cf-fd75-440b-aaeb-8e4e38166d28 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910" status: 404 len: 194 microversion: 2.1 time: 0.162049 Oct 15 20:51:37.550182 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 598/1132] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:37 2022] DELETE /compute/v2.1/os-security-groups/0f6d06ff-1b65-4a54-a9d2-07e0c7d92910 => generated 194 bytes in 166 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:37.558971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:37.561857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:37.580014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:37.583016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:37.583669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.584681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:37.587702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:37.588348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.831064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:37.842220 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81482d2a-b31a-4213-9aa2-846371144e1d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1277 microversion: 2.1 time: 0.290550 Oct 15 20:51:37.844342 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 599/1133] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:37 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1277 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:37.854890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:37.855714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:37.873918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:37.874866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:37.877599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.878630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:37.879279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:37.880849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:37.936281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:51:38.010617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.033724 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-beef6310-55c3-407b-a576-7f61caa20050 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.565576 Oct 15 20:51:38.034303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 535/1134] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:37 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:38.039305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:38.045942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:38.059384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.060150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.060764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.061729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.062354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.065625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.104658 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a415e46d-178e-432c-95c9-eaa74164e3b8 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 204 len: 0 microversion: 2.1 time: 0.258343 Oct 15 20:51:38.109387 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 600/1135] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:51:37 2022] DELETE /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:38.117651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-10f89a10-e8a9-466f-b548-911203f8fc8f req-b7edd220-b2fd-43d8-a70d-19221911b090 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a441b-8807-4eb9-932e-3c2db14bc720", "name": "network-vif-plugged", "status": "completed", "tag": "c3cd8e3f-84a3-431a-99bb-a4efc4f21579"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:38.129337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-10f89a10-e8a9-466f-b548-911203f8fc8f req-b7edd220-b2fd-43d8-a70d-19221911b090 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.130094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-10f89a10-e8a9-466f-b548-911203f8fc8f req-b7edd220-b2fd-43d8-a70d-19221911b090 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.130761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-10f89a10-e8a9-466f-b548-911203f8fc8f req-b7edd220-b2fd-43d8-a70d-19221911b090 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.146082 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-10f89a10-e8a9-466f-b548-911203f8fc8f req-b7edd220-b2fd-43d8-a70d-19221911b090 service nova] Creating event network-vif-plugged:c3cd8e3f-84a3-431a-99bb-a4efc4f21579 for instance d16a441b-8807-4eb9-932e-3c2db14bc720 on nch1-master-1 Oct 15 20:51:38.153142 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-10f89a10-e8a9-466f-b548-911203f8fc8f req-b7edd220-b2fd-43d8-a70d-19221911b090 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040641 Oct 15 20:51:38.154367 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 601/1136] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:38.192820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:38.193987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:38.200097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=6,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:38.206621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.207506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.209086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.210085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.210778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.212851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.575835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.582730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:38.586888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.593488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.604685 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee248e8d-4dde-4a76-9339-ca82d53df01a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.569031 Oct 15 20:51:38.612014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 536/1137] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:38 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:38.616912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:38.621994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:38.641295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.641697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.642215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.643017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.643581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.644089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.892786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.908525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.926906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:38.928740 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ee0018e8-6cfc-4717-8f6e-fb2b89897943 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 0.772556 Oct 15 20:51:38.941971 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 602/1138] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:38 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:38.960450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:38.961213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:38.977581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.978501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.979194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:38.982503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:38.983906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:38.984863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.072411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:39.080595 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4242c558-2478-439f-8f3d-4fbf892e42d5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1646 microversion: 2.1 time: 0.467348 Oct 15 20:51:39.100673 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 537/1139] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:38 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1646 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:39.106494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:39.107123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:39.122794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.123297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.125092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.127557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.129539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.130151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.475813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:39.489243 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-84d87c49-c04e-435b-a6cf-82145deda2ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1538 microversion: 2.1 time: 0.387125 Oct 15 20:51:39.495929 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 538/1140] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:39 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1538 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:39.501481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-937f4a50-16e8-4f6d-83c6-71568fc280cb req-7f15c614-f0dc-480f-a2ae-706d9172b790 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "94ed23bf-05fc-41bf-a0d4-547148104a6f", "tag": "a172cd8d-3a0e-48dc-be83-b54ffefe0f32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:39.535147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-937f4a50-16e8-4f6d-83c6-71568fc280cb req-7f15c614-f0dc-480f-a2ae-706d9172b790 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.539133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-937f4a50-16e8-4f6d-83c6-71568fc280cb req-7f15c614-f0dc-480f-a2ae-706d9172b790 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.539691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-937f4a50-16e8-4f6d-83c6-71568fc280cb req-7f15c614-f0dc-480f-a2ae-706d9172b790 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.555773 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-937f4a50-16e8-4f6d-83c6-71568fc280cb req-7f15c614-f0dc-480f-a2ae-706d9172b790 service nova] Creating event network-changed:a172cd8d-3a0e-48dc-be83-b54ffefe0f32 for instance 94ed23bf-05fc-41bf-a0d4-547148104a6f on nch2-master-1 Oct 15 20:51:39.564532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-937f4a50-16e8-4f6d-83c6-71568fc280cb req-7f15c614-f0dc-480f-a2ae-706d9172b790 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065825 Oct 15 20:51:39.565557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 539/1141] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:39.573641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:39.575091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:39.581907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.582647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.583273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.584169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.584940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.585550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.692737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:39.707728 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-894f23e4-640c-42c8-b4b2-533d9a6b51f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.762354 Oct 15 20:51:39.721146 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 603/1142] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:38 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 766 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:39.736324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:39.736955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:39.743244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.743748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.750144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:39.751080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:39.751677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:39.752240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.020672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.034289 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d2a410e6-6b43-467a-a06a-57abb654a4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.324401 Oct 15 20:51:40.039069 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 604/1143] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:39 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:40.060882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:40.061539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:40.078727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.079256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.079709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.080447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.083106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.083558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.276893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.291279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.295361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.306328 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-549a2c15-d9af-4ac7-88e1-8f6750a1f63c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.266129 Oct 15 20:51:40.326554 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 605/1144] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:40 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:40.337929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.353615 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d30bcff-83d8-498c-bcad-3b4355ab3908 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.786461 Oct 15 20:51:40.360585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:40.361503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:40.366298 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 540/1145] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:39 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:40.373540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:40.379792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.382936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.385169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:40.385873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.387327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.389657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.397766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.408842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.412953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.421327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.422384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.423523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.424314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.456830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:40.786846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.796992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.806501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:40.824214 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-578de0fa-5f4a-4be2-9d53-a53e30b80930 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1361 microversion: 2.25 time: 0.489201 Oct 15 20:51:40.826116 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 606/1146] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:40 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1361 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:40.865349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-615e7af6-8619-4f15-8c31-600d6f8d9df1 req-ecd0d9e9-31a3-42eb-a575-4d9eedb0d570 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bec91401-323d-4a36-88b7-88fe24aa813f", "name": "network-vif-plugged", "status": "completed", "tag": "e75505e8-a2ad-4084-9f6f-4d72bd372f6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:40.899291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-615e7af6-8619-4f15-8c31-600d6f8d9df1 req-ecd0d9e9-31a3-42eb-a575-4d9eedb0d570 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:40.900835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-615e7af6-8619-4f15-8c31-600d6f8d9df1 req-ecd0d9e9-31a3-42eb-a575-4d9eedb0d570 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:40.903413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-615e7af6-8619-4f15-8c31-600d6f8d9df1 req-ecd0d9e9-31a3-42eb-a575-4d9eedb0d570 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:40.949539 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-615e7af6-8619-4f15-8c31-600d6f8d9df1 req-ecd0d9e9-31a3-42eb-a575-4d9eedb0d570 service nova] Creating event network-vif-plugged:e75505e8-a2ad-4084-9f6f-4d72bd372f6f for instance bec91401-323d-4a36-88b7-88fe24aa813f on nch2-master-1 Oct 15 20:51:40.971058 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-615e7af6-8619-4f15-8c31-600d6f8d9df1 req-ecd0d9e9-31a3-42eb-a575-4d9eedb0d570 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.143161 Oct 15 20:51:40.972147 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 607/1147] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:40.994037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:40.997690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:41.019697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:41.021762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:41.022839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:41.023750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:41.031699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:41.033660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:41.590212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:41.612888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7234830-2cc1-4872-85dd-fd0261513333 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1646 microversion: 2.1 time: 1.245267 Oct 15 20:51:41.615026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 541/1148] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:40 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1646 bytes in 1248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:41.647324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:41.648230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:41.662518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:41.663346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:41.664028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:41.665287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:41.665966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:41.667179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:41.726044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:41.735162 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b2ef2f6d-1afd-48c8-aa1d-b0f1f77bed17 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.748042 Oct 15 20:51:41.769748 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 608/1149] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:40 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:41.777289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:41.788524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:41.804473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:41.808797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:41.816506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:41.817727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:41.818446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:41.819208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.444516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:42.456357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-736c5c99-a7c0-44ad-9589-cc16bf6fa861 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.839963 Oct 15 20:51:42.457881 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 542/1150] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:41 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:42.472296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f3b0cb35-e04e-4677-9f48-f9b87c1fd0d8 req-ca9fd43c-1e75-4fbb-8995-fa674c8f2e0f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c683117d-d7b3-4e84-beb1-c7ff7c809805", "tag": "146333fb-b86a-486a-91fe-a1828bdef9d9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:42.474911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:42.484153 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6567a2e1-7993-4f98-ae47-926ec7bf7727 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 0.712821 Oct 15 20:51:42.489076 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 609/1151] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:41 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:42.497553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:42.501770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:42.507955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f3b0cb35-e04e-4677-9f48-f9b87c1fd0d8 req-ca9fd43c-1e75-4fbb-8995-fa674c8f2e0f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.508584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f3b0cb35-e04e-4677-9f48-f9b87c1fd0d8 req-ca9fd43c-1e75-4fbb-8995-fa674c8f2e0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.509029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f3b0cb35-e04e-4677-9f48-f9b87c1fd0d8 req-ca9fd43c-1e75-4fbb-8995-fa674c8f2e0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.514462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.519130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.520048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.521018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.521759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.522493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.534791 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f3b0cb35-e04e-4677-9f48-f9b87c1fd0d8 req-ca9fd43c-1e75-4fbb-8995-fa674c8f2e0f service nova] Creating event network-changed:146333fb-b86a-486a-91fe-a1828bdef9d9 for instance c683117d-d7b3-4e84-beb1-c7ff7c809805 on nch1-master-1 Oct 15 20:51:42.548561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f3b0cb35-e04e-4677-9f48-f9b87c1fd0d8 req-ca9fd43c-1e75-4fbb-8995-fa674c8f2e0f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088310 Oct 15 20:51:42.549618 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 543/1152] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:42.560563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:42.561347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:42.575341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.575834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.576275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.576987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.577472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.577906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.725792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:42.742821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:42.748022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:42.765235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5632672-05b4-4ffe-9c8b-ed0c4d1fde49 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.274700 Oct 15 20:51:42.766210 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 610/1153] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:42 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:42.781128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:42.782181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:42.796436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.801733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.802504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.803502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:42.806842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:42.809246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:42.996940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:43.006425 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-464d0d56-4630-4afb-99a8-b75c8ff761cc tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.455173 Oct 15 20:51:43.025792 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 544/1154] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:42 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:43.039512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:43.040361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:43.060900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.063688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.064226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.065341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.068381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.068895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.144609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:43.155021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:43.156951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:43.158590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b6baa05d-d697-42b7-b87a-4ad8d4b9207f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1557 microversion: 2.25 time: 0.390590 Oct 15 20:51:43.159319 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 611/1155] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:42 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1557 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:43.185486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:43.186658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:43.206100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.206665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.207106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.207896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.210721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.211306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.505657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=31 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:43.533861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:43.549806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23662272-f225-4e82-83ac-48f435817113 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.379707 Oct 15 20:51:43.550428 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 612/1156] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:43 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:43.557548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:43.558773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:43.583207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.583971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.584648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.586125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.586603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.587042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.806239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:43.852204 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fee2a25-0b6e-4172-bc7e-97a0ed5b187c tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1646 microversion: 2.1 time: 0.823989 Oct 15 20:51:43.855523 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 545/1157] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:43 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1646 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:43.869265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:43.870875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:43.876607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid d16a441b-8807-4eb9-932e-3c2db14bc720 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:43.891737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.892563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.895048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.896917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.899302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.900674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.942216 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-701787ec-158e-4a43-94ac-38438c0bfec9 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 2049 microversion: 2.25 time: 0.387973 Oct 15 20:51:43.945900 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 613/1158] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:43 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 2049 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:43.956209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:43.967683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:43.970166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:43.986790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.987348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.987794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:43.988843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:43.989751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:43.991025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.454261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:44.467118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d89ee8b-ee37-41d6-8843-e9f3edc63e45 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.609583 Oct 15 20:51:44.468978 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 546/1159] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:43 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:44.478840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3197d2d4-77f9-4a4b-a3a4-26527ee42d4b req-170430ba-febf-4627-9110-0eef8aa4b15a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "72d34005-b40a-4e73-8f98-b0a6d2d0106b", "name": "network-vif-deleted", "tag": "54d23914-40e3-4e01-9162-9682f70a408f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:44.493412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3197d2d4-77f9-4a4b-a3a4-26527ee42d4b req-170430ba-febf-4627-9110-0eef8aa4b15a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.493879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3197d2d4-77f9-4a4b-a3a4-26527ee42d4b req-170430ba-febf-4627-9110-0eef8aa4b15a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.494333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3197d2d4-77f9-4a4b-a3a4-26527ee42d4b req-170430ba-febf-4627-9110-0eef8aa4b15a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.508693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3197d2d4-77f9-4a4b-a3a4-26527ee42d4b req-170430ba-febf-4627-9110-0eef8aa4b15a service nova] Creating event network-vif-deleted:54d23914-40e3-4e01-9162-9682f70a408f for instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b on nch2-master-1 Oct 15 20:51:44.514602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3197d2d4-77f9-4a4b-a3a4-26527ee42d4b req-170430ba-febf-4627-9110-0eef8aa4b15a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044125 Oct 15 20:51:44.515217 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 547/1160] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:44.523399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:44.524098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:44.531173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.531908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.532535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.533459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.534068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.534656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.651156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:44.661153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:44.664264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:44.671174 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c717e217-db3f-4534-b058-fbdf8f4412a6 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.151463 Oct 15 20:51:44.685107 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 548/1161] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:44 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:44.685107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:44.685107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:44.697101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.697101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.697101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.697101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.697527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.697527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.869971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid d16a441b-8807-4eb9-932e-3c2db14bc720 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:44.896816 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4e456677-74e8-423a-b0f7-760fd4cde60f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 2049 microversion: 2.25 time: 0.221580 Oct 15 20:51:44.897858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 549/1162] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:44 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 2049 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:44.904983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:44.906474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:44.913498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.915300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.918031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:44.919112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:44.919869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:44.920551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.124516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.136230 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5d9aeb5d-7c02-494c-883b-eb8319cf7886 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1358 microversion: 2.1 time: 1.188765 Oct 15 20:51:45.137158 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 614/1163] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:43 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1358 bytes in 1190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:45.143420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:45.144029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:45.163900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.164538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.164984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.165952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.166587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.167391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.167872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.181145 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57f5642c-7828-4496-944f-b1a73ea09b68 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.279871 Oct 15 20:51:45.181749 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 550/1164] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:44 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:45.187730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9586908f-7a37-4228-934b-23eaefebd55a req-06e9b477-465a-4cf5-8bb0-73caad491563 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e801c019-e5a3-4208-bed9-ca567e0996fb", "tag": "9e53d8b9-9a15-4432-8b47-46a0152c1409"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:45.199703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9586908f-7a37-4228-934b-23eaefebd55a req-06e9b477-465a-4cf5-8bb0-73caad491563 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.200423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9586908f-7a37-4228-934b-23eaefebd55a req-06e9b477-465a-4cf5-8bb0-73caad491563 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.201009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9586908f-7a37-4228-934b-23eaefebd55a req-06e9b477-465a-4cf5-8bb0-73caad491563 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.224717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9586908f-7a37-4228-934b-23eaefebd55a req-06e9b477-465a-4cf5-8bb0-73caad491563 service nova] Creating event network-changed:9e53d8b9-9a15-4432-8b47-46a0152c1409 for instance e801c019-e5a3-4208-bed9-ca567e0996fb on nch2-master-1 Oct 15 20:51:45.240443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9586908f-7a37-4228-934b-23eaefebd55a req-06e9b477-465a-4cf5-8bb0-73caad491563 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056129 Oct 15 20:51:45.241364 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 551/1165] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:45.246621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:45.247325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:45.254096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.254593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.255080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.255786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.256244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.256697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.298025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.298637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.299190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.299836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lazy-loading 'fault' on Instance uuid 72d34005-b40a-4e73-8f98-b0a6d2d0106b {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:45.378085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.389213 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a718c9f-f0e9-435b-88cb-26a1ef7535d0 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1305 microversion: 2.1 time: 0.146079 Oct 15 20:51:45.403031 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 552/1166] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:45 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1305 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:45.416844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:45.423026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.423803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.424510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.425580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.426325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.426985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.459584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3835567d-ef2c-4a64-8d62-901529d93549 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.048056 Oct 15 20:51:45.461302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 553/1167] 10.222.0.16 () {60 vars in 1217 bytes} [Sat Oct 15 20:51:45 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:51:45.467888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a88f40b9-a686-4e1b-b0f2-d97b7036cb1e req-895316d5-475b-4a23-894d-50e62a0ad45b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "name": "network-vif-plugged", "status": "completed", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:45.505147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a88f40b9-a686-4e1b-b0f2-d97b7036cb1e req-895316d5-475b-4a23-894d-50e62a0ad45b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.507971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a88f40b9-a686-4e1b-b0f2-d97b7036cb1e req-895316d5-475b-4a23-894d-50e62a0ad45b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.508805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a88f40b9-a686-4e1b-b0f2-d97b7036cb1e req-895316d5-475b-4a23-894d-50e62a0ad45b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.541796 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a88f40b9-a686-4e1b-b0f2-d97b7036cb1e req-895316d5-475b-4a23-894d-50e62a0ad45b service nova] Creating event network-vif-plugged:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:51:45.555908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a88f40b9-a686-4e1b-b0f2-d97b7036cb1e req-895316d5-475b-4a23-894d-50e62a0ad45b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091456 Oct 15 20:51:45.557578 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 554/1168] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:45.572267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:45.576202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:45.587609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.588188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.588766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.590342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.590913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.591659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.687834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:45.711853 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb7c264a-2a46-48ab-ad9c-84ca90aa62ed tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720/action" status: 202 len: 0 microversion: 2.25 time: 0.152440 Oct 15 20:51:45.712926 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 555/1169] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:51:45 2022] POST /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720/action => generated 0 bytes in 155 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:45.719484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:45.721239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:45.729427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.733473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.734038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.734567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.735555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.736106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.736669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.746111 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f7a72ba8-54f7-43a4-81a8-ef003b21cc4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.607555 Oct 15 20:51:45.749522 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 615/1170] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:45 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:45.756928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:45.757787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:45.785580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.786319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.786935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.787839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.788459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.790796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.873844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.883834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.886149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:45.893812 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-720b7e80-3073-4ad6-9e41-f41427c48295 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.140749 Oct 15 20:51:45.895218 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 616/1171] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:45 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:45.902575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:45.903413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:45.914114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.914868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.915479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:45.916365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:45.917021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:45.918040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.023946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:46.044189 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4837059-0ace-444f-a956-3f9f94d37ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.325447 Oct 15 20:51:46.063760 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 556/1172] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:45 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:46.161644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:46.162303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:46.166084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=96,nova_api:DELETE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:46.186978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.187602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.188080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.188921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.189464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.199168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.297448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid d16a441b-8807-4eb9-932e-3c2db14bc720 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:46.343479 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-19e0e542-b5c8-4ad6-90d2-b1d999dd7f4f tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 2054 microversion: 2.25 time: 0.441624 Oct 15 20:51:46.344427 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 617/1173] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:45 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 2054 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:46.368036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:46.371811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:46.389082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.393656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.395059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.395997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.401009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.403831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.695355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:46.705796 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f3d0a951-444e-4605-92bb-c23d5d72cf4c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.359742 Oct 15 20:51:46.706675 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 618/1174] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:46 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:46.711225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:46.711844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:46.718923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.719426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.719914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.720626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.721147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.721581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.793790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.795378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.796329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.797298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lazy-loading 'fault' on Instance uuid 72d34005-b40a-4e73-8f98-b0a6d2d0106b {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:46.846776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03bc9235fe1c4fc1b3097a913cbe1052', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869f1ee4c91c4067a3e32e39155b30a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:46.856191 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f2973b49-38ff-46d2-a25f-41860b7f30a5 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 200 len: 1305 microversion: 2.1 time: 0.148778 Oct 15 20:51:46.857445 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 619/1175] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:46 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 1305 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:46.871399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:46.872222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:46.900674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.902058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.902663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:46.903547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:46.904188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:46.904791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.071355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:47.084789 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b6caf54-9892-422d-8073-db27072fb690 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1542 microversion: 2.1 time: 0.928805 Oct 15 20:51:47.085701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 557/1176] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:46 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1542 bytes in 930 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:47.095210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:47.097749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:47.115832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.119491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.120233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.121836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.124553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.125262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.323870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:47.336733 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-19db5c2a-125d-40d1-bb5c-30e651776bae tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.477757 Oct 15 20:51:47.337361 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 620/1177] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:46 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:47.347280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:47.348873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:47.364456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.366056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.366713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.368378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.372294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.372954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.701995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:47.715051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:47.723104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:47.723696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:47.734141 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d8c078cc-ecfe-40af-abf0-b62f1ab6ad6a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.395432 Oct 15 20:51:47.735503 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 621/1178] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:47 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:47.736947 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b2e9b861-3cbf-4efb-986f-3c7038b4d652 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.648131 Oct 15 20:51:47.740829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:47.741543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:47.758975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.760224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 558/1179] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:47 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:47.760827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.761492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.762356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.762852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.763355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.773973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:47.779275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:47.812375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.816293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.816911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:47.821155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:47.824347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:47.824999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.009622 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b852de9-ee60-4b4d-8528-5a9559d877ab tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.272728 Oct 15 20:51:48.019890 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 622/1180] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:51:47 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-interface => generated 285 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:48.025707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:48.027613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:48.037531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.038284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.038957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.041502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.042317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.042933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.051099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:48.140007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid d16a441b-8807-4eb9-932e-3c2db14bc720 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:48.158625 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7321dd4a-a1fd-460e-a3a8-cc78e6de52de tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 2034 microversion: 2.25 time: 0.397038 Oct 15 20:51:48.160152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 559/1181] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:47 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 2034 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:48.165670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:48.166631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] [instance: 72d34005-b40a-4e73-8f98-b0a6d2d0106b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:48.174274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.175383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.176086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.178004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.178749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.179425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.216842 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] HTTP exception thrown: Instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b could not be found. Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Request method failure captured: Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b HTTP/1.1 Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxz-OUryssXhdIUyjM65rlVBELMgIR7taXlglJQyZYTQyR3L9Uj1AkZu1YsxGo1irbQRp060jNS0BKVbpgUNtoG8Yhk9LClGIufcSG9FVtz2ys8RFPqQ-Zpzd-KjUOXb5TmPKfBhuEuMksoEP7EEDanZin5YuHS8o5n-puxEwyQJ4rk Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:51:48.227929 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/869f1ee4c91c4067a3e32e39155b30a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 869f1ee4c91c4067a3e32e39155b30a6 Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-1904250504 Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 03bc9235fe1c4fc1b3097a913cbe1052 Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsTestJSON-1904250504-project Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b could not be found. Oct 15 20:51:48.230154 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '72d34005-b40a-4e73-8f98-b0a6d2d0106b'} Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b could not be found. Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:51:48.230972 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b could not be found. Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b could not be found. Oct 15 20:51:48.235175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:51:48.238524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Returning 404 to user: Instance 72d34005-b40a-4e73-8f98-b0a6d2d0106b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:51:48.238524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36aa2cc2-c535-413a-9db6-cac63d436e01 tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b" status: 404 len: 111 microversion: 2.1 time: 0.068016 Oct 15 20:51:48.238524 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 560/1182] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:48 2022] GET /compute/v2.1/servers/72d34005-b40a-4e73-8f98-b0a6d2d0106b => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:51:48.238524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-010f628a-7dad-4b6d-8e7a-9e4252f81367 req-2269c8d7-8dcd-4421-8e92-971bd0502da9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c683117d-d7b3-4e84-beb1-c7ff7c809805", "name": "network-vif-plugged", "status": "completed", "tag": "146333fb-b86a-486a-91fe-a1828bdef9d9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:48.259105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-010f628a-7dad-4b6d-8e7a-9e4252f81367 req-2269c8d7-8dcd-4421-8e92-971bd0502da9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.259769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-010f628a-7dad-4b6d-8e7a-9e4252f81367 req-2269c8d7-8dcd-4421-8e92-971bd0502da9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.260633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-010f628a-7dad-4b6d-8e7a-9e4252f81367 req-2269c8d7-8dcd-4421-8e92-971bd0502da9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.274443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-010f628a-7dad-4b6d-8e7a-9e4252f81367 req-2269c8d7-8dcd-4421-8e92-971bd0502da9 service nova] Creating event network-vif-plugged:146333fb-b86a-486a-91fe-a1828bdef9d9 for instance c683117d-d7b3-4e84-beb1-c7ff7c809805 on nch1-master-1 Oct 15 20:51:48.281018 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-010f628a-7dad-4b6d-8e7a-9e4252f81367 req-2269c8d7-8dcd-4421-8e92-971bd0502da9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049252 Oct 15 20:51:48.282340 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 561/1183] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:48.288681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": "auto"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:48.293204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:48.301174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.302129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.302801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.303823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:48.304649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:48.305346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:48.345648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'flavor' on Instance uuid d16a441b-8807-4eb9-932e-3c2db14bc720 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:51:48.372302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Going to try to live migrate instance to nch2-master-1 {{(pid=138688) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:51:48.581426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:48.598680 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87aa5dab-5b53-4fbf-958e-09c113922dc8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.577237 Oct 15 20:51:48.601864 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 623/1184] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:48 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:48.614326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-998932ab-6486-4d1b-865d-4fb9d098bfbe tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe75847e430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:49.681073 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-998932ab-6486-4d1b-865d-4fb9d098bfbe tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/35000ccc-1243-4d65-b5b9-450ce2755579" status: 202 len: 0 microversion: 2.1 time: 1.077406 Oct 15 20:51:49.682116 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 624/1185] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:48 2022] DELETE /compute/v2.1/os-security-groups/35000ccc-1243-4d65-b5b9-450ce2755579 => generated 0 bytes in 1079 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:49.688616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:49.689419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:49.695995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:49.696612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:49.697206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:49.698753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:49.699346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:49.699878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:50.095953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:50.106064 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-58f0fe47-6d49-40fe-bf84-9e81ecb20162 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.422296 Oct 15 20:51:50.125728 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 625/1186] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:49 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:50.134158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:50.146795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:50.171161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:50.177605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:50.178536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:50.179603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:50.185375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:50.188149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:50.579623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:50.598885 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-55a7660e-d6ce-4af3-b996-1f1bfdb225ae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.468932 Oct 15 20:51:50.600159 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 626/1187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:50 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:50.615598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:50.616817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:50.633205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:50.635094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:50.637246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:50.641786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:50.642794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:50.643606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:50.943076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:50.963764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:50.965201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:50.977835 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a24ec0af-403d-4014-9a57-438f78588bbe tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.375251 Oct 15 20:51:51.008238 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 627/1188] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:50 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:51.013887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:51.014510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:51.021269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:51.021772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:51.022217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:51.022943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:51.023448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:51.023877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:51.521215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:51.533730 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d01dc5c-1dd2-4763-9dae-0d4d2839f0cf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.523550 Oct 15 20:51:51.554279 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 628/1189] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:51 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:51.564567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3dce62c5-4aac-46d7-8cde-e5962c84a46b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] Calling method '.version_select at 0x7fe74f9d6a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:52.584009 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3dce62c5-4aac-46d7-8cde-e5962c84a46b tempest-SecurityGroupsTestJSON-1904250504 tempest-SecurityGroupsTestJSON-1904250504-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/818202c3-2675-4802-86fc-077c2a1d884e" status: 202 len: 0 microversion: 2.1 time: 1.028151 Oct 15 20:51:52.584636 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 629/1190] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:51:51 2022] DELETE /compute/v2.1/os-security-groups/818202c3-2675-4802-86fc-077c2a1d884e => generated 0 bytes in 1029 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:51:52.590089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-36421571-f6a4-4084-8c4d-324f33557a84 req-51870636-9b33-47de-8d7c-c5b6e298e4ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bec91401-323d-4a36-88b7-88fe24aa813f", "tag": "e75505e8-a2ad-4084-9f6f-4d72bd372f6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:52.617272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-36421571-f6a4-4084-8c4d-324f33557a84 req-51870636-9b33-47de-8d7c-c5b6e298e4ee service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:52.621455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-36421571-f6a4-4084-8c4d-324f33557a84 req-51870636-9b33-47de-8d7c-c5b6e298e4ee service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:52.622118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-36421571-f6a4-4084-8c4d-324f33557a84 req-51870636-9b33-47de-8d7c-c5b6e298e4ee service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:52.652855 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-36421571-f6a4-4084-8c4d-324f33557a84 req-51870636-9b33-47de-8d7c-c5b6e298e4ee service nova] Creating event network-changed:e75505e8-a2ad-4084-9f6f-4d72bd372f6f for instance bec91401-323d-4a36-88b7-88fe24aa813f on nch2-master-1 Oct 15 20:51:52.668733 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-36421571-f6a4-4084-8c4d-324f33557a84 req-51870636-9b33-47de-8d7c-c5b6e298e4ee service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082834 Oct 15 20:51:52.669658 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 630/1191] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:52.685305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:52.688169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:52.711599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:52.712456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:52.714094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:52.715143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:52.717681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:52.719216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:53.352900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:53.370610 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e8cf2c60-c155-4d39-80ea-f4884c714d7b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.691167 Oct 15 20:51:53.375615 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 631/1192] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:52 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:53.400101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-cf2e6687-3694-4430-a843-645c39ce17ac req-5dcf6a43-7ff3-4b3a-8999-656bafc367c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "94ed23bf-05fc-41bf-a0d4-547148104a6f", "name": "network-vif-plugged", "status": "completed", "tag": "a172cd8d-3a0e-48dc-be83-b54ffefe0f32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:53.466275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cf2e6687-3694-4430-a843-645c39ce17ac req-5dcf6a43-7ff3-4b3a-8999-656bafc367c0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:53.474792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5f760100-7743-46fe-89d0-cd76dea50b7b tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720/action" status: 202 len: 0 microversion: 2.25 time: 5.190923 Oct 15 20:51:53.476014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 562/1193] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:51:48 2022] POST /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720/action => generated 0 bytes in 5193 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:51:53.482623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:53.483717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:53.490439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cf2e6687-3694-4430-a843-645c39ce17ac req-5dcf6a43-7ff3-4b3a-8999-656bafc367c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:53.491134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cf2e6687-3694-4430-a843-645c39ce17ac req-5dcf6a43-7ff3-4b3a-8999-656bafc367c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:53.519948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:53.525310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:53.532368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:53.535836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:53.536346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:53.536802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:53.575044 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-cf2e6687-3694-4430-a843-645c39ce17ac req-5dcf6a43-7ff3-4b3a-8999-656bafc367c0 service nova] Creating event network-vif-plugged:a172cd8d-3a0e-48dc-be83-b54ffefe0f32 for instance 94ed23bf-05fc-41bf-a0d4-547148104a6f on nch2-master-1 Oct 15 20:51:53.607413 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-cf2e6687-3694-4430-a843-645c39ce17ac req-5dcf6a43-7ff3-4b3a-8999-656bafc367c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.224575 Oct 15 20:51:53.614604 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 632/1194] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:53.651517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:53.656084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:53.703268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:53.705413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:53.709066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:53.719477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:53.725900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:53.728764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:53.985170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.004008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.016002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.029544 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a40cd20-1323-4c2b-a59c-4ed94af49864 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.411601 Oct 15 20:51:54.031702 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 633/1195] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:53 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:54.038255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:54.039541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:54.049867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:54.050811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:54.051580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:54.052950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:54.053824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:54.054683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:54.426112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.440828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4218100-7e11-4f10-9789-a499d6c0299d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.963008 Oct 15 20:51:54.462267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 563/1196] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:53 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 985 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:54.470742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:54.477920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:54.490845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:54.495428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:54.496142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:54.497123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:54.501698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:54.502387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:54.861516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.870449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.871958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.873171 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d483c3be-261d-4cfc-b090-ae76f11f3ba6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.409831 Oct 15 20:51:54.873731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 564/1197] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:54 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:54.888888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:54.893251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:54.908737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:54.909526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:54.910131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:54.911048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:54.911518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:54.911953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:54.945896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:54.957730 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64ee6ebc-f49b-43c1-a192-a5e7329adf56 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.923931 Oct 15 20:51:54.975407 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 634/1198] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:54 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 937 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:55.062770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:55.063662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:55.077347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:55.078093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:55.080099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:55.082333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:55.085167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:55.085620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:55.186630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:55.190807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:55.197023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:55.202070 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04d7ee54-b5bc-4388-afa9-a8813a43dd10 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.317221 Oct 15 20:51:55.202736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:55.204469 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 565/1199] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:54 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:55.231924 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2e585d9e-9c60-40fa-9a43-b6212ca9ef11 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.174725 Oct 15 20:51:55.245545 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 635/1200] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:55 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:55.469075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:55.469851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:55.477075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:55.480582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:55.483934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:55.484660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:55.496213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:55.497020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:55.503959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:55.906935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:55.909806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:55.918000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:55.919401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:55.920147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:55.921249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:55.922461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:55.923083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.039593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.056893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7612ed76-b978-469c-8c60-215e13dc555c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1318 microversion: 2.1 time: 0.591934 Oct 15 20:51:56.058819 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 566/1201] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:55 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1318 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:56.063713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:56.070957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:56.087323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.090163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.090915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.092219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.094660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.095354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.326477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.335185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.343875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.346241 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-004d9a9e-0ed2-4fe2-8f56-245e681d032e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.450653 Oct 15 20:51:56.347764 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 636/1202] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:55 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:56.355089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:56.370525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:56.378579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.380353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.380907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.382674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.384386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.385500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.452932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.461495 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5bc3070b-f9d4-41fc-939f-75b696a905a4 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.401289 Oct 15 20:51:56.462232 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 567/1203] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:56 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:56.469795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:56.470994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:56.479924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.480617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.481192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.481893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.482546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.483037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.581191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.594845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.596806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.602781 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c971ad45-a5cd-4caf-b378-ff68c6594f88 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1555 microversion: 2.91 time: 0.139452 Oct 15 20:51:56.604580 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 568/1204] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:56 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1555 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:56.716900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ee9630f2-14d7-4b5b-b512-9076334c3d59 req-3d680e2f-5392-48d5-b2ab-2c7e32d3671b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e801c019-e5a3-4208-bed9-ca567e0996fb", "name": "network-vif-plugged", "status": "completed", "tag": "9e53d8b9-9a15-4432-8b47-46a0152c1409"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:56.735606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee9630f2-14d7-4b5b-b512-9076334c3d59 req-3d680e2f-5392-48d5-b2ab-2c7e32d3671b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:56.736304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee9630f2-14d7-4b5b-b512-9076334c3d59 req-3d680e2f-5392-48d5-b2ab-2c7e32d3671b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:56.736940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee9630f2-14d7-4b5b-b512-9076334c3d59 req-3d680e2f-5392-48d5-b2ab-2c7e32d3671b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:56.762739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ee9630f2-14d7-4b5b-b512-9076334c3d59 req-3d680e2f-5392-48d5-b2ab-2c7e32d3671b service nova] Creating event network-vif-plugged:9e53d8b9-9a15-4432-8b47-46a0152c1409 for instance e801c019-e5a3-4208-bed9-ca567e0996fb on nch2-master-1 Oct 15 20:51:56.769195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ee9630f2-14d7-4b5b-b512-9076334c3d59 req-3d680e2f-5392-48d5-b2ab-2c7e32d3671b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062872 Oct 15 20:51:56.770363 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 569/1205] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:56.956316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:56.970345 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c595578e-7753-407a-a816-b29dd5774cc3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.619450 Oct 15 20:51:56.971257 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 637/1206] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:56 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:57.092825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:57.094144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:57.102049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.102837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.103852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.104814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.105463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.106284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.315706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.324915 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5bc977b2-1c77-4d67-95ea-bfedd06d69a8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1468 microversion: 2.1 time: 0.240202 Oct 15 20:51:57.337680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 570/1207] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:57 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1468 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:57.348561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:57.354259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:57.360749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.361353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.362646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.363563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.364200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.364903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.380007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:57.382979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:57.405564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.406058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.406880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.407208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.407732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.408090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.557945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.568405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.569893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.571126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f186ce50-e9da-4a1a-9f4a-df4fec819085 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.195189 Oct 15 20:51:57.571676 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 571/1208] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:57 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:51:57.577265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:57.578298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:57.584456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.585099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.585543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.586235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.587037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.587726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.592609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:57.773515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.799440 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-980f2336-40e6-4991-8ec0-9352875748d8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1319 microversion: 2.1 time: 0.220662 Oct 15 20:51:57.820538 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 572/1209] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:57 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1319 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:57.831724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:57.832698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:57.853292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.854709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.855423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.856247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:57.856818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:57.857401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:57.930831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.947570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.949346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:57.961739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04cedbfc-8eba-4077-9dc0-17a6a633a6c3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1537 microversion: 2.91 time: 0.138319 Oct 15 20:51:57.979253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 573/1210] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:57 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1537 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:57.996091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d19ffc55-3dbd-4d9a-8626-54f81f6a9763 req-b8d023f8-745a-4758-bc12-e5d877519d58 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a441b-8807-4eb9-932e-3c2db14bc720", "name": "network-vif-unplugged", "status": "completed", "tag": "c3cd8e3f-84a3-431a-99bb-a4efc4f21579"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:58.013837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d19ffc55-3dbd-4d9a-8626-54f81f6a9763 req-b8d023f8-745a-4758-bc12-e5d877519d58 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.013837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d19ffc55-3dbd-4d9a-8626-54f81f6a9763 req-b8d023f8-745a-4758-bc12-e5d877519d58 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.014332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d19ffc55-3dbd-4d9a-8626-54f81f6a9763 req-b8d023f8-745a-4758-bc12-e5d877519d58 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.027681 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d19ffc55-3dbd-4d9a-8626-54f81f6a9763 req-b8d023f8-745a-4758-bc12-e5d877519d58 service nova] Creating event network-vif-unplugged:c3cd8e3f-84a3-431a-99bb-a4efc4f21579 for instance d16a441b-8807-4eb9-932e-3c2db14bc720 on nch1-master-1 Oct 15 20:51:58.033111 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d19ffc55-3dbd-4d9a-8626-54f81f6a9763 req-b8d023f8-745a-4758-bc12-e5d877519d58 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.043926 Oct 15 20:51:58.033861 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 574/1211] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:51:57 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:58.039066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:58.039928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:58.045332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:58.047493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.048087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.048443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.049411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.049789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.050228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.055067 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2e19dcb6-dd24-48c6-8e7d-7b66d2e0934d tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1467 microversion: 2.1 time: 0.710209 Oct 15 20:51:58.056542 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 638/1212] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:57 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1467 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:58.087431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-1177473349"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:51:58.091282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:58.100151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.101383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.102112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.103135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.104364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.104957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.242322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:58.255544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e96f6a45-3132-4191-9fd8-dd785a5d2b2c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.216897 Oct 15 20:51:58.256188 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 575/1213] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:58 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:58.347095 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c03cff2-92db-4838-abcf-ae14dd014153 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "POST /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11/action" status: 202 len: 0 microversion: 2.1 time: 0.264629 Oct 15 20:51:58.347749 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 639/1214] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:51:58 2022] POST /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11/action => generated 0 bytes in 266 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:51:58.358915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f7fa6af-92cd-4b23-9abe-4f2a71b82954 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74faa0af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:58.413205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f7fa6af-92cd-4b23-9abe-4f2a71b82954 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.058061 Oct 15 20:51:58.414548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 576/1215] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:58 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:58.598783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:58.599499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:58.607128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.607628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.608068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.608774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.609267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.609704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.827298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:58.828077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:58.834681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.835286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.836053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.836914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:58.837749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:58.839000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:58.964605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:58.973749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:58.975523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:58.976955 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1769ad24-f711-4c07-8805-3ce0e9a6cca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.382451 Oct 15 20:51:58.977450 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 640/1216] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:58 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:58.991492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:58.994943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:58.996210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:59.003393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.003699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.004200 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e673b0a6-6ab1-4e22-995b-1753c658f1f8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1468 microversion: 2.1 time: 0.182078 Oct 15 20:51:59.004346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.004997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.005623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.006125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.013957 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 577/1217] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:58 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1468 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:59.027022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:59.027647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:59.034150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.034856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.035461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.036318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.036945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.037549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.158143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.162663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:51:59.169831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.171444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.179234 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-38f32f83-0cf0-4b7b-89b1-050b4c6e70b3 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1537 microversion: 2.91 time: 0.190314 Oct 15 20:51:59.180463 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 641/1218] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:51:58 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1537 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:51:59.206837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.222885 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8f33c7f-d88d-455b-9d87-5d3b71ec6355 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805" status: 200 len: 1468 microversion: 2.1 time: 0.199895 Oct 15 20:51:59.223844 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 578/1219] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:59 2022] GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805 => generated 1468 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:59.247586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:59.254195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.255585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Searching by: {'reservation_id': 'r-fwaylv6l', 'deleted': False, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe'} {{(pid=138689) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:51:59.263801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.263801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.263801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.264652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.265541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.266556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.280690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:59.281342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:59.293200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.293709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.294153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.296365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.298124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.298742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.420762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.multi_cell_list [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138689) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:51:59.511704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.519667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.520225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.520701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.587076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:51:59.599898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d965aed1-862a-40a8-b1a6-4f2fb2f77f61 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.323801 Oct 15 20:51:59.600494 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 579/1220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:59 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:59.605454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3cfb20d6-eec7-4f21-a30d-fd582076ef30 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fa60670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:59.671209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3cfb20d6-eec7-4f21-a30d-fd582076ef30 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.069653 Oct 15 20:51:59.672453 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 580/1221] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:51:59 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:51:59.702643 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0b9e021c-5b6c-42ad-9956-6f29c4d5a274 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?reservation_id=r-fwaylv6l" status: 200 len: 2929 microversion: 2.1 time: 0.460519 Oct 15 20:51:59.703768 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 642/1222] 10.222.0.16 () {58 vars in 1197 bytes} [Sat Oct 15 20:51:59 2022] GET /compute/v2.1/servers/detail?reservation_id=r-fwaylv6l => generated 2929 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:51:59.730685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31105b78-883f-4792-9557-f70df3434f6b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:59.748804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_groups [None req-31105b78-883f-4792-9557-f70df3434f6b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Querying cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) for 2 instances {{(pid=138688) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Oct 15 20:51:59.749598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31105b78-883f-4792-9557-f70df3434f6b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.750283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31105b78-883f-4792-9557-f70df3434f6b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.750722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31105b78-883f-4792-9557-f70df3434f6b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.771237 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31105b78-883f-4792-9557-f70df3434f6b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/dfac27d3-d366-4ce8-a977-90fbd824648a" status: 200 len: 270 microversion: 2.1 time: 0.046257 Oct 15 20:51:59.772244 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 581/1223] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:51:59 2022] GET /compute/v2.1/os-server-groups/dfac27d3-d366-4ce8-a977-90fbd824648a => generated 270 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:51:59.784173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:51:59.784795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:51:59.795696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.796271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.799309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:51:59.801530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:51:59.802126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:51:59.802571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.006458 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9d977a87-bcea-4f70-ae2e-7c6cd1bc7cd6 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805" status: 200 len: 1619 microversion: 2.1 time: 0.226693 Oct 15 20:52:00.009177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:00.009853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:00.018401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.018904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.019343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.022971 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 643/1224] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:51:59 2022] GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805 => generated 1619 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:00.024199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.024844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.027029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.048678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:00.055460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:00.069190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.070136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.070860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.073465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.075036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.076263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.259126 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1bd81ecf-45c2-4e4e-9b83-9a2b2113c6dc tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 200 len: 1619 microversion: 2.1 time: 0.230279 Oct 15 20:52:00.260505 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 644/1225] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:00 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 1619 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:00.271108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a91d6b48-ec79-456e-8684-725bac1b2445 req-c6a82dd3-962c-441b-9b9e-f018981367a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d16a441b-8807-4eb9-932e-3c2db14bc720", "tag": "c3cd8e3f-84a3-431a-99bb-a4efc4f21579"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:00.297529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:00.303731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a91d6b48-ec79-456e-8684-725bac1b2445 req-c6a82dd3-962c-441b-9b9e-f018981367a4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.304546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a91d6b48-ec79-456e-8684-725bac1b2445 req-c6a82dd3-962c-441b-9b9e-f018981367a4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.306040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a91d6b48-ec79-456e-8684-725bac1b2445 req-c6a82dd3-962c-441b-9b9e-f018981367a4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.327541 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a91d6b48-ec79-456e-8684-725bac1b2445 req-c6a82dd3-962c-441b-9b9e-f018981367a4 service nova] Creating event network-changed:c3cd8e3f-84a3-431a-99bb-a4efc4f21579 for instance d16a441b-8807-4eb9-932e-3c2db14bc720 on nch1-master-1 Oct 15 20:52:00.334271 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a91d6b48-ec79-456e-8684-725bac1b2445 req-c6a82dd3-962c-441b-9b9e-f018981367a4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072336 Oct 15 20:52:00.334969 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 645/1226] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:00.345778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:00.346899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:00.373437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.374247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.374860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.375769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.376861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.377748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.430435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:00.459048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:00.461455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:00.463262 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-59c4a4e3-e8df-4056-8df4-7bcd6f0f981b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.458044 Oct 15 20:52:00.464483 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 582/1227] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:00 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:00.534030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1129943168", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}, "os:scheduler_hints": {"different_host": "0028073f-e418-41fb-830e-0c49db2dee33"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:00.541581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:00.558104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:00.573924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:00.585520 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-02116b99-f6b3-477e-b773-a14c347f0b7c tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1540 microversion: 2.91 time: 0.248730 Oct 15 20:52:00.586373 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 646/1228] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:00 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1540 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:00.638558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Action: 'action', calling method: >, body: {"unshelve": {"host": "nch2-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:00.646211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:00.659082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:00.662201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.662805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.664279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.666045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.669083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.669661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.761198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "nch2-master-1". {{(pid=138689) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 20:52:00.852127 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-96fef731-921e-4a6b-92ed-106f3d236b12 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action" status: 202 len: 0 microversion: 2.91 time: 0.237292 Oct 15 20:52:00.857245 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 647/1229] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:52:00 2022] POST /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee/action => generated 0 bytes in 241 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:52:00.867228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:00.868266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:00.877298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.878326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.879251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:00.880314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:00.880965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:00.882216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.188411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:01.202100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:01.202502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.203056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:01.203636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:01.204217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:01.212587 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-20c71437-9cf3-4c40-b452-768f40df2b68 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.353674 Oct 15 20:52:01.214692 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 648/1230] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:00 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:01.227406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b2bc1f3e-966f-48b9-ac40-1d960c35741c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fc580d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:01.231646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:01.233357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:01.233817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.236520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:01.237097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:01.239112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.281224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:01.288813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:01.305511 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b2bc1f3e-966f-48b9-ac40-1d960c35741c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.085644 Oct 15 20:52:01.308228 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 649/1231] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:01 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:01.309688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:52:01.312082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:52:01.320043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:01.324055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:01.350930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:01.354190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:01.354999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.359678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:01.360386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:01.361022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.389889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-365e5bc8-0846-4d39-b1cc-69e43ccca28c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.884805 Oct 15 20:52:01.390960 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 583/1232] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 894 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:52:01.405706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb113207-04ce-4c8e-bdae-807cb7f63ffe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:01.406846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb113207-04ce-4c8e-bdae-807cb7f63ffe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:01.483376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eb113207-04ce-4c8e-bdae-807cb7f63ffe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.492742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb113207-04ce-4c8e-bdae-807cb7f63ffe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1217 microversion: 2.1 time: 0.093103 Oct 15 20:52:01.492742 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 584/1233] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:01 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1217 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:01.498368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:01.499725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:01.507552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:01.508409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:01.509020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.511128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:01.511802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:01.512370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:01.526048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.538140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.539613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.556871 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9b52131a-4c4d-4bfe-be8b-1582a46d5ba9 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1488 microversion: 2.91 time: 0.246929 Oct 15 20:52:01.561668 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 650/1234] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:01 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1488 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:01.704640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.715696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.717905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:01.719849 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-17ffb62e-4a91-4712-9918-7300e581cb6c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.225829 Oct 15 20:52:01.733745 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 585/1235] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:01 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:02.239435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:02.240712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:02.248157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.249323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.250029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.252163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.252936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.253741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.323470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e554c650-1508-48e8-9a9d-2661f5071b90 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb07a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:02.392057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e554c650-1508-48e8-9a9d-2661f5071b90 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.073498 Oct 15 20:52:02.392926 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 586/1236] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:02 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:02.456477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.466374 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9089a1a7-4e27-48cb-a978-ed452565262c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.231949 Oct 15 20:52:02.467248 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 651/1237] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:02 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:02.516142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:02.516779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:02.527536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.528688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.529320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.530265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.531214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.532193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.587760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:02.588717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:02.599902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.601323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.604136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.604970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.605472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.605915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.637921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.649712 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cefeed47-3575-4719-8cf9-79574affc66c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1213 microversion: 2.1 time: 0.136899 Oct 15 20:52:02.651066 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 587/1238] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:02 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1213 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:02.691331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.700476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.701957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.708956 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fe667a26-6820-40c3-a7e0-f1a0ee1d54ba tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.125149 Oct 15 20:52:02.709551 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 652/1239] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:02 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:02.747942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:02.748557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:02.755118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.755610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.756085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.756740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:02.757224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:02.758133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:02.909131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.910953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bf72d2e5-ab19-40b5-9d76-385995c61242 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_flavor-1792932638", "ram": 512, "vcpus": 1, "disk": 10, "id": 312922969, "swap": 1024, "rxtx_factor": 1, "OS-FLV-EXT-DATA:ephemeral": 10}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:02.918339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.919838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:02.921346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2137a8cd-6f38-48a2-a0bd-4fad20032faa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.177727 Oct 15 20:52:02.922043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 588/1240] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:02 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:02.930729 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bf72d2e5-ab19-40b5-9d76-385995c61242 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 413 microversion: 2.1 time: 0.160576 Oct 15 20:52:02.931373 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 653/1241] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:02 2022] POST /compute/v2.1/flavors => generated 413 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:52:02.946022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a686605-d9ab-4b74-b9ef-3bbb0ec4cf03 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:02.973849 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a686605-d9ab-4b74-b9ef-3bbb0ec4cf03 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.031535 Oct 15 20:52:02.974440 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 589/1242] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Oct 15 20:52:02 2022] POST /compute/v2.1/flavors/312922969/os-extra_specs => generated 66 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.111792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.113560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.114144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.114688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.115535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.116107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.116603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.141519 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe27697a-6e53-487d-9717-07ecce552707 tempest-AZV2TestJSON-1272875166 tempest-AZV2TestJSON-1272875166-project] 10.222.0.16 "GET /compute/v2.1/os-availability-zone" status: 200 len: 97 microversion: 2.1 time: 0.085932 Oct 15 20:52:03.142315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 590/1243] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/os-availability-zone => generated 97 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:52:03.154131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cc08281c-6455-4c56-aeb1-4c8a8c8c9247 tempest-FlavorsExtraSpecsTestJSON-432418575 tempest-FlavorsExtraSpecsTestJSON-432418575-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.164090 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cc08281c-6455-4c56-aeb1-4c8a8c8c9247 tempest-FlavorsExtraSpecsTestJSON-432418575 tempest-FlavorsExtraSpecsTestJSON-432418575-project] 10.222.0.16 "GET /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.181825 Oct 15 20:52:03.164636 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 654/1244] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Oct 15 20:52:02 2022] GET /compute/v2.1/flavors/312922969/os-extra_specs => generated 66 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.176270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b12dee1-f49f-41b0-95b3-ecd496721478 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:03.183314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=5,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:03.203985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b12dee1-f49f-41b0-95b3-ecd496721478 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.031407 Oct 15 20:52:03.204584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 591/1245] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Oct 15 20:52:03 2022] POST /compute/v2.1/flavors/312922969/os-extra_specs => generated 66 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.216112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d03bdf74-f895-44d2-8900-5ecbca1bd28b tempest-FlavorsExtraSpecsTestJSON-432418575 tempest-FlavorsExtraSpecsTestJSON-432418575-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.225525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d03bdf74-f895-44d2-8900-5ecbca1bd28b tempest-FlavorsExtraSpecsTestJSON-432418575 tempest-FlavorsExtraSpecsTestJSON-432418575-project] 10.222.0.16 "GET /compute/v2.1/flavors/312922969/os-extra_specs/hw:numa_nodes" status: 200 len: 22 microversion: 2.1 time: 0.013811 Oct 15 20:52:03.226089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 655/1246] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/flavors/312922969/os-extra_specs/hw:numa_nodes => generated 22 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.236613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4653af6-498b-4249-9482-48cf660e99f8 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:03.274024 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c4653af6-498b-4249-9482-48cf660e99f8 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.040980 Oct 15 20:52:03.276053 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 592/1247] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Oct 15 20:52:03 2022] POST /compute/v2.1/flavors/312922969/os-extra_specs => generated 66 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.293136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ccb20e9-9291-4b6f-b552-5c4752da5060 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.302505 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ccb20e9-9291-4b6f-b552-5c4752da5060 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.019838 Oct 15 20:52:03.303418 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 656/1248] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/flavors/312922969/os-extra_specs => generated 66 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:52:03.315076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1ad47d6-d050-42e7-8ae8-2f4fab7720d3 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Action: 'update', calling method: >, body: {"hw:numa_nodes": "2"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:03.358941 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c1ad47d6-d050-42e7-8ae8-2f4fab7720d3 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/312922969/os-extra_specs/hw:numa_nodes" status: 200 len: 22 microversion: 2.1 time: 0.048726 Oct 15 20:52:03.359558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 593/1249] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:03 2022] PUT /compute/v2.1/flavors/312922969/os-extra_specs/hw:numa_nodes => generated 22 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.368538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-924633b2-c44b-425c-8e47-8b232dbe975b tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.377160 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-924633b2-c44b-425c-8e47-8b232dbe975b tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.012303 Oct 15 20:52:03.378103 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 657/1250] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/flavors/312922969/os-extra_specs => generated 66 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.388465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47275d52-0492-42eb-bb15-fad99d95dd6c tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.409540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4c9b5cf1-d9c2-4e33-91ae-9bd24e059707 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c93a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.424063 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47275d52-0492-42eb-bb15-fad99d95dd6c tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/312922969/os-extra_specs/hw:numa_nodes" status: 200 len: 0 microversion: 2.1 time: 0.040172 Oct 15 20:52:03.425225 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 594/1251] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 20:52:03 2022] DELETE /compute/v2.1/flavors/312922969/os-extra_specs/hw:numa_nodes => generated 0 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:52:03.434940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f08aa3d0-ae84-419e-a39c-dfa62f5e15bd tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.466787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f08aa3d0-ae84-419e-a39c-dfa62f5e15bd tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/312922969/os-extra_specs/hw:cpu_policy" status: 200 len: 0 microversion: 2.1 time: 0.036547 Oct 15 20:52:03.467361 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 595/1252] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 20:52:03 2022] DELETE /compute/v2.1/flavors/312922969/os-extra_specs/hw:cpu_policy => generated 0 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 20:52:03.476518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50c700c7-4e62-4f31-af74-80709a545695 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.483738 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4c9b5cf1-d9c2-4e33-91ae-9bd24e059707 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.079355 Oct 15 20:52:03.484319 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 658/1253] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:03.489030 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50c700c7-4e62-4f31-af74-80709a545695 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/312922969/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.017002 Oct 15 20:52:03.493781 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 596/1254] 10.222.0.16 () {58 vars in 1200 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/flavors/312922969/os-extra_specs => generated 19 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:52:03.494831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.495800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:03.505346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac41a34e-b8e6-4baa-8f00-e1e49653b1d3 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.529290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.530010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.530702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.531980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.532473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.533264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.534912 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac41a34e-b8e6-4baa-8f00-e1e49653b1d3 tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/312922969" status: 202 len: 0 microversion: 2.1 time: 0.033383 Oct 15 20:52:03.536656 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 597/1255] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:52:03 2022] DELETE /compute/v2.1/flavors/312922969 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:52:03.545958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5146caab-a5c6-499a-81b3-6a0d9acd709f tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.568505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5146caab-a5c6-499a-81b3-6a0d9acd709f tempest-FlavorsExtraSpecsTestJSON-2069301313 tempest-FlavorsExtraSpecsTestJSON-2069301313-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.027576 Oct 15 20:52:03.569478 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 598/1256] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:03.676649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.677950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:03.694413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.696408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.698067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.699313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.700077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.700698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.710171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:03.720346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-133b9ce9-3c36-4004-9f0e-570e6f947243 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.232707 Oct 15 20:52:03.721094 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 659/1257] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:03.739937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.741423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:03.754685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.756686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.757502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.758566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.761525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.764785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.863815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:03.873692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:03.875839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:03.881652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:03.882836 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0deb4cf0-0fb0-40b1-b51d-00d84117ed81 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.147382 Oct 15 20:52:03.883716 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 660/1258] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:03.912882 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abc22593-b43e-4785-8bd0-0204c0c9cfd5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1269 microversion: 2.1 time: 0.240696 Oct 15 20:52:03.914333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 599/1259] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1269 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:03.949068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:03.949829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:03.958149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.960107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.961620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:03.964820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:03.966344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:03.967053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:04.209986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:04.236526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:04.238379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:04.239577 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f2f48ba4-a4b2-4bb4-a802-ca2f8d354e2e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.295828 Oct 15 20:52:04.240601 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 661/1260] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:03 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:04.502589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc91bfe9-56df-4c36-8848-d9e383bddf04 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9ad160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:04.592092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc91bfe9-56df-4c36-8848-d9e383bddf04 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.094498 Oct 15 20:52:04.592744 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 600/1261] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:04 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:04.748096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:04.748893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:04.769561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:04.770075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:04.771656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:04.772542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:04.773015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:04.773540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:04.916362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:04.916906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:04.937946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:04.941805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:04.942487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:04.943751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:04.944381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:04.945654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:05.021362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.044936 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98db868d-f53a-42e2-a932-ac48774d4802 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.301514 Oct 15 20:52:05.053179 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 662/1262] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:04 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:05.079761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:05.084199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:05.099172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:05.101577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:05.104463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:05.105434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:05.107012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:05.110392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:05.127170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.139722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.144649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.161481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c617950-54df-437e-99d4-7c377a98c124 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.250517 Oct 15 20:52:05.162329 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 601/1263] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:04 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:05.268311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:05.268926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:05.283176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:05.283770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:05.284219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:05.284949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:05.285475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:05.285905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:05.501518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.517018 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-14d7ea61-bdb8-4372-9bbe-8d5b46ef9668 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1269 microversion: 2.1 time: 0.448324 Oct 15 20:52:05.535521 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 663/1264] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:05 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1269 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:05.612115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0691c77-3e2a-4492-a25a-2c76c9d78455 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb55550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:05.705291 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0691c77-3e2a-4492-a25a-2c76c9d78455 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.098332 Oct 15 20:52:05.706754 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 664/1265] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:05 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:05.711477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.720744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.732092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:05.736295 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efe2fa07-0fce-4c1e-bb24-5d1da8360b42 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.471825 Oct 15 20:52:05.751732 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 602/1266] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:05 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:06.081558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:06.083490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:06.095462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.096726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.097565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.101413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.102133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.102769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.109743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:06.200213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:06.201641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:06.212216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.214975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.218042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.219819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.220503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.221852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.354852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:06.365604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:06.367321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:06.374258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-369c8766-7452-4717-9c61-954b8b2ef843 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.179126 Oct 15 20:52:06.389995 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 603/1267] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:06 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:06.398291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-eb1c4420-060c-4c18-8648-e952db19ebc3 req-4f750d64-8d9b-40b1-8858-fa70c2cd3ef2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "name": "network-vif-plugged", "status": "completed", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:06.414590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb1c4420-060c-4c18-8648-e952db19ebc3 req-4f750d64-8d9b-40b1-8858-fa70c2cd3ef2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.415640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb1c4420-060c-4c18-8648-e952db19ebc3 req-4f750d64-8d9b-40b1-8858-fa70c2cd3ef2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.416428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb1c4420-060c-4c18-8648-e952db19ebc3 req-4f750d64-8d9b-40b1-8858-fa70c2cd3ef2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.436470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-eb1c4420-060c-4c18-8648-e952db19ebc3 req-4f750d64-8d9b-40b1-8858-fa70c2cd3ef2 service nova] Creating event network-vif-plugged:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:52:06.445690 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-eb1c4420-060c-4c18-8648-e952db19ebc3 req-4f750d64-8d9b-40b1-8858-fa70c2cd3ef2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053093 Oct 15 20:52:06.452998 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 604/1268] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:06.495468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:06.507598 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e426952b-2443-4d24-86ec-9a675b64659a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1543 microversion: 2.1 time: 0.429336 Oct 15 20:52:06.508245 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 665/1269] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:06 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1543 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:06.558873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:06.560703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:06.580862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.581884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.582589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.586021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.588286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.588971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.701345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:06.706771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:06.717263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e41e2a50-29ee-41a0-9a5c-cea59aca6155 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1277 microversion: 2.1 time: 0.174171 Oct 15 20:52:06.718923 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 605/1270] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:06 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1277 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:06.723492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ca266a51-cfad-4caf-8220-1fd6ab90338e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c5e50>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:06.786803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:06.787731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:06.800090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.821883 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ca266a51-cfad-4caf-8220-1fd6ab90338e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.103813 Oct 15 20:52:06.823592 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 666/1271] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:06 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:06.824736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.825770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:06.826747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:06.827299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:06.827835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.168671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:07.180330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:07.184102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:07.185581 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-492b4e8a-709c-4076-8a5c-fa0255a411aa tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.404181 Oct 15 20:52:07.186636 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 606/1272] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:06 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:07.404072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:07.405540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:07.424259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:07.424773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:07.425268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.426006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:07.426508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:07.426944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.536408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:07.537693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:07.551841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:07.553495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:07.555639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.560618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:07.561598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:07.564469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.698626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:07.717521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:07.727562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:07.742133 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c8f0e63-16a6-4224-bd60-409d210286f1 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.342712 Oct 15 20:52:07.742739 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 667/1273] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:07 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:07.755190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:07.757890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:07.771285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:07.771808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:07.780015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.782737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:07.783396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:07.784733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:07.994505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:08.003943 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-faf07773-b109-45e8-aab8-4a81be9e950b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1542 microversion: 2.1 time: 0.472028 Oct 15 20:52:08.007756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:08.008261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 607/1274] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:07 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1542 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:08.027529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea07119d-2d83-4978-bba2-b4aeca38ccbe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9a3160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:08.038396 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-181b70f2-bd9c-4816-b08b-29192512ab23 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1277 microversion: 2.1 time: 0.289499 Oct 15 20:52:08.039728 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 668/1275] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:07 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1277 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:08.180753 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea07119d-2d83-4978-bba2-b4aeca38ccbe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.171229 Oct 15 20:52:08.197963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 608/1276] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:08 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:08.214489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:08.215111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:08.222190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:08.222683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:08.223125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:08.223838 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:08.224300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:08.224730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:08.553359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:08.570889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:08.573915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:08.579545 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a80327c2-7535-4277-9578-8e174a1141a4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.370254 Oct 15 20:52:08.580664 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 669/1277] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:08 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:08.776859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:08.777843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:08.785504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:08.786425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:08.787094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:08.788113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:08.788794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:08.789502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:08.963822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.002814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.008036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.035786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:09.036411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:09.042576 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b587555-f41d-449d-8d57-d601f92e3345 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.267796 Oct 15 20:52:09.045153 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 609/1278] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:08 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:09.068353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:09.068353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:09.077627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.078241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.078694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.082626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.083119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.083563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.090207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.090714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.091156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.091874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.092331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.092755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.289545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.305477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80a817bd-3d30-4631-99fa-73b8e2cbe3c1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1277 microversion: 2.1 time: 0.241084 Oct 15 20:52:09.307849 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 610/1279] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:09 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1277 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:09.330074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b554f9a5-18cc-4d72-af21-9dfd73b83c48 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74faa0af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:09.333817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:09.404878 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b554f9a5-18cc-4d72-af21-9dfd73b83c48 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.080615 Oct 15 20:52:09.419285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 611/1280] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:09 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:09.493740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.508389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:09.512554 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2e836417-01de-422d-a327-be18278391ac tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1476 microversion: 2.1 time: 0.480691 Oct 15 20:52:09.529361 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 670/1281] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:09 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1476 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:09.607241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:09.607867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:09.615682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.616740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.617578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.618532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.619998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.620664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.750915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Calling method '.version_select at 0x7fe74f881790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:09.756996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.765820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.778589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.821540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.822715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.826943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.932335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.933380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.934104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.957995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.970911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.972810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:09.977346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:09.978032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:09.979101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:09.981660 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-218774bf-f385-4807-91f3-306c2be34aec tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.379391 Oct 15 20:52:09.982211 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 671/1282] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:09 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:10.054295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.055584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.056234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.064828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c56c771-3697-47ac-81ee-dcf9d1cb7fe6 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail?with_servers=True" status: 200 len: 2943 microversion: 2.53 time: 0.633113 Oct 15 20:52:10.065705 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 612/1283] 10.222.0.16 () {60 vars in 1243 bytes} [Sat Oct 15 20:52:09 2022] GET /compute/v2.1/os-hypervisors/detail?with_servers=True => generated 2943 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:10.079258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:10.080401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:10.085292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Calling method '.version_select at 0x7fe74f8815e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:10.089642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.090179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.090714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.101477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.102145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.102700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.104071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.106861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.107034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.107360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.107603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.107796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.176802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.176802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.176802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.218194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.218870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.219311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.234663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:10.244645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:10.246453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:10.252732 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-470e9bac-e069-4cf2-aa62-ba7e0b4d3268 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.180494 Oct 15 20:52:10.253622 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 672/1284] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:10 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:10.311118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.311611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.312052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.318864 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d709d92b-ddaa-4492-a7f5-641e76b8921f tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors?with_servers=True" status: 200 len: 1634 microversion: 2.53 time: 0.237644 Oct 15 20:52:10.329520 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 613/1285] 10.222.0.16 () {60 vars in 1222 bytes} [Sat Oct 15 20:52:10 2022] GET /compute/v2.1/os-hypervisors?with_servers=True => generated 1634 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:10.334559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Calling method '.version_select at 0x7fe75492a550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:10.334721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:10.336126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:10.339215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.340380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.341008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.346237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.346901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.347924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.348728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.359692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.360414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.369315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=80,nova_api:SAVEPOINT=7,nova_api:RELEASE=7,nova_api:UPDATE=1,nova_api:DELETE=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:10.375111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.375608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.376051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.414579 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffd2a493-3622-4da1-91d2-eb32e4310161 tempest-HypervisorAdminV253TestBase-1743491102 tempest-HypervisorAdminV253TestBase-1743491102-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/c555cd9a-6156-420c-be15-8c9307b54c46" status: 200 len: 791 microversion: 2.53 time: 0.080158 Oct 15 20:52:10.441553 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 614/1286] 10.222.0.16 () {60 vars in 1298 bytes} [Sat Oct 15 20:52:10 2022] GET /compute/v2.1/os-hypervisors/c555cd9a-6156-420c-be15-8c9307b54c46 => generated 791 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:52:10.453755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66162803-1133-4c70-bfb2-ff5b6149774b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f881670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:10.521461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:10.536122 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f8bc3fcb-53f7-49d6-8c5d-7ba712f2932b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1277 microversion: 2.1 time: 0.205358 Oct 15 20:52:10.575462 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66162803-1133-4c70-bfb2-ff5b6149774b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.132531 Oct 15 20:52:10.576043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 615/1287] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:10 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:10.580062 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 673/1288] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:10 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1277 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:10.586498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:10.587414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:10.594676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.595175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.595612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.596327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:10.596846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:10.597302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:10.912054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:10.955621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc347133-fd15-4619-9259-d041b074802c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1474 microversion: 2.1 time: 0.373646 Oct 15 20:52:10.963906 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 616/1289] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:10 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1474 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:11.020756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:11.026136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:11.044161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.045370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.046345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.048513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.049992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.051359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.288438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:11.289542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:11.356287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.356739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.357230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.358106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.358443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.358883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.488177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:11.520430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.534910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.537822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.554010 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c631b8f-eab6-4fa2-a76a-6834004d7818 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.269194 Oct 15 20:52:11.562252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 617/1290] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:11 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:11.567649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:11.568568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:11.582677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.584033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.585057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.587990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.590227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.591471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.603883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.627118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.628810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.630081 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-32e4ffd7-ddad-4673-b248-05a3e9f35ae4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.624139 Oct 15 20:52:11.630634 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 674/1291] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:11 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:11.652034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8af069c7-8179-4f8e-bf41-f5003ba2d4da tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb44a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:11.756412 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8af069c7-8179-4f8e-bf41-f5003ba2d4da tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.111287 Oct 15 20:52:11.757072 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 675/1292] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:11 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:11.875795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:11.889309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "name": "network-vif-unplugged", "status": "completed", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:11.889953 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddb15b43-c288-418f-b264-c5bdad7e9a3f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1277 microversion: 2.1 time: 0.326088 Oct 15 20:52:11.890801 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 618/1293] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:11 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1277 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:11.921148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:11.922017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:11.923433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:11.967928 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] Creating event network-vif-unplugged:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:52:11.987659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:52:11.991274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:11.992491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:12.019797 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e21317db-5d7b-46ad-9ae9-2c4e2a5b61c3 req-a65084ff-bd3b-4115-84a6-19ec623d5a0d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.125211 Oct 15 20:52:12.020511 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 676/1294] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:11 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:52:12.021305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.022029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.022697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.024514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.026519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.027176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.584310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:12.585764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:12.604272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.615593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.616473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.617267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.617741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.618177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.635155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:12.643705 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a600e47-0ffb-4067-be19-9eb8bd06037a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1471 microversion: 2.1 time: 0.657036 Oct 15 20:52:12.648598 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 619/1295] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:11 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1471 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:12.660208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:12.661259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:12.674746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.676164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.682732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.683832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.684544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.685250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.800538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:12.814681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:12.818833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:12.835171 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2a5db8ea-90ad-40d8-8153-49709af4fbd5 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.253830 Oct 15 20:52:12.837567 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 677/1296] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:12 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:12.864868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:12.866326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:12.883926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.884903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.886354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:12.887815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:12.890087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:12.890755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.022060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.026170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.026817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.028221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.032521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.033222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.093894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for project 94e961feee714587bb2ffe545d286c5c. Resources: {'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:13.110517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for user e32a1c53b4be44188b07c768405961ed and project 94e961feee714587bb2ffe545d286c5c. Resources: {'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:13.121460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:13.143599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:13.172315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:13.183576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:13.185538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2143d1b-efc2-49e3-97a7-bd4674ef2636 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.529184 Oct 15 20:52:13.188239 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 620/1297] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:12 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:13.232389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a8c78ff-f182-4270-97fd-a06950903add tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fa98b80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:13.323657 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a8c78ff-f182-4270-97fd-a06950903add tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.115515 Oct 15 20:52:13.325451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 621/1298] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:13 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:13.340060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:13.340903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:13.359433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.360059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.360680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.361551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.363499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.364217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.518751 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-82a18890-fc46-4338-b7f6-ac80b1dfd186 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3/action" status: 202 len: 0 microversion: 2.1 time: 0.678999 Oct 15 20:52:13.522425 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 678/1299] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:52:12 2022] POST /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3/action => generated 0 bytes in 681 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:52:13.531515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:13.532696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:13.544809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.545476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.546236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.547307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.548039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.548750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.563217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:13.580986 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4bc357d-b3eb-440c-a683-ae30bb780a64 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1277 microversion: 2.1 time: 0.247171 Oct 15 20:52:13.599076 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 622/1300] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:13 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1277 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:13.878127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:13.882867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:13.908706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.910269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.913678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:13.914802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:13.921568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:13.922399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:14.416005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:14.425440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:14.428850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:14.460068 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77324126-437d-4577-bf52-fad19f0ed105 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.594414 Oct 15 20:52:14.479640 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 623/1301] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:13 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:14.498383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:14.506822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:14.519204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:14.534102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:14.535000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:14.539055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:14.539876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:14.542498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:14.596111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:15.057594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.081375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.089872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.090190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.091524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9c6e81d0-7f55-4ad3-ba55-62cb121d1c48 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.610059 Oct 15 20:52:15.110289 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 624/1302] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:14 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:15.112129 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7ea81c27-1b43-40cf-ba58-4264e185dfc4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1470 microversion: 2.1 time: 1.584400 Oct 15 20:52:15.114233 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 679/1303] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:13 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1470 bytes in 1591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:15.123465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:15.124387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:15.127000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-691f86f5-2422-499b-9651-6cb2f583f420 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe754590160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:15.147244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:15.148124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:15.148487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:15.151178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:15.152275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:15.153019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:15.216421 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-691f86f5-2422-499b-9651-6cb2f583f420 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.104829 Oct 15 20:52:15.225244 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 625/1304] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:15 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:15.343526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.353252 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-76e7f32b-76c0-4b36-91da-2f572408eabf tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1275 microversion: 2.1 time: 0.237641 Oct 15 20:52:15.357244 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 680/1305] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:15 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1275 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:15.514368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:15.515751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:15.526586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:15.527666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:15.528284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:15.529242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:15.529880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:15.530486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:15.683866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.693990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.695388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:15.715682 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fba8ea96-4938-4426-b532-1b9cc30847af tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.206832 Oct 15 20:52:15.716295 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 626/1306] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:15 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:15.977367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "name": "network-vif-plugged", "status": "completed", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:16.002109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.002626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.003180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.011961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:16.035169 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] Creating event network-vif-plugged:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:52:16.045684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:52:16.053268 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-df129264-53a7-4b80-844a-a112db3a8d8b req-89aa150a-3ee5-4c40-b386-a6b72fc23d9c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079772 Oct 15 20:52:16.054140 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 681/1307] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:16.123348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.124763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:16.132588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.133487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.134198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.135586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.136285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.137318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.140656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.141414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:16.155434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.155938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.156382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.157142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.157614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.158047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.388800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.401433 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-104df262-bf39-4e5a-a3f8-6f5040f92870 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1470 microversion: 2.1 time: 0.263347 Oct 15 20:52:16.403854 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 682/1308] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1470 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:16.409925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de4ee7a6-72ff-45cb-8b07-d55823e49bbf tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb7b4c0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.439550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.455742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.458813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.462459 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5110b1d-5794-4390-a475-84de408331e3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.342684 Oct 15 20:52:16.467680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 627/1309] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:16.480660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.481569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:16.491033 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de4ee7a6-72ff-45cb-8b07-d55823e49bbf tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.086263 Oct 15 20:52:16.492024 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 683/1310] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:16.500818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.500818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.500818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.500818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.501593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.510586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.598685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.607510 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af8132b8-dba0-4675-811b-910653b98414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1275 microversion: 2.1 time: 0.138641 Oct 15 20:52:16.608227 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 628/1311] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1275 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:16.686531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-10921115-780d-4015-ab81-e45d51247772 tempest-NetworksTest-1874001912 tempest-NetworksTest-1874001912-project-admin] Calling method '.version_select at 0x7fe74fb6e280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.745960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.746591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:16.759286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.760345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.760897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.761657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:16.768029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:16.774844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:16.841594 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-10921115-780d-4015-ab81-e45d51247772 tempest-NetworksTest-1874001912 tempest-NetworksTest-1874001912-project-admin] 10.222.0.16 "GET /compute/v2.1/os-networks" status: 200 len: 5359 microversion: 2.1 time: 0.337732 Oct 15 20:52:16.842175 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 684/1312] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/os-networks => generated 5359 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:16.851085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3400043d-e38f-4513-a62d-699ff552c6f4 tempest-NetworksTest-1874001912 tempest-NetworksTest-1874001912-project-admin] Calling method '.version_select at 0x7fe74fa2ddc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:16.870280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.879128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.880900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:16.886749 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37d8cbc4-cae6-45fd-8314-4d6ffe5639d7 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.146594 Oct 15 20:52:16.887804 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 629/1313] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:16.972113 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3400043d-e38f-4513-a62d-699ff552c6f4 tempest-NetworksTest-1874001912 tempest-NetworksTest-1874001912-project-admin] 10.222.0.16 "GET /compute/v2.1/os-networks/dd4fc1bf-26ef-4766-8df5-50a70c32acab" status: 200 len: 652 microversion: 2.1 time: 0.124563 Oct 15 20:52:16.972696 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 685/1314] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/os-networks/dd4fc1bf-26ef-4766-8df5-50a70c32acab => generated 652 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:16.982352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c44c8a96-deac-441f-8f4c-b59db2227730 tempest-NetworksTest-1874001912 tempest-NetworksTest-1874001912-project-admin] Calling method '.version_select at 0x7fe74fcd2040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:17.211959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c44c8a96-deac-441f-8f4c-b59db2227730 tempest-NetworksTest-1874001912 tempest-NetworksTest-1874001912-project-admin] 10.222.0.16 "GET /compute/v2.1/os-networks" status: 200 len: 5359 microversion: 2.1 time: 0.233695 Oct 15 20:52:17.212605 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 630/1315] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:52:16 2022] GET /compute/v2.1/os-networks => generated 5359 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:17.428658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:17.429858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:17.445498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:17.472597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:17.473963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:17.474751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:17.477600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:17.479590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:17.480406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:17.494796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:17.495711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:17.507758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:17.510847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:17.511667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:17.514918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:17.515582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:17.516950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:17.884242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:17.899776 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e55cd454-1265-4496-993c-77ea0bbd423e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1470 microversion: 2.1 time: 0.474480 Oct 15 20:52:17.901120 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 686/1316] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:17 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1470 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:17.933079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-54a558e8-44a0-41bf-add6-9164dc5f7059 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb55040>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:17.936549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:17.945308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:17.971706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:17.973644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab96ddcb-6547-4ad7-a5c9-6dac796764b6 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.484193 Oct 15 20:52:17.977616 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 631/1317] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:17 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:17.989808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:17.997967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:18.020686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:18.022285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:18.023279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:18.024796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:18.025823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:18.026504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:18.034073 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-54a558e8-44a0-41bf-add6-9164dc5f7059 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.129888 Oct 15 20:52:18.034964 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 687/1318] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:17 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:18.040400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:18.040956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:18.049915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:18.050419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:18.050955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:18.051944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:18.052828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:18.053309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:18.139823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:18.168301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b81c58d9-71c3-4a56-9d8e-d915f9239496 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1275 microversion: 2.1 time: 0.186671 Oct 15 20:52:18.172134 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 632/1319] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:17 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1275 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:18.226961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:18.241382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:18.242851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:18.262317 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea231c4e-597b-4a42-9692-8a320712a653 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.222998 Oct 15 20:52:18.273059 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 688/1320] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:18 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:18.927632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:18.928234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:18.940667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:18.941207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:18.941675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:18.942385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:18.952598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:18.953359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.012424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:19.013219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:19.031312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:19.033496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:19.051837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.052997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:19.054581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:19.060234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.354809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.367443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fc77bec0-64f7-4577-b092-bdc85ef6c211 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1536 microversion: 2.1 time: 0.442779 Oct 15 20:52:19.383885 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 633/1321] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:18 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1536 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:19.395457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae8e1f4c-cc13-4c83-98d6-f49c19b028f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fcd2040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:19.486060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae8e1f4c-cc13-4c83-98d6-f49c19b028f8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.100804 Oct 15 20:52:19.494981 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 634/1322] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:19 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:19.494981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:19.495421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:19.512620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:19.513170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:19.513623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.516200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:19.516861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:19.517514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.647565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.663012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.671893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.684547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86173f0a-a5a9-4865-9a19-29b3a3c7cb09 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1275 microversion: 2.1 time: 0.196250 Oct 15 20:52:19.685858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.687049 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 635/1323] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:19 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1275 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:19.698002 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-443981d1-9462-400f-a58d-4a517eec26f2 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 0.685791 Oct 15 20:52:19.709655 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 689/1324] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:19 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:19.718252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:19.719803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:19.736267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:19.737987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:19.739801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.741678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:19.744021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:19.745381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:19.896866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.917604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.919819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:19.934917 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81afde44-ade9-420b-b213-cc86f3d71039 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.246436 Oct 15 20:52:19.935831 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 636/1325] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:19 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:20.396426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:20.397879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:20.418624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:20.419730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:20.428312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:20.429303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:20.430001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:20.430675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:20.503777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1d8c7bb7-c33a-4e30-b02d-3ec186080d0e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe754706550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:20.592470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1d8c7bb7-c33a-4e30-b02d-3ec186080d0e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.093474 Oct 15 20:52:20.596080 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 637/1326] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:20 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:20.716067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:20.716697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:20.732547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:20.739915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:20.741734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:20.743961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:20.746048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:20.746763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:20.933537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:20.945609 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1611fb63-3998-44cc-8161-f39fa5f9b0de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1536 microversion: 2.1 time: 0.550689 Oct 15 20:52:20.957945 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 690/1327] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:20 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1536 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:20.964220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:20.965788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:20.977346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:20.978107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:20.979354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:20.980242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:20.980879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:20.981499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:21.042264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.052289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e46dabf-8a68-4044-9cdd-ec4d8312a61b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.340148 Oct 15 20:52:21.053248 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 638/1328] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:20 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:21.057846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:21.058468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:21.065370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:21.065877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:21.066326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:21.067121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:21.067590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:21.068152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:21.231085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.248376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.257384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.271138 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b47e753c-ec10-4361-8078-233cf0639691 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.216367 Oct 15 20:52:21.271942 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 639/1329] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:21 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:21.494325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestMultiNic-server-1648701138", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "0d76e8ba-5b5e-4cb7-a4d5-39b8684a64a1"}, {"uuid": "f6cd282c-e4df-4f61-9f57-56007129fbf5"}, {"uuid": "0d76e8ba-5b5e-4cb7-a4d5-39b8684a64a1"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:21.700906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] validate_networks() for [('0d76e8ba-5b5e-4cb7-a4d5-39b8684a64a1', None, None, None, None, None), ('f6cd282c-e4df-4f61-9f57-56007129fbf5', None, None, None, None, None), ('0d76e8ba-5b5e-4cb7-a4d5-39b8684a64a1', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:21.932744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.942802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.958952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:21.960483 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3bce3b95-d42e-4d72-a2ee-c66657f7a60a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1567 microversion: 2.25 time: 1.010971 Oct 15 20:52:21.965444 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 691/1330] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:20 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1567 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:21.972972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f5fb5861-53af-4a30-9eb3-e6462db7b303 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb4e0d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:22.081902 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f5fb5861-53af-4a30-9eb3-e6462db7b303 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.115073 Oct 15 20:52:22.083230 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 692/1331] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:21 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:22.092860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:22.098096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:22.111416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:22.114420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:22.115522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:22.116631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:22.120129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:22.121069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:22.627344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:22.652142 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a5c49868-61fb-47f2-b35d-9d900bd34e92 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1536 microversion: 2.1 time: 0.567544 Oct 15 20:52:22.652833 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 693/1332] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:22 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1536 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:22.686870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:22.688496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:22.706074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:22.709064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:22.709945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:22.711671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:22.712816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:22.714947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:23.060824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:23.546274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:23.566400 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0636ed5d-0f2f-4772-908c-364b8cb0c2fe tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.887082 Oct 15 20:52:23.577338 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 694/1333] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:22 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 918 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:23.584997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:23.585661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:23.600768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:23.605593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:23.606179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:23.606947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:23.609316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:23.609775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:23.829878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:23.845305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:23.854823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:23.874424 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-112a8a8c-0f0d-4c5b-b1fa-4c164c3edd55 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1546 microversion: 2.91 time: 0.301946 Oct 15 20:52:23.877417 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 695/1334] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:23 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1546 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:23.901207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:23.902590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:23.914085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:23.929729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:23.932546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:23.936749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:23.937743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:23.938283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:24.797726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:24.815281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:24.817074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:24.823710 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1a6f6840-7da7-4d0e-b56e-f938b515d3b4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1542 microversion: 2.25 time: 0.942242 Oct 15 20:52:24.824311 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 696/1335] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:23 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1542 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:24.854872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d71003a1-dc8b-47ee-9b33-c8314fa114de tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c5700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:25.615820 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d71003a1-dc8b-47ee-9b33-c8314fa114de tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.790422 Oct 15 20:52:25.617573 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 697/1336] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:24 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:25.796589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:25.798313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:25.799034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:25.799771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:25.800392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:25.826765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:25.829708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:25.831680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:25.832676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:25.836010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:25.838109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:25.884361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Getting quotas for project 1c30aa11fd244564be716550bfe5aaa6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:25.895207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Getting quotas for user 8bbc891924f84d99aac22518f69e7b99 and project 1c30aa11fd244564be716550bfe5aaa6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:25.912636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:52:25.915301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:52:25.915843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-880bd8b1-1628-42b1-b0b5-408c9c2362d7 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Calling method '.version_select at 0x7fe7541c5c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.003871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2fdd3805-6957-492c-b5ab-c54c0cac1f95 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.514893 Oct 15 20:52:26.024319 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 640/1337] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 4536 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:52:26.029496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.047632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:26.059895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.061165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.061864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.062839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.063529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.064183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.084454 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-880bd8b1-1628-42b1-b0b5-408c9c2362d7 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 371 microversion: 2.1 time: 0.465551 Oct 15 20:52:26.085101 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 698/1338] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 20:52:25 2022] GET /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 371 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:26.105743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-90e53705-0e43-44a2-8c41-4bbbf5bd6fa6 req-ac7bc173-b732-4b5d-8626-6e2ec0b65a89 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "name": "network-vif-unplugged", "status": "completed", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:26.147854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-90e53705-0e43-44a2-8c41-4bbbf5bd6fa6 req-ac7bc173-b732-4b5d-8626-6e2ec0b65a89 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.148359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-90e53705-0e43-44a2-8c41-4bbbf5bd6fa6 req-ac7bc173-b732-4b5d-8626-6e2ec0b65a89 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.148807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-90e53705-0e43-44a2-8c41-4bbbf5bd6fa6 req-ac7bc173-b732-4b5d-8626-6e2ec0b65a89 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.188064 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-90e53705-0e43-44a2-8c41-4bbbf5bd6fa6 req-ac7bc173-b732-4b5d-8626-6e2ec0b65a89 service nova] Creating event network-vif-unplugged:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:52:26.200007 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-90e53705-0e43-44a2-8c41-4bbbf5bd6fa6 req-ac7bc173-b732-4b5d-8626-6e2ec0b65a89 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.108677 Oct 15 20:52:26.207360 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 699/1339] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:26 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:26.233219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0568b758-3695-4b85-83a6-ab6c2967d86e req-4935c8a9-9de3-4a4b-bea2-972e5664dd4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4d0f01cf-e928-45bc-9c9c-3f34fd9d0945", "tag": "4144980c-d9d6-49c0-b7b0-6e5557bc8c1d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:26.264990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0568b758-3695-4b85-83a6-ab6c2967d86e req-4935c8a9-9de3-4a4b-bea2-972e5664dd4e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.265813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0568b758-3695-4b85-83a6-ab6c2967d86e req-4935c8a9-9de3-4a4b-bea2-972e5664dd4e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.266436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0568b758-3695-4b85-83a6-ab6c2967d86e req-4935c8a9-9de3-4a4b-bea2-972e5664dd4e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.286539 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0568b758-3695-4b85-83a6-ab6c2967d86e req-4935c8a9-9de3-4a4b-bea2-972e5664dd4e service nova] Creating event network-changed:4144980c-d9d6-49c0-b7b0-6e5557bc8c1d for instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 on nch2-master-1 Oct 15 20:52:26.293688 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0568b758-3695-4b85-83a6-ab6c2967d86e req-4935c8a9-9de3-4a4b-bea2-972e5664dd4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083883 Oct 15 20:52:26.294776 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 700/1340] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:26.301149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.302489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:26.310060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.310789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.312035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.313448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.314578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.317089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.412490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:26.421480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-466b52f8-5c36-41ed-883b-41ce67a41189 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1529 microversion: 2.1 time: 0.395872 Oct 15 20:52:26.426044 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 641/1341] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1529 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:26.438535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e92ffd1d-2245-407e-ac34-d8b360fb2bc2 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '.version_select at 0x7fe7542c4940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.447270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e92ffd1d-2245-407e-ac34-d8b360fb2bc2 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.447270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e92ffd1d-2245-407e-ac34-d8b360fb2bc2 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.447270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e92ffd1d-2245-407e-ac34-d8b360fb2bc2 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.478579 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e92ffd1d-2245-407e-ac34-d8b360fb2bc2 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 2961 microversion: 2.25 time: 0.047373 Oct 15 20:52:26.489783 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 642/1342] 10.222.0.16 () {60 vars in 1184 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/os-migrations => generated 2961 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:26.496691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.497373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:26.504412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.504910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.506316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.511155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.512014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.512661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.597088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:26.606362 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f45417d5-885a-4aff-850d-4568672195a7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.310141 Oct 15 20:52:26.606945 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 701/1343] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:26.626028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-193d7c06-99bd-44f2-80f3-060055ec1a90 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.627097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:26.629025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-193d7c06-99bd-44f2-80f3-060055ec1a90 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:26.641353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:26.643717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:26.650679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d55f0a4-9f8a-48bf-81b8-96a57ca2563d tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1533 microversion: 2.91 time: 0.159510 Oct 15 20:52:26.651793 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 643/1344] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1533 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:26.657154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Action: 'update', calling method: .version_select at 0x7fe74fa9cca0>, body: {"quota_set": {"force": true, "cores": 0}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:26.787978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.788722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.789386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.791050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.791971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.793133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.818351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-193d7c06-99bd-44f2-80f3-060055ec1a90 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:26.836424 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-193d7c06-99bd-44f2-80f3-060055ec1a90 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1213 microversion: 2.1 time: 0.216321 Oct 15 20:52:26.852551 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 702/1345] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1213 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:26.864717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.865312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:26.876484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.877656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.879344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.881384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:26.882844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:26.883580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:26.948536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40358dc1-4fe2-47e8-8445-3492a52257a1 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 328 microversion: 2.1 time: 0.294529 Oct 15 20:52:26.949537 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 644/1346] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:26 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 328 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:26.990156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:26.990767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:27.043260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.052221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.060350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.061243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.067925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.068655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.080546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:27.095456 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a1ba6743-9ad4-4534-8a27-74d87e6b732d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1529 microversion: 2.1 time: 0.241839 Oct 15 20:52:27.101111 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 703/1347] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1529 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:27.112716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-68ae3584-a35a-4c39-a98f-a71fc1d382b4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75845a3a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:27.187590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-68ae3584-a35a-4c39-a98f-a71fc1d382b4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.085056 Oct 15 20:52:27.189787 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 704/1348] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:27 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:27.208162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:27.208783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:27.215300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.216094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.216736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.224335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.225201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.226618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.367752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] Lazy-loading 'services' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:52:27.397205 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e3a15115-ceb9-4f7c-b85c-060299c8b173 tempest-UnshelveToHostMultiNodesTest-954584868 tempest-UnshelveToHostMultiNodesTest-954584868-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1938 microversion: 2.91 time: 0.206132 Oct 15 20:52:27.398371 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 705/1349] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:27 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1938 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:27.404206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3619bea5-c851-45d2-9384-e7df46d6b2e2 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:27.410703 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-3619bea5-c851-45d2-9384-e7df46d6b2e2 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] HTTP exception thrown: Flavor d658aa56-cb28-45ff-aa29-056466f85584 could not be found. Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3619bea5-c851-45d2-9384-e7df46d6b2e2 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Request method failure captured: Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/flavors/d658aa56-cb28-45ff-aa29-056466f85584 HTTP/1.1 Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxys_DLdzRwiyR1pvdG_WAwYEVGj_ubYilNBhL1CaDshPvIAK2meAB8Tkr-i1Rp1ZKkoS5yhZeY-YOj6qlFwG0vqIdIkdnxLHHtvRAR5Ksrf9Uu7nJSyM34H7uCu00p8CZcqLc0d-tpz2gvymtoOdLA2xyRo31TCqgBozOFHt5Px0dg Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 08e2c30e5266478893afa293c836a3ed Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-MigrationsAdminTest-425200727 Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:52:27.412888 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/08e2c30e5266478893afa293c836a3ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_08e2c30e5266478893afa293c836a3ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/08e2c30e5266478893afa293c836a3ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/08e2c30e5266478893afa293c836a3ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-MigrationsAdminTest-425200727 Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 08e2c30e5266478893afa293c836a3ed Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-MigrationsAdminTest-425200727 Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-MigrationsAdminTest-425200727-project-admin Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 81fdbd60e924457692039abcb456cb89 Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-MigrationsAdminTest-425200727-project-admin Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: exception: Flavor d658aa56-cb28-45ff-aa29-056466f85584 could not be found. Oct 15 20:52:27.414311 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'd658aa56-cb28-45ff-aa29-056466f85584'} Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_manage.py", line 43, in _delete Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi flavor.destroy() Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 558, in destroy Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_flavor = self._flavor_destroy(self._context, Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 544, in _flavor_destroy Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return _flavor_destroy(context, flavor_id=flavor_id, flavorid=flavorid) Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 185, in _flavor_destroy Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=(flavor_id or flavorid)) Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor d658aa56-cb28-45ff-aa29-056466f85584 could not be found. Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:27.415086 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_manage.py", line 45, in _delete Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor d658aa56-cb28-45ff-aa29-056466f85584 could not be found. Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor d658aa56-cb28-45ff-aa29-056466f85584 could not be found. Oct 15 20:52:27.417729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:27.421685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3619bea5-c851-45d2-9384-e7df46d6b2e2 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Returning 404 to user: Flavor d658aa56-cb28-45ff-aa29-056466f85584 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:52:27.422797 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3619bea5-c851-45d2-9384-e7df46d6b2e2 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/d658aa56-cb28-45ff-aa29-056466f85584" status: 404 len: 109 microversion: 2.1 time: 0.023155 Oct 15 20:52:27.423841 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 706/1350] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:52:27 2022] DELETE /compute/v2.1/flavors/d658aa56-cb28-45ff-aa29-056466f85584 => generated 109 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:52:27.446903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid d16a441b-8807-4eb9-932e-3c2db14bc720 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:52:27.474768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e49e7ca3-9666-4385-9f42-b9f9848794ad tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 2034 microversion: 2.25 time: 0.488367 Oct 15 20:52:27.477914 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 645/1351] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:26 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 2034 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:27.484090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:27.485435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:27.492669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.494596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.495295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.496398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.497147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.498148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.534492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:27.545466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:52:27.626895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1594930034", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:27.731802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:27.797701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5babdd0b-78ab-4929-bb50-a5ec5c7db9b8 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 204 len: 0 microversion: 2.91 time: 0.317327 Oct 15 20:52:27.799406 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 646/1352] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:52:27 2022] DELETE /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 0 bytes in 321 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 15 20:52:27.806255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c352df04-bf68-45a4-9892-092ab5ca2669 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Action: 'create', calling method: .version_select at 0x7fe74fcd2ca0>, body: {"keypair": {"name": "tempest-keypair-1078380761", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPtFj9OgBmAxlLvWw9UuJGn4x88NEC/uKL0ZgntLvgyfg1qP/uRweQrbNYXhGpYyp1okgIsvdv1TfK2qKtiUBSdT6tDcwgGR4dlVxvBU3dnOqPBbrqDCZR3/vMXa+hEZdA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:27.817512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:27.828844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c352df04-bf68-45a4-9892-092ab5ca2669 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for project f400a79cb647438f968a895142c5410d. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:27.836259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c352df04-bf68-45a4-9892-092ab5ca2669 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for user 180d9b7141ff4792ab7ab19e21a3d7ac and project f400a79cb647438f968a895142c5410d. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:27.872774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c352df04-bf68-45a4-9892-092ab5ca2669 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.25 time: 0.071454 Oct 15 20:52:27.875000 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 647/1353] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 20:52:27 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 75 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 20:52:27.907730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:27.910079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:27.934205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.935828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.936355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:27.940162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:27.940746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:27.941282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.086674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:28.087667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:28.088380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:28.089439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:28.090031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:28.103807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.104586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.105247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.106303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.106936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.107543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.132311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Getting quotas for project 43c5ca0165104167a90e69f6f45fdeff. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:28.141423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Getting quotas for user 2d8b0084c7654102b93527cb74465443 and project 43c5ca0165104167a90e69f6f45fdeff. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:28.159670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.utils [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] cores quota exceeded for 43c5ca0165104167a90e69f6f45fdeff, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=138689) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Oct 15 20:52:28.160870 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] HTTP exception thrown: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Request method failure captured: Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 145 Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2F8spFw6S-N1ONzHKAC_PxTz3NEiH4vhcP1l7yYxanY5GIVccN6HtU6eHi8lfLivEGgjI5StcYlDJw1kd0P-SZ98GiWHUa6x7AS1W8px_SjAX5PtH8q48KkScrgELRnfjW7OGDTxMl8rvxxOwSdZBl-wvA3mK37snqH72Xo5mx_j4 Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:52:28.165064 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 2d8b0084c7654102b93527cb74465443 Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1594930034", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} Oct 15 20:52:28.166517 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: exception: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-1594930034', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452'}}} Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Oct 15 20:52:28.167338 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['cores'] Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1362, in _provision_instances Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:28.168938 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.170632 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:28.172193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:28.172193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:28.172193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:52:28.172193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:52:28.172193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Oct 15 20:52:28.172193 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:52:28.173881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Returning 403 to user: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:52:28.175349 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-22592522-3a5e-47aa-8afb-6c165223dea9 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 111 microversion: 2.1 time: 0.750370 Oct 15 20:52:28.176134 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 707/1354] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:27 2022] POST /compute/v2.1/servers => generated 111 bytes in 752 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:52:28.195855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-850814303", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:28.273557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:28.330371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:28.341267 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7dcf148-9469-489b-9465-3f316c3156ed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.463160 Oct 15 20:52:28.360735 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 648/1355] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:27 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:28.366133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:28.370024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:28.378264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.379005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.379627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.380788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.381614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.382222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.482158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:28.505318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:28.507449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:28.514419 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5485d2c4-f70f-435f-a7bf-45a6536d7558 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.152282 Oct 15 20:52:28.515615 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 649/1356] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:28 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:28.524621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:28.525755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:28.534755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.535786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.536400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.537538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.538405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.540768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.642593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:28.662342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33697649-d951-4790-9dee-8a4591fde490 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1265 microversion: 2.1 time: 0.145186 Oct 15 20:52:28.673743 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 650/1357] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:28 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:28.689684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Action: 'update', calling method: .version_select at 0x7fe7546d50d0>, body: {"quota_set": {"cores": 20}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:28.793739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:28.794275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:28.794859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:28.795378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:28.795877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:28.816113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.819039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.825806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.826591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.827453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.828486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.829254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.830585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.831188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:28.831855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.832539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:28.833621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:28.894485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:28.909240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Getting quotas for user e32a1c53b4be44188b07c768405961ed and project 94e961feee714587bb2ffe545d286c5c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:28.940910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-943d12d5-f90c-422c-ae2b-74da364f8763 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 329 microversion: 2.1 time: 0.265719 Oct 15 20:52:28.949829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:52:28.950929 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 651/1358] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:28 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 329 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:28.953196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:52:28.956681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c13cd007-b644-417e-9181-e3bf0f52516a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7541d3820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:29.027371 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c13cd007-b644-417e-9181-e3bf0f52516a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.075070 Oct 15 20:52:29.027953 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 652/1359] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:28 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:29.036532 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e9183bdc-b25c-4390-9aeb-2b71c3d3d1d3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.859014 Oct 15 20:52:29.042362 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 708/1360] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 866 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:52:29.042705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66c2c32b-340c-46ea-83aa-a952350e56b8 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Calling method '.version_select at 0x7fe7546ae700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:29.053542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-975e4e84-2399-4e84-8d40-06eb7f4557aa tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:29.054472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-975e4e84-2399-4e84-8d40-06eb7f4557aa tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:29.183993 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66c2c32b-340c-46ea-83aa-a952350e56b8 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 371 microversion: 2.1 time: 0.154232 Oct 15 20:52:29.185072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 653/1361] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 20:52:29 2022] GET /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 371 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:29.204636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Action: 'update', calling method: .version_select at 0x7fe7546d5430>, body: {"quota_set": {"force": true, "instances": 0}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:29.215113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-975e4e84-2399-4e84-8d40-06eb7f4557aa tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:29.227465 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-975e4e84-2399-4e84-8d40-06eb7f4557aa tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1212 microversion: 2.1 time: 0.177029 Oct 15 20:52:29.228452 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 709/1362] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:29 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1212 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:29.346217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:29.346777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:29.347153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:29.347909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:29.349458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:29.350578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:29.375010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:29.375919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:29.386201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:29.386710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:29.387148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:29.387863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:29.395614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:29.396199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:29.453944 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7add72ca-32ac-4aa5-850d-bf68f911a01d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 328 microversion: 2.1 time: 0.259046 Oct 15 20:52:29.467111 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 654/1363] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:29 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 328 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:29.554267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:29.562705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:29.590886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:29.593493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:29.596563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:29.600781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:29.602027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:29.604270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:29.774404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:29.787787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:29.797468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:29.810359 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9f02ffb-7dec-48a5-9d45-c18b951f9a66 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.265624 Oct 15 20:52:29.830351 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 655/1364] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:29 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:29.836408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-762994008", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:29.917106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:30.117023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:30.132106 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-82871844-798c-42e7-86a9-eb6f32801b7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.761576 Oct 15 20:52:30.132948 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 710/1365] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:29 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:30.162107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:30.163050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:30.177446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.181733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.182380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.183158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.183607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.184038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.230631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:30.340110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:30.364029 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-75ea1bda-1312-4681-ad56-725ebcb9ff5b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1271 microversion: 2.1 time: 0.216620 Oct 15 20:52:30.368234 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 711/1366] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:30 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1271 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:30.377561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b031e074-26fd-4ef6-ac90-c2f26bd18ccb tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f43aaf0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:30.479722 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b031e074-26fd-4ef6-ac90-c2f26bd18ccb tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.105883 Oct 15 20:52:30.480945 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 712/1367] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:30 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:30.491770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5a70df8b-8269-4869-a2b4-092cee3fd2a0 req-0f6f255e-4b7b-4130-86ab-fb601e3b6419 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "name": "network-vif-plugged", "status": "completed", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:30.522256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a70df8b-8269-4869-a2b4-092cee3fd2a0 req-0f6f255e-4b7b-4130-86ab-fb601e3b6419 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.523126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a70df8b-8269-4869-a2b4-092cee3fd2a0 req-0f6f255e-4b7b-4130-86ab-fb601e3b6419 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.523817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a70df8b-8269-4869-a2b4-092cee3fd2a0 req-0f6f255e-4b7b-4130-86ab-fb601e3b6419 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.553461 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5a70df8b-8269-4869-a2b4-092cee3fd2a0 req-0f6f255e-4b7b-4130-86ab-fb601e3b6419 service nova] Creating event network-vif-plugged:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:52:30.560962 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5a70df8b-8269-4869-a2b4-092cee3fd2a0 req-0f6f255e-4b7b-4130-86ab-fb601e3b6419 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078464 Oct 15 20:52:30.562538 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 713/1368] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:30.569156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:30.582048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:30.588830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.590614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.591841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.592876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.595405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:30.595917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.596018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:30.596439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:30.596699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.596858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:30.597374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:30.612516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.612999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.613492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.614247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.614687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.625195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.676649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Getting quotas for project 43c5ca0165104167a90e69f6f45fdeff. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:30.686787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Getting quotas for user 2d8b0084c7654102b93527cb74465443 and project 43c5ca0165104167a90e69f6f45fdeff. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:30.713863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.utils [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] instances quota exceeded for 43c5ca0165104167a90e69f6f45fdeff, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=138688) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Oct 15 20:52:30.715433 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] HTTP exception thrown: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Oct 15 20:52:30.723169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Request method failure captured: Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 144 Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx2F8spFw6S-N1ONzHKAC_PxTz3NEiH4vhcP1l7yYxanY5GIVccN6HtU6eHi8lfLivEGgjI5StcYlDJw1kd0P-SZ98GiWHUa6x7AS1W8px_SjAX5PtH8q48KkScrgELRnfjW7OGDTxMl8rvxxOwSdZBl-wvA3mK37snqH72Xo5mx_j4 Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:52:30.735403 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2d8b0084c7654102b93527cb74465443 Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-762994008", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} Oct 15 20:52:30.737209 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: exception: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-762994008', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452'}}} Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Oct 15 20:52:30.738210 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['instances'] Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1362, in _provision_instances Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:30.739694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.741963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:30.743447 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:30.743447 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:30.743447 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:52:30.743447 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:52:30.743447 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Oct 15 20:52:30.743447 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:30.746268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Returning 403 to user: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:52:30.747260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08c49c6c-0153-48f5-90a7-5551198d83b3 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 119 microversion: 2.1 time: 0.915772 Oct 15 20:52:30.757941 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-89fe8545-62a6-48d9-80b9-3706e90ffdd2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1264 microversion: 2.1 time: 0.192914 Oct 15 20:52:30.759390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 656/1369] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:29 2022] POST /compute/v2.1/servers => generated 119 bytes in 928 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:52:30.762622 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 714/1370] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:30 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1264 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:30.776815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Action: 'update', calling method: .version_select at 0x7fe7544d3dc0>, body: {"quota_set": {"instances": 10}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:30.855087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:30.858153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:30.877357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.878005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.878579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.879401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.879994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.880788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.956435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.956948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.957434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:30.958194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:30.958660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:30.966272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.039331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:31.051043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:31.059523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:31.067326 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4770ef7a-ed22-40db-b77f-a2118c29e007 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1539 microversion: 2.91 time: 0.223298 Oct 15 20:52:31.093483 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 715/1371] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:30 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1539 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:31.160444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:31.161207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:31.172585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:31.174771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:31.175437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.177580 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57bf91e3-2d5c-4bd1-a42e-a21933f593c0 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 329 microversion: 2.1 time: 0.416497 Oct 15 20:52:31.178625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:31.179341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:31.179934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.196328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 657/1372] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:30 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 329 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:31.211469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3c308cb-e945-4e8f-bc6a-ebaa8c380b5d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Calling method '.version_select at 0x7fe74fefc9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:31.371887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3c308cb-e945-4e8f-bc6a-ebaa8c380b5d tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 371 microversion: 2.1 time: 0.164946 Oct 15 20:52:31.372589 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 658/1373] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 20:52:31 2022] GET /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 371 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:31.388597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Action: 'update', calling method: .version_select at 0x7fe75426faf0>, body: {"quota_set": {"force": true, "ram": 0}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:31.517455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:31.522752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:31.523521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.525805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:31.528968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:31.529894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.632487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=5,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:31.640742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-688eaca2-f3fc-4f07-86b3-65ed526b4a73 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 325 microversion: 2.1 time: 0.258324 Oct 15 20:52:31.653553 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 659/1374] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:31 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 325 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:31.653553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:31.653553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:31.679498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:31.680457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:31.685561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.687407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:31.688403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:31.690504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:31.985459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:31.992798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:31.995348 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f67a921-3b63-4b07-8c15-5e5129cce62e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.839191 Oct 15 20:52:32.014312 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 716/1375] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:31 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:32.019716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b4673da0-5ac1-49d4-bb04-4435a671dc47 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f3c6430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:32.021436 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09e1bfad-3e69-462d-bae5-fc535cbf6cdc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1313 microversion: 2.1 time: 0.372540 Oct 15 20:52:32.025548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 660/1376] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:31 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1313 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:32.043854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-683907005", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:32.090741 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b4673da0-5ac1-49d4-bb04-4435a671dc47 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.074723 Oct 15 20:52:32.097899 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 717/1377] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:32 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:32.110529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:32.111383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:32.118166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:32.118661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:32.119101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:32.120352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:32.120818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:32.135476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:32.160744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:32.776392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:32.778144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:32.778918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:32.779580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:32.780250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:32.801798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:32.803808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:32.805724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:32.806807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:32.809855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:32.810649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:32.879030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Getting quotas for project 43c5ca0165104167a90e69f6f45fdeff. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:32.895105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Getting quotas for user 2d8b0084c7654102b93527cb74465443 and project 43c5ca0165104167a90e69f6f45fdeff. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:32.925154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.utils [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] ram quota exceeded for 43c5ca0165104167a90e69f6f45fdeff, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=138688) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Oct 15 20:52:32.927880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] HTTP exception thrown: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Request method failure captured: Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 144 Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx2F8spFw6S-N1ONzHKAC_PxTz3NEiH4vhcP1l7yYxanY5GIVccN6HtU6eHi8lfLivEGgjI5StcYlDJw1kd0P-SZ98GiWHUa6x7AS1W8px_SjAX5PtH8q48KkScrgELRnfjW7OGDTxMl8rvxxOwSdZBl-wvA3mK37snqH72Xo5mx_j4 Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:52:32.932767 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2d8b0084c7654102b93527cb74465443 Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-683907005", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} Oct 15 20:52:32.934543 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: exception: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-683907005', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452'}}} Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Oct 15 20:52:32.935313 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['ram'] Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1362, in _provision_instances Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:32.937500 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.941463 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:32.944520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:32.944520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:32.944520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:52:32.944520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:52:32.944520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Oct 15 20:52:32.944520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:32.946747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Returning 403 to user: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:52:32.949429 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f3b30c9-eb2c-4a83-b370-837780c24478 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 110 microversion: 2.1 time: 0.910026 Oct 15 20:52:32.951014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 661/1378] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:32 2022] POST /compute/v2.1/servers => generated 110 bytes in 925 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:52:32.957138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:32.958266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:32.993168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:32.998095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.001761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.006180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:33.007011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.008921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.103620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:33.109296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.110979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.111888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lazy-loading 'fault' on Instance uuid e7fbd3d0-a708-4f72-9920-c8ec922f75ee {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:52:33.550317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:33.567124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:33.568912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '58842ce40beb4def95f41884b083ec49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab4db46c126d4607943fef56702f5eaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:33.592881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d7146f9-2c30-4821-bbf1-2c3e5764c913 tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 200 len: 1544 microversion: 2.91 time: 0.640377 Oct 15 20:52:33.598122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 662/1379] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:32 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 1544 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:33.617490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Action: 'update', calling method: .version_select at 0x7fe74fc133a0>, body: {"quota_set": {"ram": 51200}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:33.666915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:33.681016 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c053d026-ca85-402a-9dee-026195fcae03 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 1.576892 Oct 15 20:52:33.682836 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 718/1380] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:32 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 1584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:33.726891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:33.729636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:33.747155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:33.747653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.748140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.756098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:33.756612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.758184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.789261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:33.790602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.792520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.793933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:33.800417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:33.803816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:33.967919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3c2cceb-1d32-412a-acf3-49da4da1c135 tempest-QuotasAdminNegativeTest-782165462 tempest-QuotasAdminNegativeTest-782165462-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 200 len: 329 microversion: 2.1 time: 0.357271 Oct 15 20:52:33.977900 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 663/1381] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:33 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 329 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:33.983351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:33.988750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:33.999255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.001087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.001794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.002889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.003697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.004504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.012776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:34.025569 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9023577f-04be-494b-bd0d-2c6ff3c173b7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.329797 Oct 15 20:52:34.026905 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 719/1382] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:33 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:34.031941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-97bd789a-a675-49da-ba48-90bfed00c2c8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f3c6160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:34.118805 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-97bd789a-a675-49da-ba48-90bfed00c2c8 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.090638 Oct 15 20:52:34.119445 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 720/1383] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:34 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:34.145437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c84d0cd2-40c3-4b05-bec8-01e6467e1332 req-65b009c0-a633-455c-b1a0-854969e5ebd5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d0f01cf-e928-45bc-9c9c-3f34fd9d0945", "name": "network-vif-plugged", "status": "completed", "tag": "4144980c-d9d6-49c0-b7b0-6e5557bc8c1d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:34.166637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c84d0cd2-40c3-4b05-bec8-01e6467e1332 req-65b009c0-a633-455c-b1a0-854969e5ebd5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.167190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c84d0cd2-40c3-4b05-bec8-01e6467e1332 req-65b009c0-a633-455c-b1a0-854969e5ebd5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.167763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c84d0cd2-40c3-4b05-bec8-01e6467e1332 req-65b009c0-a633-455c-b1a0-854969e5ebd5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.213103 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c84d0cd2-40c3-4b05-bec8-01e6467e1332 req-65b009c0-a633-455c-b1a0-854969e5ebd5 service nova] Creating event network-vif-plugged:4144980c-d9d6-49c0-b7b0-6e5557bc8c1d for instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 on nch2-master-1 Oct 15 20:52:34.222512 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c84d0cd2-40c3-4b05-bec8-01e6467e1332 req-65b009c0-a633-455c-b1a0-854969e5ebd5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100366 Oct 15 20:52:34.223605 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 721/1384] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:34.231773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-1654666662", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "9c7544ec-ea93-44b4-996f-09fc127b83d3"}], "security_groups": [{"name": "tempest-securitygroup--2037552112"}], "key_name": "tempest-keypair-1078380761", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:34.346122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:34.365350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-656b7ccf-0a2d-4e4f-ad59-f54b66ed4b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1334 microversion: 2.1 time: 0.385867 Oct 15 20:52:34.379179 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 664/1385] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:33 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1334 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:34.389123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ed55d7c-fe49-43d4-ab0d-f17d81888c94 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Action: 'update', calling method: .version_select at 0x7fe74fc135e0>, body: {"quota_set": {"ram": 0}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:34.399233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ed55d7c-fe49-43d4-ab0d-f17d81888c94 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Policy check for os_compute_api:os-quota-sets:update failed with credentials {'is_admin': False, 'user_id': '2d8b0084c7654102b93527cb74465443', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '43c5ca0165104167a90e69f6f45fdeff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ed55d7c-fe49-43d4-ab0d-f17d81888c94 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Request method failure captured: Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff HTTP/1.1 Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 25 Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx2F8spFw6S-N1ONzHKAC_PxTz3NEiH4vhcP1l7yYxanY5GIVccN6HtU6eHi8lfLivEGgjI5StcYlDJw1kd0P-SZ98GiWHUa6x7AS1W8px_SjAX5PtH8q48KkScrgELRnfjW7OGDTxMl8rvxxOwSdZBl-wvA3mK37snqH72Xo5mx_j4 Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:52:34.404705 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/43c5ca0165104167a90e69f6f45fdeff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 43c5ca0165104167a90e69f6f45fdeff Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-938872829 Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2d8b0084c7654102b93527cb74465443 Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-QuotasAdminNegativeTest-938872829-project Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: {"quota_set": {"ram": 0}} Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Oct 15 20:52:34.406164 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '43c5ca0165104167a90e69f6f45fdeff', 'body': {'quota_set': {'ram': 0}}} Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/quota_sets.py", line 170, in update Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._update(req, id, body, []) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:52:34.407099 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/quota_sets.py", line 189, in _update Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(qs_policies.POLICY_ROOT % 'update', {'project_id': id}) Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.408273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:34.413260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:34.413260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:34.413260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:52:34.413260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:52:34.413260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Oct 15 20:52:34.413260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.415483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ed55d7c-fe49-43d4-ab0d-f17d81888c94 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:52:34.417282 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ed55d7c-fe49-43d4-ab0d-f17d81888c94 tempest-QuotasAdminNegativeTest-938872829 tempest-QuotasAdminNegativeTest-938872829-project] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff" status: 403 len: 116 microversion: 2.1 time: 0.036041 Oct 15 20:52:34.422725 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 665/1386] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:52:34 2022] PUT /compute/v2.1/os-quota-sets/43c5ca0165104167a90e69f6f45fdeff => generated 116 bytes in 42 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:52:34.540252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] validate_networks() for [('9c7544ec-ea93-44b4-996f-09fc127b83d3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:34.585423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-578a3fe7-a890-433f-b569-ec07bcc30888 req-2f63194e-a35b-4704-a92b-b51eed385545 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:34.606958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-578a3fe7-a890-433f-b569-ec07bcc30888 req-2f63194e-a35b-4704-a92b-b51eed385545 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.607458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-578a3fe7-a890-433f-b569-ec07bcc30888 req-2f63194e-a35b-4704-a92b-b51eed385545 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.608361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-578a3fe7-a890-433f-b569-ec07bcc30888 req-2f63194e-a35b-4704-a92b-b51eed385545 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.638731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-578a3fe7-a890-433f-b569-ec07bcc30888 req-2f63194e-a35b-4704-a92b-b51eed385545 service nova] Creating event network-changed:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch1-master-1 Oct 15 20:52:34.647490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-578a3fe7-a890-433f-b569-ec07bcc30888 req-2f63194e-a35b-4704-a92b-b51eed385545 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066417 Oct 15 20:52:34.652683 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 666/1387] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:34.664825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:34.667956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] [instance: e7fbd3d0-a708-4f72-9920-c8ec922f75ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:34.679628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.685985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.687140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.689161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.690467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.692367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.742024 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] HTTP exception thrown: Instance e7fbd3d0-a708-4f72-9920-c8ec922f75ee could not be found. Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Request method failure captured: Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee HTTP/1.1 Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzXZE1-rQYtVRjchO2AuL0Q_zW1NpMjBo4mG707FkJGD-v8L2lDBdLwkUiaeXh_t4f5tTBtMZzPJU5etIBldF7T4T2PeROWqGC3uHSSiMjR7x4TPD09i8ViA3IlIZFFQ7It3BhLJgbG9hKzGtodPKpxnt2B8JDAw-zyTfy-mbLIMzM Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.91 Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ab4db46c126d4607943fef56702f5eaf Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-UnshelveToHostMultiNodesTest-451820246 Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:52:34.743484 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ab4db46c126d4607943fef56702f5eaf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ab4db46c126d4607943fef56702f5eaf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ab4db46c126d4607943fef56702f5eaf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ab4db46c126d4607943fef56702f5eaf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-UnshelveToHostMultiNodesTest-451820246 Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ab4db46c126d4607943fef56702f5eaf Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-UnshelveToHostMultiNodesTest-451820246 Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-UnshelveToHostMultiNodesTest-451820246-project Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 58842ce40beb4def95f41884b083ec49 Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-UnshelveToHostMultiNodesTest-451820246-project Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e7fbd3d0-a708-4f72-9920-c8ec922f75ee could not be found. Oct 15 20:52:34.745194 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e7fbd3d0-a708-4f72-9920-c8ec922f75ee'} Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e7fbd3d0-a708-4f72-9920-c8ec922f75ee could not be found. Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:52:34.746078 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e7fbd3d0-a708-4f72-9920-c8ec922f75ee could not be found. Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e7fbd3d0-a708-4f72-9920-c8ec922f75ee could not be found. Oct 15 20:52:34.747372 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:52:34.758218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] Returning 404 to user: Instance e7fbd3d0-a708-4f72-9920-c8ec922f75ee could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:52:34.763320 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9c287945-2523-4121-bc0d-c9dc6568262a tempest-UnshelveToHostMultiNodesTest-451820246 tempest-UnshelveToHostMultiNodesTest-451820246-project] 10.222.0.16 "GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee" status: 404 len: 111 microversion: 2.91 time: 0.108931 Oct 15 20:52:34.766590 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 667/1388] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:34 2022] GET /compute/v2.1/servers/e7fbd3d0-a708-4f72-9920-c8ec922f75ee => generated 111 bytes in 113 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:52:34.781692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:34.785988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:34.795664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.796316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.811464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:34.812328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:34.812820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:34.813534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.202517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:35.220916 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4761370d-df44-468c-b992-26f80acec1ba tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 0.449647 Oct 15 20:52:35.223084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 668/1389] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:34 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:35.246161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:35.247092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:35.275086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:35.279744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:35.283973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.285218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:35.286621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:35.289066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.508751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:35.509322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:35.509777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:35.510203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:35.511615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:35.520968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:35.539394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:35.540513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:35.542256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.546621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:35.564921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:35.567155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.578003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:35.587893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-166b1f7b-564d-4f3e-ba5f-64f40867a1c6 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1317 microversion: 2.1 time: 0.354418 Oct 15 20:52:35.608229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for project f400a79cb647438f968a895142c5410d. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:35.612881 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 669/1390] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:35 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1317 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:35.614572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Getting quotas for user 180d9b7141ff4792ab7ab19e21a3d7ac and project f400a79cb647438f968a895142c5410d. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:35.625611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13fac6e1-22cc-4a9e-b36a-d03ce9afc4a7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fc17040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:35.630325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:52:35.632968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:52:35.707697 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13fac6e1-22cc-4a9e-b36a-d03ce9afc4a7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.092744 Oct 15 20:52:35.708300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 670/1391] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:35 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:35.728317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:35.728949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:35.734580 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-13d69e75-554e-4919-a899-23907c4c9643 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.25 time: 1.508183 Oct 15 20:52:35.736813 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 722/1392] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:52:34 2022] POST /compute/v2.1/servers => generated 406 bytes in 1512 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:52:35.747731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:35.759305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:35.759965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.760886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7e0a21c7-12d7-4831-bf19-57bd585aa470 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:35.762877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:35.763357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:35.763971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:35.766134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7e0a21c7-12d7-4831-bf19-57bd585aa470 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:35.916833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e0a21c7-12d7-4831-bf19-57bd585aa470 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:35.926375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e0a21c7-12d7-4831-bf19-57bd585aa470 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:35.931178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e0a21c7-12d7-4831-bf19-57bd585aa470 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:35.934193 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7e0a21c7-12d7-4831-bf19-57bd585aa470 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1286 microversion: 2.25 time: 0.189642 Oct 15 20:52:35.949884 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 723/1393] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:35 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1286 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:36.150572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:36.166181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-462088f1-2019-4e97-bdd7-d22943802e1b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.456706 Oct 15 20:52:36.169395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 671/1394] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:35 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:36.250767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:36.251438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:36.259830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:36.265121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:36.265121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:36.265707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:36.266392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:36.269601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:36.634518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:36.635157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:36.654992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:36.655518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:36.657690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:36.660659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:36.663945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:36.664547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:36.808636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:36.825626 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-848da349-d343-4ff0-b1c2-ca921584357f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 0.569730 Oct 15 20:52:36.828124 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 724/1395] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:36 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:36.833967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3601e46c-2170-400f-9d08-84a728ae28ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f43ad30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:36.925232 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3601e46c-2170-400f-9d08-84a728ae28ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.095686 Oct 15 20:52:36.929399 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 725/1396] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:36 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:36.964602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:36.965678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:36.982537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:36.983296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:36.983924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:36.984867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:36.987913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:36.988781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.155316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:37.165010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:37.170864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:37.178563 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-430ac44c-faf4-439e-b2bf-0d383d39826e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1282 microversion: 2.25 time: 0.218614 Oct 15 20:52:37.179733 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 726/1397] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:36 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1282 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:37.195719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:37.196572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:37.204977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.209155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.209809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.210719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.221796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.222535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.429998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:37.444098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4cdf0840-8f9d-4c7a-8455-b4ae0e9ac971 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1466 microversion: 2.1 time: 0.829909 Oct 15 20:52:37.447435 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 672/1398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:36 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1466 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:37.461355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-952b4906-176f-44d8-b558-e6859cee2ef9 req-941ff363-60d8-4aca-8386-8320574f679c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "tag": "94e6278a-0fe8-4e57-8500-38f6a69ef5b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:37.497734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-952b4906-176f-44d8-b558-e6859cee2ef9 req-941ff363-60d8-4aca-8386-8320574f679c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.498435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-952b4906-176f-44d8-b558-e6859cee2ef9 req-941ff363-60d8-4aca-8386-8320574f679c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.501434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-952b4906-176f-44d8-b558-e6859cee2ef9 req-941ff363-60d8-4aca-8386-8320574f679c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.546850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:37.560394 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-952b4906-176f-44d8-b558-e6859cee2ef9 req-941ff363-60d8-4aca-8386-8320574f679c service nova] Creating event network-changed:94e6278a-0fe8-4e57-8500-38f6a69ef5b8 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:52:37.575407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-952b4906-176f-44d8-b558-e6859cee2ef9 req-941ff363-60d8-4aca-8386-8320574f679c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.124355 Oct 15 20:52:37.580206 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 673/1399] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:37.605820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:37.613029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:37.628081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.629893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.630916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.631865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.632718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.633383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.664064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:37.677600 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-73522180-bc6e-4f11-a1c8-a3d0b376bd35 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.483059 Oct 15 20:52:37.678756 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 727/1400] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:37 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:37.853850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:37.854884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:37.866300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27b0ea29-00da-495d-b146-cd7eea28f243 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1617 microversion: 2.1 time: 0.284271 Oct 15 20:52:37.867347 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 674/1401] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:37 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1617 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:37.878741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.891431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.892174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.893192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:37.893853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:37.894504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:37.944764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1eb68c56-69fe-4ac0-8ff6-f00723586944 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fcd2550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:38.035075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1eb68c56-69fe-4ac0-8ff6-f00723586944 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.095841 Oct 15 20:52:38.038724 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 675/1402] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:37 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:38.126154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-280114913", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}, "os:scheduler_hints": {"different_host": ["0028073f-e418-41fb-830e-0c49db2dee33"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:38.235859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:38.332241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:38.347713 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7682dc3d-8b9a-4b5a-b046-36fd745ecc7d tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 0.497962 Oct 15 20:52:38.349656 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 728/1403] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:37 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:38.369473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:38.370365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:38.382732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:38.386943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:38.387503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:38.389185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:38.389768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:38.391911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:38.544382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:38.579849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:38.587150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:38.590311 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b1ad85d5-7d57-4d20-be5b-c64dc42c5385 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1356 microversion: 2.25 time: 0.237872 Oct 15 20:52:38.591388 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 729/1404] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:38 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1356 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:38.706737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:38.707526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:38.718959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:38.725936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:38.726694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:38.728094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:38.729236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:38.731866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.250246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:39.264631 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ed311e17-f5c4-4d12-93e8-339063f228cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.566878 Oct 15 20:52:39.269561 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 730/1405] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:38 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:39.304636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35aa53e4-93ad-4a1b-bfa1-ecb982c6fd6e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb449d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:39.386394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:39.386925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:39.387329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:39.387998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:39.388606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:39.399906 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35aa53e4-93ad-4a1b-bfa1-ecb982c6fd6e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.127979 Oct 15 20:52:39.400937 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 731/1406] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:39 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:39.410821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:39.411721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:39.412362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.413217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:39.413750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:39.415493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:39.415493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.415671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:39.433750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:39.437212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:39.438063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.442057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:39.442799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:39.444765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.464813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:39.488465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:39.536230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:52:39.538887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:52:39.684022 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdae34c1-e643-4dc3-9944-c54d1fa77414 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.564077 Oct 15 20:52:39.693455 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 676/1407] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 1574 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:52:39.722735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:39.723777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:39.737716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:39.739365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:39.740109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.741333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:39.742050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:39.742769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:39.981251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:40.001241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:40.007946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:40.016162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ced4c411-b12c-4acc-90a9-3c4eceaa91d8 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1344 microversion: 2.25 time: 0.314754 Oct 15 20:52:40.036641 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 677/1408] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:39 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1344 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:40.059983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b268ac1-a422-47f4-a2de-78cf862ce62b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:40.060870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b268ac1-a422-47f4-a2de-78cf862ce62b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:40.252065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9b268ac1-a422-47f4-a2de-78cf862ce62b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:40.263975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b268ac1-a422-47f4-a2de-78cf862ce62b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1216 microversion: 2.1 time: 0.225884 Oct 15 20:52:40.268483 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 678/1409] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:40 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1216 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:40.295358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:40.296104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:40.313309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:40.313803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:40.314240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:40.325205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:40.326130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:40.326603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:40.472923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:40.487227 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-19d03fb9-2119-49e1-9c38-ff6c02c73af9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 1.083959 Oct 15 20:52:40.489283 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 732/1410] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:39 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 1088 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:40.502942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ceb1a5e7-4579-4de8-aac9-724df993f7dc req-56122082-1eb9-4e12-aa21-364a5e2ee6c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "tag": "de10a24b-8347-41f7-9b6b-e7ae78428202"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:40.534045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ceb1a5e7-4579-4de8-aac9-724df993f7dc req-56122082-1eb9-4e12-aa21-364a5e2ee6c0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:40.534985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ceb1a5e7-4579-4de8-aac9-724df993f7dc req-56122082-1eb9-4e12-aa21-364a5e2ee6c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:40.536831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ceb1a5e7-4579-4de8-aac9-724df993f7dc req-56122082-1eb9-4e12-aa21-364a5e2ee6c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:40.591848 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ceb1a5e7-4579-4de8-aac9-724df993f7dc req-56122082-1eb9-4e12-aa21-364a5e2ee6c0 service nova] Creating event network-changed:de10a24b-8347-41f7-9b6b-e7ae78428202 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:52:40.602513 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ceb1a5e7-4579-4de8-aac9-724df993f7dc req-56122082-1eb9-4e12-aa21-364a5e2ee6c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.108690 Oct 15 20:52:40.603625 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 733/1411] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:40.617776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8d471ea2-fb47-4c2a-b426-a026b2c98650 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fb7bb80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:40.709277 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8d471ea2-fb47-4c2a-b426-a026b2c98650 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.103076 Oct 15 20:52:40.710078 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 734/1412] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:40 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:41.052555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:41.053735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:41.089503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:41.093826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:41.094287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:41.100145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:41.102738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:41.103243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:41.428763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:41.454517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a459779-f524-4151-94bd-6eb51059ff0f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 1.165638 Oct 15 20:52:41.456877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 679/1413] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:40 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 1169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:41.467664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:41.468800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:41.498507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:41.499009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:41.499451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:41.500172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:41.500636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:41.514479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:41.545884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:41.566553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:41.568684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:41.574870 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bad8914f-f5ec-46b6-8d08-fb1872e81918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1344 microversion: 2.25 time: 0.526980 Oct 15 20:52:41.576059 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 735/1414] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:41 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1344 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:52:41.598411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:41.599324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:41.621755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:41.622266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:41.622811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:41.629149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:41.635445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:41.636445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:41.762630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:41.803605 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a00afdd8-7ad7-42e4-a94b-ee9335e572df tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1268 microversion: 2.1 time: 0.344421 Oct 15 20:52:41.804559 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 680/1415] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:41 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1268 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:41.810418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d2c2aceb-a4f0-4b82-88c8-d166813f41f3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9eb790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:41.914337 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d2c2aceb-a4f0-4b82-88c8-d166813f41f3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1453 microversion: 2.1 time: 0.108127 Oct 15 20:52:41.915837 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 681/1416] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:41 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1453 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:42.075230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:42.086981 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5eb0e825-002b-4f44-afa9-89a3e0afc8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 0.509628 Oct 15 20:52:42.088348 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 736/1417] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:41 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:42.305281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e7e0abdc-55b8-4ecc-97e5-66aa52c5e7b9 req-0fcfd924-6547-4afc-9954-7585fbd327ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "name": "network-vif-plugged", "status": "completed", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:42.338484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e7e0abdc-55b8-4ecc-97e5-66aa52c5e7b9 req-0fcfd924-6547-4afc-9954-7585fbd327ec service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.339580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e7e0abdc-55b8-4ecc-97e5-66aa52c5e7b9 req-0fcfd924-6547-4afc-9954-7585fbd327ec service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.340016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e7e0abdc-55b8-4ecc-97e5-66aa52c5e7b9 req-0fcfd924-6547-4afc-9954-7585fbd327ec service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:42.390408 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e7e0abdc-55b8-4ecc-97e5-66aa52c5e7b9 req-0fcfd924-6547-4afc-9954-7585fbd327ec service nova] Creating event network-vif-plugged:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch1-master-1 Oct 15 20:52:42.399986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e7e0abdc-55b8-4ecc-97e5-66aa52c5e7b9 req-0fcfd924-6547-4afc-9954-7585fbd327ec service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100421 Oct 15 20:52:42.404898 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 737/1418] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:52:42.486432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:42.487431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:42.494474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:42.498658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.499172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.499607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:42.500359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.500824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.501319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.504584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.505062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:42.505787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.506255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.506679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:42.520510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:42.574905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:42.607739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c61bff79-267d-4242-b532-c211211757c4 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.347123 Oct 15 20:52:42.608351 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 682/1419] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:52:42 2022] GET /compute/v2.1/os-services => generated 947 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:42.632264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:42.633988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:42.644042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.644784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.645452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:42.646358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:42.646979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:42.647576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.036223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.046361 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a2283bc2-0d0d-4630-a0b3-5fb4b58c0bd8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.565432 Oct 15 20:52:43.047694 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 738/1420] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:42 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:43.055593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.060090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.060898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.061727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.062688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.063868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.064485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.093066 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb39ea83-c053-45a8-bff9-e17f13f92d0a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=ncdv-master-1" status: 200 len: 581 microversion: 2.1 time: 0.044005 Oct 15 20:52:43.094115 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 739/1421] 10.222.0.16 () {58 vars in 1174 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/os-services?host=ncdv-master-1 => generated 581 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:43.098952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.099646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:43.108893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.109833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.110519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.111537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.112368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.113751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.162032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.171969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.173708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.174961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c4959d1f-fe59-4475-8b88-a0501d33e348 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.550574 Oct 15 20:52:43.180816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.193824 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 683/1422] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:42 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:43.196241 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d8de6b54-06ba-4c2c-9a72-2ee5af7687ad tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1274 microversion: 2.1 time: 0.100986 Oct 15 20:52:43.196890 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 740/1423] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1274 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:43.201956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c852c55-c254-4b7e-b7ae-1f574b8fb3b5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fcd2ca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.203947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.212620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.213020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.214374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.215847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.216630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.217310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.243762 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9cc0283e-8938-4b60-86b4-6cb6dff6c88a tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.045621 Oct 15 20:52:43.244667 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 741/1424] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:43.250287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.252693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:43.260357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.262048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.262643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.263531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.264146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.264739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.272307 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c852c55-c254-4b7e-b7ae-1f574b8fb3b5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1462 microversion: 2.1 time: 0.074347 Oct 15 20:52:43.274175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 684/1425] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1462 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:43.283773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.288295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.289602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.290350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.291460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.292220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.292929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.324459 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-290c8379-95bb-4465-a118-b0bebb04a693 tempest-ServicesAdminTestJSON-34343036 tempest-ServicesAdminTestJSON-34343036-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.048878 Oct 15 20:52:43.325421 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 685/1426] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/os-services => generated 947 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:43.330662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd77dffe-9a3d-4635-b158-e0931a4f2afa tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f865e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.399051 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd77dffe-9a3d-4635-b158-e0931a4f2afa tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 200 len: 1462 microversion: 2.1 time: 0.072540 Oct 15 20:52:43.400816 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 686/1427] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 1462 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:43.415251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:43.417471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:43.426266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.427103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.427786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.429554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.430740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.431336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.553295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.565275 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d00c62c-1a6d-4e32-8486-6e628c19d8e1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1312 microversion: 2.1 time: 0.318499 Oct 15 20:52:43.566552 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 742/1428] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1312 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:43.571464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-7461186c-c451-4802-b3cd-c9e2ddf1bbfb req-8a7e82bf-ba37-4551-af56-2176a6862245 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "tag": "4ed7714f-38f6-40a9-96ad-267754f29c1d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:43.582486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7461186c-c451-4802-b3cd-c9e2ddf1bbfb req-8a7e82bf-ba37-4551-af56-2176a6862245 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:43.583762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7461186c-c451-4802-b3cd-c9e2ddf1bbfb req-8a7e82bf-ba37-4551-af56-2176a6862245 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:43.584474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7461186c-c451-4802-b3cd-c9e2ddf1bbfb req-8a7e82bf-ba37-4551-af56-2176a6862245 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:43.600995 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-7461186c-c451-4802-b3cd-c9e2ddf1bbfb req-8a7e82bf-ba37-4551-af56-2176a6862245 service nova] Creating event network-changed:4ed7714f-38f6-40a9-96ad-267754f29c1d for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:52:43.606646 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-7461186c-c451-4802-b3cd-c9e2ddf1bbfb req-8a7e82bf-ba37-4551-af56-2176a6862245 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038537 Oct 15 20:52:43.607400 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 743/1429] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:43.876546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:43.884531 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4aa5817b-5b06-46fc-a733-0a5495cd0740 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1480 microversion: 2.1 time: 0.473225 Oct 15 20:52:43.890048 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 687/1430] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:43 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1480 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:44.075200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:44.076229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:44.083235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.084036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.084562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.085909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.086535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.087323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.205447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:44.206986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:44.235357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.235957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.237663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.238686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.239381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.240045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.537005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:44.549647 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c180f065-78b4-4cd1-8778-3a5f1157569e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.482017 Oct 15 20:52:44.550249 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 744/1431] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:44 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:44.554968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:44.555591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:44.575046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.575929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.576573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.577550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.578210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.578834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.812387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:44.826872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:44.830456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:44.832604 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1d643b8-4700-4b30-8e29-238f425a0deb tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.631133 Oct 15 20:52:44.835186 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 688/1432] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:44 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:44.843951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:44.847060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:44.855308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.856518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.857334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.859649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.874844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.875846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:44.935788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:44.946739 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e835476c-f94e-488f-8a9c-77319a297e44 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.395086 Oct 15 20:52:44.946982 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 745/1433] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:44 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:44.958247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-38566750-21c9-48b3-aada-1918f3d0cba4 req-957dcfcc-d22a-451c-bd02-e64dbf0a7ee3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c05e4f55-245d-45f2-8567-e665ccbca061", "tag": "0e64f25a-9a1d-4631-8907-36be7c15d6f3"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:44.988153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-38566750-21c9-48b3-aada-1918f3d0cba4 req-957dcfcc-d22a-451c-bd02-e64dbf0a7ee3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:44.989594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-38566750-21c9-48b3-aada-1918f3d0cba4 req-957dcfcc-d22a-451c-bd02-e64dbf0a7ee3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:44.990532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-38566750-21c9-48b3-aada-1918f3d0cba4 req-957dcfcc-d22a-451c-bd02-e64dbf0a7ee3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.011499 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-38566750-21c9-48b3-aada-1918f3d0cba4 req-957dcfcc-d22a-451c-bd02-e64dbf0a7ee3 service nova] Creating event network-changed:0e64f25a-9a1d-4631-8907-36be7c15d6f3 for instance c05e4f55-245d-45f2-8567-e665ccbca061 on nch1-master-1 Oct 15 20:52:45.021199 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-38566750-21c9-48b3-aada-1918f3d0cba4 req-957dcfcc-d22a-451c-bd02-e64dbf0a7ee3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066408 Oct 15 20:52:45.022020 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 746/1434] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:45.042211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:45.043167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:45.050098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:45.050855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:45.051493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.052978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:45.053642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:45.054274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.322445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:45.331644 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b3f5a5f4-33a0-452a-a91d-2b770a611184 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1467 microversion: 2.1 time: 0.308587 Oct 15 20:52:45.344656 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 747/1435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:45 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1467 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:45.359502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-1672421181"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:45.362351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:45.369669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:45.370666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:45.371330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.372247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:45.372919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:45.373614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.652952 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7a60c141-6f53-480d-a634-b52a292e9d46 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "POST /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f/action" status: 202 len: 0 microversion: 2.1 time: 0.298380 Oct 15 20:52:45.653658 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 748/1436] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:52:45 2022] POST /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f/action => generated 0 bytes in 300 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Oct 15 20:52:45.681739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:45.685398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:45.697358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:45.698948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:45.699659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.700634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:45.702376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:45.703196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:45.924309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=26 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:46.017385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:46.026916 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ddd3f0b7-433c-4f66-8076-c31dc759c81b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.369412 Oct 15 20:52:46.030615 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 749/1437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:45 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:46.034449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:46.038088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-61354209-9ce1-4e97-9863-c976992de727 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f43ac10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:46.069060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-652f3954-d1b2-43b8-a0ad-47adeb3dce88 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1460 microversion: 2.1 time: 1.232214 Oct 15 20:52:46.070715 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 689/1438] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:44 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1460 bytes in 1234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:46.106549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:46.109928 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-61354209-9ce1-4e97-9863-c976992de727 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.077665 Oct 15 20:52:46.112152 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 750/1439] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:46 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:46.114687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:46.126546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:46.127177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:46.129739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.130285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.131865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.132835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.133531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.134206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.134346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.134820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.135269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.142779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.143590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.144252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.371087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:46.380199 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1de932d1-2121-4d51-83f2-12608bacba4e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.266974 Oct 15 20:52:46.383327 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 751/1440] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:46 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:46.389413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-df2f6215-f853-460e-9c26-df55c59d7479 req-aeeed2c0-69ca-4063-ac10-db082cf64219 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ed61048c-5a28-4c6d-becc-59c01ab7089d", "tag": "c1c0e9d4-bc1c-433c-bbab-d38dd8f326f7"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:46.411403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-df2f6215-f853-460e-9c26-df55c59d7479 req-aeeed2c0-69ca-4063-ac10-db082cf64219 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.412211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-df2f6215-f853-460e-9c26-df55c59d7479 req-aeeed2c0-69ca-4063-ac10-db082cf64219 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.412882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-df2f6215-f853-460e-9c26-df55c59d7479 req-aeeed2c0-69ca-4063-ac10-db082cf64219 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.436191 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-df2f6215-f853-460e-9c26-df55c59d7479 req-aeeed2c0-69ca-4063-ac10-db082cf64219 service nova] Creating event network-changed:c1c0e9d4-bc1c-433c-bbab-d38dd8f326f7 for instance ed61048c-5a28-4c6d-becc-59c01ab7089d on nch2-master-1 Oct 15 20:52:46.461127 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-df2f6215-f853-460e-9c26-df55c59d7479 req-aeeed2c0-69ca-4063-ac10-db082cf64219 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075380 Oct 15 20:52:46.462023 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 752/1441] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:46.467628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:46.468718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:46.476306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.477236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.477986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.478958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.480632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.481353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.784819 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2b71cfba-d633-4f81-af90-dc48066a9f7d tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1611 microversion: 2.1 time: 0.320964 Oct 15 20:52:46.809298 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 753/1442] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:46 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1611 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:46.814312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:46.814844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:46.832871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.833662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.834834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.835694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:46.837023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:46.837783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:46.929809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:46.977335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:46.983471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:46.986402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70ddeee7-9b39-4d7d-b168-cd21f99f9df3 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.914280 Oct 15 20:52:46.991609 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 690/1443] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:46 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 920 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:47.039634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '81fdbd60e924457692039abcb456cb89', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '08e2c30e5266478893afa293c836a3ed', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:47.040916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] flavor_id is None. Assuming migration. {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4196}} Oct 15 20:52:47.041562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 20:52:47.055982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:47.057664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:47.061943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:47.071932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:47.075437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:47.076346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:47.078161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:47.081313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:47.081945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:47.388150 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0fe1443f-adef-4f4c-af53-8b11c8aedb4a tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363/action" status: 202 len: 0 microversion: 2.1 time: 0.577199 Oct 15 20:52:47.388790 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 754/1444] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:52:46 2022] POST /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363/action => generated 0 bytes in 579 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:52:47.396969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce2e993f-acbd-4e5e-aab5-338a2738e398 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f463dc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:47.487676 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce2e993f-acbd-4e5e-aab5-338a2738e398 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.096022 Oct 15 20:52:47.496245 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 755/1445] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:47 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:47.501415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:47.502224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:47.508962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:47.509505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:47.510084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:47.511360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:47.511937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:47.512373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:47.540551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:47.554404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8308ce5d-30ad-4218-94c3-ea571d68bbd7 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.501635 Oct 15 20:52:47.555264 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 691/1446] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:47 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:47.576551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:47.577312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:47.583990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:47.584811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:47.585506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:47.586522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:47.587298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:47.588199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:47.744124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:47.756442 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-07155fcf-0868-41d5-889e-9524ed7a77a7 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1469 microversion: 2.1 time: 0.258649 Oct 15 20:52:47.757529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 756/1447] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:47 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1469 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:47.910991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:47.922527 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0177396c-ea76-4780-adc7-201b5bf20f8a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.351623 Oct 15 20:52:47.923730 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 692/1448] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:47 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:48.024140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:48.024799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:48.033559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.034568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.037536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.038842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.040563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.042540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.088487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:48.272534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:48.286174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:48.288298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:48.289801 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ee5a920b-2486-4df6-b3ed-37fbac9b6152 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.274852 Oct 15 20:52:48.290670 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 757/1449] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:48 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:48.507325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9af9281-6c11-428d-b7cc-bb48d088af53 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75418b4c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:48.584113 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9af9281-6c11-428d-b7cc-bb48d088af53 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.084388 Oct 15 20:52:48.584688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:48.585595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:48.593391 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 693/1450] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:48 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:48.594194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.595183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.595870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.597162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.598123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.598966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.781011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:48.781663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:48.798015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.803009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.804260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.811432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.811974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.812424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.868709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:48.880356 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f2c47b4a-27b9-4aed-993e-c97ce89e82cf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.301096 Oct 15 20:52:48.895955 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 758/1451] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:48 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:48.911136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-03231c0b-e41d-418e-aae4-9e676b6ec5e1 req-b14a0df8-01b2-4d0b-894c-3159b9a98062 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "name": "network-vif-unplugged", "status": "completed", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:48.937213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-03231c0b-e41d-418e-aae4-9e676b6ec5e1 req-b14a0df8-01b2-4d0b-894c-3159b9a98062 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:48.937785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-03231c0b-e41d-418e-aae4-9e676b6ec5e1 req-b14a0df8-01b2-4d0b-894c-3159b9a98062 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:48.941962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-03231c0b-e41d-418e-aae4-9e676b6ec5e1 req-b14a0df8-01b2-4d0b-894c-3159b9a98062 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:48.958226 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-03231c0b-e41d-418e-aae4-9e676b6ec5e1 req-b14a0df8-01b2-4d0b-894c-3159b9a98062 service nova] Creating event network-vif-unplugged:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:52:48.969023 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-03231c0b-e41d-418e-aae4-9e676b6ec5e1 req-b14a0df8-01b2-4d0b-894c-3159b9a98062 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.070150 Oct 15 20:52:48.979146 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 759/1452] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:48 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:48.992282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:48.998778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:49.016697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:49.018941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:49.019496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:49.022629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:49.023213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:49.025599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:49.234783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:49.244095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf0f0537-6a66-4dca-89f9-10ab6c8dca37 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1469 microversion: 2.1 time: 0.468112 Oct 15 20:52:49.246247 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 694/1453] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:48 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1469 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:49.293738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:49.302158 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-59c2f469-98a0-4790-ba32-507ee48f108c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.321431 Oct 15 20:52:49.303987 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 760/1454] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:48 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:49.319692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:49.320496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:49.329792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:49.330290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:49.330725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:49.331661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:49.332123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:49.332967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:49.559977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:49.568544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:49.570237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:49.571507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d86b165d-a134-4c75-8ead-e18fe0019540 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.257462 Oct 15 20:52:49.572195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 695/1455] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:49 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:49.600428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a7dbddd-959c-47d3-a2d7-d66f7ac7d713 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe754231c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:49.653183 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a7dbddd-959c-47d3-a2d7-d66f7ac7d713 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.056778 Oct 15 20:52:49.654637 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 761/1456] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:49 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:49.744135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TenantUsagesTestJSON-server-308751500", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:49.803880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:52:49.908202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:49.909640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:49.922534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:49.924347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:49.925068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:49.926253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:49.928449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:49.931209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.023537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:52:50.024266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:52:50.024760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:52:50.025216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:52:50.025845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:52:50.036288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.036787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.037271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.038026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.038495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.038931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.059232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Getting quotas for project b2227fe5e2aa4eddba19afff48b1d11a. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:52:50.064869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Getting quotas for user e92c87e9b9c840e6a4bc73ee0593a2c6 and project b2227fe5e2aa4eddba19afff48b1d11a. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:52:50.080207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:52:50.082445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:52:50.131816 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2347eb2b-18cc-4949-84eb-2871bbab17ae tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.521237 Oct 15 20:52:50.137108 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 696/1457] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:52:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 523 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:52:50.152557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-157ec5a8-ba9a-4811-be2a-49f4b9165e9d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:50.153395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-157ec5a8-ba9a-4811-be2a-49f4b9165e9d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:50.239345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-157ec5a8-ba9a-4811-be2a-49f4b9165e9d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:50.248957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-157ec5a8-ba9a-4811-be2a-49f4b9165e9d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1213 microversion: 2.1 time: 0.101418 Oct 15 20:52:50.249571 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 697/1458] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:50 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1213 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:50.274076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:50.274700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:50.283096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.283848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.284862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.286114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.286953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.287830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.383581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:50.392963 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b7d94065-1f15-4d69-858c-78510ccaff2c tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.490320 Oct 15 20:52:50.393579 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 762/1459] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:49 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:50.400278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:50.400802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:50.423831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.424387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.425190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.426220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.426937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.427617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.662810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:50.684083 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9cbc0fe3-5ba2-48b3-90d6-555a1c57a019 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1469 microversion: 2.1 time: 0.414492 Oct 15 20:52:50.685337 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 698/1460] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:50 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1469 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:50.695388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:50.696954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:50.713490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.714313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.714931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.715935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:50.724149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:50.724994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:50.867857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:50.894646 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0c9827cb-d2b8-4573-9041-bfa216557c72 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.496059 Oct 15 20:52:50.898529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 763/1461] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:50 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:50.919172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b10d093b-19c6-4868-846d-f4634d685946 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f3c68b0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:51.023965 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b10d093b-19c6-4868-846d-f4634d685946 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.121964 Oct 15 20:52:51.034695 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 764/1462] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:50 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:51.274005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:51.274485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:51.299458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:51.301814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:51.302279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:51.306488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:51.307048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:51.309814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:51.355485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:51.373310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:51.375508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:51.377919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31a08759-111e-465d-804f-f122217f5f38 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.690295 Oct 15 20:52:51.380911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 699/1463] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:50 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:51.417642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:51.418680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:51.434289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:51.435090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:51.435549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:51.436482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:51.436988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:51.437478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:51.452344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:51.610993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:51.639777 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1b80b31e-a4a7-4aae-b6d4-f8cc64f80a08 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1209 microversion: 2.1 time: 0.369983 Oct 15 20:52:51.642523 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 765/1464] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:51 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1209 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:51.709854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:51.710495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:51.721604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:51.722073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:51.722521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:51.723253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:51.730060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:51.730653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.021223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:52.041705 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c1c5342-95ee-491a-a50b-4954df24527b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.629720 Oct 15 20:52:52.057509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 700/1465] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:51 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:52.068289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:52.069781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:52.081493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:52.082225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:52.082832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.084156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:52.084836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:52.095970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.391194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:52.406127 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f4bf8e5-222e-4d1a-8ae8-05c2a60471d9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1469 microversion: 2.1 time: 0.701840 Oct 15 20:52:52.410308 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 766/1466] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:51 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1469 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:52.423608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d31b5ea1-b0db-42fb-8a9c-ee0371e07e84 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c9940>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:52.456752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:52.486293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c64072e8-d808-4fd5-b13e-407e2adbb357 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.416964 Oct 15 20:52:52.489106 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 701/1467] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:52 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:52.494046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:52.494651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:52.498641 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d31b5ea1-b0db-42fb-8a9c-ee0371e07e84 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.086731 Oct 15 20:52:52.505281 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 767/1468] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:52 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:52.505577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:52.505577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:52.506045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.506928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:52.507548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:52.508147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.672713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:52.674641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:52.687916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:52.688708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:52.691673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.692710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:52.693827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:52.695055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:52.936833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:52.952975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:52.960424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:52.964412 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d921e327-c9e3-4f25-a404-6a75f2c4f918 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.474092 Oct 15 20:52:52.965360 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 702/1469] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:52 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:53.074189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:53.075708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:53.077640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:53.087088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:53.087597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:53.088046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:53.091498 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9f5959d2-094f-41fc-a936-652bc647b74c tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1273 microversion: 2.1 time: 0.425615 Oct 15 20:52:53.091985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:53.092664 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 768/1470] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:52 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1273 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:53.112582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:53.113825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:53.434920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:53.435546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:53.437603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=10 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:53.464776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:53.469111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:53.469962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:53.470766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:53.471248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:53.472105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:53.817642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:53.831780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2062a925-5ba9-437d-b7b3-65407dd3d368 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1469 microversion: 2.1 time: 0.401368 Oct 15 20:52:53.833063 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 769/1471] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:53 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1469 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:53.838693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:53.840219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:53.848446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:53.850464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:53.851366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:53.851970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:53.852704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:53.853399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:53.853836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:53.909918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:53.923276 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d89c8625-4240-424d-90d6-eb6d9d7a6041 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.856103 Oct 15 20:52:53.937420 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 703/1472] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:53 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:53.950480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1461cc64-593a-4a34-ae10-366373ad2fe7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75533d8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:54.030496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1461cc64-593a-4a34-ae10-366373ad2fe7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.091425 Oct 15 20:52:54.032055 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 704/1473] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:53 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:54.045550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:54.048788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:54.067741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.068352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.070254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.071896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.072604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.076343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.095127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:54.110741 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17c0ba57-7e46-4530-9223-2f0fac86e640 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.272654 Oct 15 20:52:54.111846 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 770/1474] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:53 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:54.121198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:54.125535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:54.134136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.134964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.135647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.137143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.137877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.138527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.257797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:54.269350 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7b44873e-d56d-4d87-9576-c8260297749b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1271 microversion: 2.1 time: 0.151054 Oct 15 20:52:54.277917 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 771/1475] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:54 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1271 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:54.512884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:54.525562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:54.530303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:54.531786 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e75e4665-2047-4c75-9803-b33e277cab86 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.498516 Oct 15 20:52:54.534047 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 705/1476] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:54 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:54.857763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:54.858538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:54.867139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.868458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.869135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.870088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.871979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.872839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.955569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:54.956498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:54.965087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.965953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.966860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:54.967917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:54.968587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:54.970341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.107782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.127008 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-92ebb329-c816-472c-bf3c-9b2bf8ed5708 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.273907 Oct 15 20:52:55.142492 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 772/1477] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:54 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:55.155653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-657f14f1-a7e2-4aae-8817-6d691f66de7c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c5040>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:55.244040 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-657f14f1-a7e2-4aae-8817-6d691f66de7c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.098866 Oct 15 20:52:55.246337 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 773/1478] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:55 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:55.248914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.251173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:55.251783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:55.260937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:55.262111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:55.262806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.264311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:55.264564 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61a62f69-0560-4794-b94c-85bdf30c9b9a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.313420 Oct 15 20:52:55.265081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:55.265903 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 706/1479] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:54 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:55.266145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.299449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:55.300123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:55.324735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:55.325281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:55.328096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.330406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:55.332690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:55.335394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.577348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.586887 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ee990ce-1a66-47da-b2e0-116e9af91f8b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.339450 Oct 15 20:52:55.595092 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 774/1480] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:55 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:55.600399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:55.601022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:55.619054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.630072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:55.630575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:55.631022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.632172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:55.632679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:55.634283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4663bfc7-945e-45fe-b1e0-c0062b80562b tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.343746 Oct 15 20:52:55.634437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:55.638824 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 707/1481] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:55 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:55.872535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.892612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.896283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:55.898365 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-37f500df-2d40-4048-8550-ccde64d8771a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.301934 Oct 15 20:52:55.899553 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 775/1482] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:55 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:56.159289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:56.159917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:56.168935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.169920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.170791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.171725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.172560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.173535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.261388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f936cd2d-bf0e-4cf7-88cb-3fc5ff9f10e5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2cd0d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:56.313926 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f936cd2d-bf0e-4cf7-88cb-3fc5ff9f10e5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.057524 Oct 15 20:52:56.314839 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 776/1483] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:56 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:56.320263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:56.321651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:56.330920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.332416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.333188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.334206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.335304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.336060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.446786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:56.455949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6d2f4af-36e8-426f-930c-2e1fb13cd0e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.301913 Oct 15 20:52:56.469879 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 708/1484] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:56 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:56.479099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b16c755c-ab47-4ab8-ae4d-35222ac7b62e req-57587af3-c161-4ff2-954b-279c890bfe2c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0", "tag": "954a6345-f2b2-44b7-bdf4-c5c592d58ca6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:56.490266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b16c755c-ab47-4ab8-ae4d-35222ac7b62e req-57587af3-c161-4ff2-954b-279c890bfe2c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.491032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b16c755c-ab47-4ab8-ae4d-35222ac7b62e req-57587af3-c161-4ff2-954b-279c890bfe2c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.491724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b16c755c-ab47-4ab8-ae4d-35222ac7b62e req-57587af3-c161-4ff2-954b-279c890bfe2c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.510119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b16c755c-ab47-4ab8-ae4d-35222ac7b62e req-57587af3-c161-4ff2-954b-279c890bfe2c service nova] Creating event network-changed:954a6345-f2b2-44b7-bdf4-c5c592d58ca6 for instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 on nch1-master-1 Oct 15 20:52:56.516514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b16c755c-ab47-4ab8-ae4d-35222ac7b62e req-57587af3-c161-4ff2-954b-279c890bfe2c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045182 Oct 15 20:52:56.517395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 709/1485] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:56.622408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:56.623405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:56.637792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.638924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.639604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.640587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.641355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.642025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.909132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:56.924310 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4e06869f-88d5-4f53-a57f-b591b91bb144 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.306325 Oct 15 20:52:56.940314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 710/1486] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:56 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:56.952942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:56.954139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:56.961129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.961923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.962673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.963640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:56.964494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:56.965190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:56.968314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:56.977596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-632ebaeb-5ede-4e0e-8c26-fe56b3ec9538 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.661275 Oct 15 20:52:57.008068 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 777/1487] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:56 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:57.020450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:57.021946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:57.032256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.033284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.034748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:57.036177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.036819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.037782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:57.231785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:57.254767 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9a90689-eb9b-4f66-ba9f-ed33e583a55e tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.306282 Oct 15 20:52:57.271686 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 711/1488] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:56 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:57.280858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a767ba7d-57ab-4f00-8173-0e707424a757 req-e48ca63f-56cd-4045-8576-e9f9d71820ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "94ed23bf-05fc-41bf-a0d4-547148104a6f", "name": "network-vif-plugged", "status": "completed", "tag": "a172cd8d-3a0e-48dc-be83-b54ffefe0f32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:57.293073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a767ba7d-57ab-4f00-8173-0e707424a757 req-e48ca63f-56cd-4045-8576-e9f9d71820ea service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.293835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a767ba7d-57ab-4f00-8173-0e707424a757 req-e48ca63f-56cd-4045-8576-e9f9d71820ea service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.294616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a767ba7d-57ab-4f00-8173-0e707424a757 req-e48ca63f-56cd-4045-8576-e9f9d71820ea service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:57.311973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a767ba7d-57ab-4f00-8173-0e707424a757 req-e48ca63f-56cd-4045-8576-e9f9d71820ea service nova] Creating event network-vif-plugged:a172cd8d-3a0e-48dc-be83-b54ffefe0f32 for instance 94ed23bf-05fc-41bf-a0d4-547148104a6f on nch2-master-1 Oct 15 20:52:57.318534 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a767ba7d-57ab-4f00-8173-0e707424a757 req-e48ca63f-56cd-4045-8576-e9f9d71820ea service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044448 Oct 15 20:52:57.319517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 712/1489] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:57.332510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04ea8dc1-bd62-4186-93f4-09e670c93ad2 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75418b9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:57.351791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:57.362873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:57.364775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:57.366288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b7f639a-ebd4-4f23-8e26-c7900756e5c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.355968 Oct 15 20:52:57.381567 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 778/1490] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:57 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:57.406819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04ea8dc1-bd62-4186-93f4-09e670c93ad2 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.080465 Oct 15 20:52:57.407748 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 713/1491] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:57 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:57.485577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:57.486190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:57.492835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.493959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.494614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:57.496021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.496594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.498010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:57.765596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:57.769257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:52:57.776125 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4eaf9ca8-fa3a-4196-a2e5-f946ddd901e9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.294771 Oct 15 20:52:57.790204 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 779/1492] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:57 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:57.954699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:57.955443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:57.962259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.962760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.963215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:57.963931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:57.965600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:57.966049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.012798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:58.020269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:58.025001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.025517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.026276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.027252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.028365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.028800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.132887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:58.143983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46d3bab0-a6e7-4888-993e-0c7bd3be3e80 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.192879 Oct 15 20:52:58.157317 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 714/1493] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:57 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:58.290865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:58.292522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:58.303722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.304493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.305158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.305882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.306781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.307294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.324265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:58.334971 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0686f90-4193-4cd0-9407-bc7bd936cca9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.327512 Oct 15 20:52:58.341985 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 780/1494] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:58 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:58.404892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:58.405829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:58.431119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.432957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.434956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.437492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.438262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.440431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.765791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:58.797153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d1d23be9-2080-4355-8293-7dc3c727729f tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.513109 Oct 15 20:52:58.798350 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 715/1495] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:58 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:58.814487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-132406d3-96af-466e-ac11-48ba195afdeb tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7585785e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:58.899537 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-132406d3-96af-466e-ac11-48ba195afdeb tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.088162 Oct 15 20:52:58.908664 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 716/1496] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:52:58 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:58.920672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:58.921695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:58.928518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.929407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.930094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:58.931090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:58.931778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:58.932432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.001967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:59.016290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:59.021172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:59.023430 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3a56b237-4ecc-44ea-9664-b6b57f6bcf68 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.631635 Oct 15 20:52:59.029462 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 781/1497] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:52:58 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:52:59.168500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:59.169388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:59.184006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.187563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.188265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.189470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.191325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.192399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.391436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:59.405936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-677aefde-c518-4dfd-bf1e-9bae9f8e9667 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.495662 Oct 15 20:52:59.407492 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 717/1498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:58 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:52:59.413753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-51d26053-de79-4f51-844d-0b045e9d51de req-bb125f0c-b20a-419b-8c15-12678da105f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed61048c-5a28-4c6d-becc-59c01ab7089d", "name": "network-vif-plugged", "status": "completed", "tag": "c1c0e9d4-bc1c-433c-bbab-d38dd8f326f7"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:52:59.467184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-51d26053-de79-4f51-844d-0b045e9d51de req-bb125f0c-b20a-419b-8c15-12678da105f3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.467984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-51d26053-de79-4f51-844d-0b045e9d51de req-bb125f0c-b20a-419b-8c15-12678da105f3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.473720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-51d26053-de79-4f51-844d-0b045e9d51de req-bb125f0c-b20a-419b-8c15-12678da105f3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.515043 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-51d26053-de79-4f51-844d-0b045e9d51de req-bb125f0c-b20a-419b-8c15-12678da105f3 service nova] Creating event network-vif-plugged:c1c0e9d4-bc1c-433c-bbab-d38dd8f326f7 for instance ed61048c-5a28-4c6d-becc-59c01ab7089d on nch2-master-1 Oct 15 20:52:59.531765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-51d26053-de79-4f51-844d-0b045e9d51de req-bb125f0c-b20a-419b-8c15-12678da105f3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.121991 Oct 15 20:52:59.532426 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 718/1499] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:52:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:52:59.548726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:59.550372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:59.569609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.575728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.576396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.577371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.578735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.579346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.880004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:52:59.894845 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c90ea72-9fb3-4b75-a49a-bd6abb33f23a tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.730895 Oct 15 20:52:59.895647 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 782/1500] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:59 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:52:59.912737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:52:59.915185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:52:59.931407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.933865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.934612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:52:59.938979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:52:59.939801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:52:59.940469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:00.050076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=26 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:00.087191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:00.097469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f69a6f6-ca0e-4df7-813b-fbe62f65e174 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.552895 Oct 15 20:53:00.110763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 719/1501] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:59 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:00.127637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c16eab4-91f9-4468-bafc-700177b651c7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe758576af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:00.244627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c16eab4-91f9-4468-bafc-700177b651c7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.125734 Oct 15 20:53:00.253095 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 720/1502] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:00 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:00.256139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:00.260439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:00.262688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=90 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:00.280070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:00.280859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:00.281510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:00.282115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:00.284943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:00.286621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:00.330510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:00.340134 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23b35cfc-f38e-4c95-86fb-f24e69f86995 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.442500 Oct 15 20:53:00.362363 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 783/1503] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:52:59 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:00.434580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:00.436184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:00.459187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:00.460053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:00.460629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:00.461624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:00.462268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:00.462847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:00.572226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:00.583392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:00.585386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:00.586984 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e327dfe-a9b7-4353-aedb-b33575ad0850 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.339447 Oct 15 20:53:00.587551 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 721/1504] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:00 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:00.923489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:00.925630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:00.940189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:00.941099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:00.942032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:00.945893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:00.946893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:00.947585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:01.312050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:01.316572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:01.330251 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3438992f-3756-4067-b50c-e25543eb2bf8 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1316 microversion: 2.1 time: 0.410872 Oct 15 20:53:01.334526 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 722/1505] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:00 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1316 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:01.351804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:01.353951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:01.374213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:01.374729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:01.375183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:01.379039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:01.379532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:01.379983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:01.811287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:01.828595 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1d0bd5d-3dde-4e77-9382-86fcf67bbfcd tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.492905 Oct 15 20:53:01.829225 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 723/1506] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:01 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:01.835019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6349be1-fd2c-4bf5-ada4-e791a0c3a99a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75426ff70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:01.926102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6349be1-fd2c-4bf5-ada4-e791a0c3a99a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.094624 Oct 15 20:53:01.930559 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 724/1507] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:01 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:01.945822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:01.947704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:01.955206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:01.955978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:01.956642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:01.957549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:01.958171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:01.958823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.196596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:02.205735 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9e8881dc-7d93-4816-a917-0b41c5c73ef4 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 1.776974 Oct 15 20:53:02.208182 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 784/1508] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:00 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 1781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:02.220612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0b6ca7e2-bbe7-4d5e-9d18-ce535fc770a7 req-e5a5bad6-346c-4458-8180-134df92b1b56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-plugged", "status": "completed", "tag": "de10a24b-8347-41f7-9b6b-e7ae78428202"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:02.244374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0b6ca7e2-bbe7-4d5e-9d18-ce535fc770a7 req-e5a5bad6-346c-4458-8180-134df92b1b56 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.244912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0b6ca7e2-bbe7-4d5e-9d18-ce535fc770a7 req-e5a5bad6-346c-4458-8180-134df92b1b56 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.245556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0b6ca7e2-bbe7-4d5e-9d18-ce535fc770a7 req-e5a5bad6-346c-4458-8180-134df92b1b56 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.269847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:02.273389 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0b6ca7e2-bbe7-4d5e-9d18-ce535fc770a7 req-e5a5bad6-346c-4458-8180-134df92b1b56 service nova] Creating event network-vif-plugged:de10a24b-8347-41f7-9b6b-e7ae78428202 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:02.278675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb1cb821-de04-47eb-ac87-277b1b49193d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.344988 Oct 15 20:53:02.281445 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 725/1509] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:01 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:02.286448 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0b6ca7e2-bbe7-4d5e-9d18-ce535fc770a7 req-e5a5bad6-346c-4458-8180-134df92b1b56 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075869 Oct 15 20:53:02.287464 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 785/1510] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:02.288597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:02.289273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:02.299496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.300283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.300900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.301823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.302461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.303068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.365081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:02.366002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:02.384802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.385607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.386883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.388037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.388901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.389545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.553178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:02.563872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:02.565369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:02.573455 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1494ad24-9d41-43ca-8498-f3b6f8584047 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.290688 Oct 15 20:53:02.575268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 726/1511] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:02 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:02.855891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:02.856558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:02.866848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.867581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.868181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:02.869114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:02.875060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:02.875721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.236333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.248979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9610d725-3955-4b87-b26c-4c7e73bda45f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.396811 Oct 15 20:53:03.262409 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 727/1512] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:02 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:03.282165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab1ce1bf-b5ec-45ff-a8dc-72366fce649b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74fec6ca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:03.282659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.293112 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6f4de8d6-3b55-4b20-bcaf-4f18d2917ef7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1465 microversion: 2.1 time: 0.935097 Oct 15 20:53:03.297522 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 786/1513] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:02 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1465 bytes in 942 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:03.309208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:03.309954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:03.324046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.324708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.328124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.329389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.331992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.332698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.361887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab1ce1bf-b5ec-45ff-a8dc-72366fce649b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.098266 Oct 15 20:53:03.363373 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 728/1514] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:03 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:03.369646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:03.370576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:03.379834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.380676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.382313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.383239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.384135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.384793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.565624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.577221 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64507c0f-3fa0-437b-bf94-4931e6b9d8c2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.273357 Oct 15 20:53:03.578838 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 787/1515] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:03 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:03.582761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.586127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:03.587154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:03.592731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13431441-3e32-4553-986e-05a3d4fb64e0 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.228085 Oct 15 20:53:03.598451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 729/1516] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:03 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:03.599757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.607582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.610308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.616748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.617477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.618115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.618928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:03.632870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:03.653594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.654972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.657145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.660375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:03.660375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:03.660375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:03.906206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.941656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.950260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:03.954296 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-107f5ea8-39fe-4b99-b2e6-a8c9cdeb6a52 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.346672 Oct 15 20:53:03.956512 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 730/1517] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:03 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:04.175060 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-54ca7667-82b4-4751-b619-a4d918c00e80 tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1616 microversion: 2.1 time: 0.594946 Oct 15 20:53:04.199119 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 788/1518] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:03 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1616 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:04.276478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:04.278214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:04.290044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:04.292017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:04.295122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:04.297640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:04.300330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:04.300973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:04.347470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1708244732", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}, "os:scheduler_hints": {"same_host": "0028073f-e418-41fb-830e-0c49db2dee33"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:04.643871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:04.657529 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82ac70a2-9fca-4e11-82a0-56b94b5fc91f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.385126 Oct 15 20:53:04.671684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 731/1519] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:04 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:04.684428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d132292-81d3-4e53-99b1-31e5a2ce5ead tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7541d3af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:08.780486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:53:09.040970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d132292-81d3-4e53-99b1-31e5a2ce5ead tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 4.360261 Oct 15 20:53:09.051543 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 732/1520] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:04 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 4373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:09.061551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-plugged", "status": "completed", "tag": "4ed7714f-38f6-40a9-96ad-267754f29c1d"}, {"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-plugged", "status": "completed", "tag": "94e6278a-0fe8-4e57-8500-38f6a69ef5b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:09.080903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.081721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.082357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.098440 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] Creating event network-vif-plugged:4ed7714f-38f6-40a9-96ad-267754f29c1d for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:09.099359 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] Creating event network-vif-plugged:94e6278a-0fe8-4e57-8500-38f6a69ef5b8 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:09.106711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-45ac57db-7314-49b7-8de3-242deed2a3a5 req-18302557-cf27-4d8f-a8c0-4dd9a8232942 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.058963 Oct 15 20:53:09.113802 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 733/1521] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:09 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:09.122008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:09.123349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:09.130598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.131434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.132164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.139132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.140742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.141890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.332103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:53:09.333222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:53:09.334121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:53:09.334740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:53:09.335369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:53:09.358918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.360938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.361674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.362658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.363562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.364218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.395080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:53:09.404191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Getting quotas for user 1ddcfe1b0eb5431dbbe4777b8dd77436 and project b107efbbeb064e1fbf3c8bf31da12cbe. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:53:09.424234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:53:09.436167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:53:09.517710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:09.532070 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f97dd26-67dc-4d65-a2e1-f4705c69c72e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.416237 Oct 15 20:53:09.532987 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 734/1522] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:09 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:09.545374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:09.546389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:09.566023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.566906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.567145 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9905fe37-0481-4fc9-b3dd-a60da8f36a79 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.219458 Oct 15 20:53:09.568576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.568923 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 789/1523] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:53:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 5228 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:53:09.571107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.571814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.577691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.587325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:09.589214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:09.627367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.627879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.629082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.630024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.630638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.632241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.935895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:09.942914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:09.949820 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7201879a-595c-44d1-961b-9b1be85d55a3 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.407417 Oct 15 20:53:09.950671 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 735/1524] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:09 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:09.956309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:09.957392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:09.966879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.967631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.968304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.969300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:09.969989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:09.970647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:09.977528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:09.980963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:09.982213 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81ec6d91-02a6-4732-bcb5-c2a230428751 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.411473 Oct 15 20:53:09.991984 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 790/1525] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:09 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:09.998701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c86d336e-e10b-4760-97df-3f740d62533c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:10.003152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c86d336e-e10b-4760-97df-3f740d62533c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:10.098390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c86d336e-e10b-4760-97df-3f740d62533c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:10.105877 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c86d336e-e10b-4760-97df-3f740d62533c tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1217 microversion: 2.1 time: 0.112503 Oct 15 20:53:10.115878 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 791/1526] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:09 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1217 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:10.115878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb7ca478-a716-475b-8abd-1960bee26790 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f313160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:10.179890 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb7ca478-a716-475b-8abd-1960bee26790 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.071102 Oct 15 20:53:10.191902 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 792/1527] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:10 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:10.246493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:10.255710 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba1f942d-0108-4551-a67c-e9f8bc90f28a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1355 microversion: 2.1 time: 0.303644 Oct 15 20:53:10.262161 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 736/1528] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:09 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1355 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:10.564505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:10.568709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:10.577008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:10.577531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:10.577981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:10.578876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:10.580445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:10.580986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:10.748209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:10.796719 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-14acdff7-002e-474f-abae-8004b05fbef2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.232570 Oct 15 20:53:10.801739 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 793/1529] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:10 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:10.983928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:10.984556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:10.992238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:10.992903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:10.993481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:10.994215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:10.994699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:10.995127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.008662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.010274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:11.026469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.027629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.028323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.029326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.030012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.030686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.231007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:11.249788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:11.251911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:11.253292 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1bb40b2d-9708-4c66-a568-0c76599667ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.248669 Oct 15 20:53:11.262325 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 794/1530] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:11 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:11.279675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.280324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:11.288256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:11.290539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.291040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.291999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.293936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.294412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.294847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.329957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-94964355-3bbf-4396-a917-38349b0fbcbb tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.349742 Oct 15 20:53:11.350099 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 737/1531] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:10 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:11.372934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c05e4f55-245d-45f2-8567-e665ccbca061", "name": "network-vif-plugged", "status": "completed", "tag": "0e64f25a-9a1d-4631-8907-36be7c15d6f3"}, {"server_uuid": "6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0", "name": "network-vif-plugged", "status": "completed", "tag": "954a6345-f2b2-44b7-bdf4-c5c592d58ca6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:11.403354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.404398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.405135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.433414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] Creating event network-vif-plugged:0e64f25a-9a1d-4631-8907-36be7c15d6f3 for instance c05e4f55-245d-45f2-8567-e665ccbca061 on nch1-master-1 Oct 15 20:53:11.434086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] Creating event network-vif-plugged:954a6345-f2b2-44b7-bdf4-c5c592d58ca6 for instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 on nch1-master-1 Oct 15 20:53:11.441510 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b5f991cf-e8d0-4a1f-8dbc-18756e17f734 req-39dfbfb2-6c6b-4433-9744-60c550070488 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.087599 Oct 15 20:53:11.447012 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 738/1532] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:11 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:11.455235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdf1d19e-dd59-4c04-a0ae-a39b9c9a369f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75856f1f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.459709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:11.480280 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ab99a586-3bd4-4f18-aba9-aea00b6cdd62 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1213 microversion: 2.1 time: 0.216492 Oct 15 20:53:11.493779 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 795/1533] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:11 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1213 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:11.499007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.499809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:11.513624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.514399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.515033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.516076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.516705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.517323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.549491 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdf1d19e-dd59-4c04-a0ae-a39b9c9a369f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.100829 Oct 15 20:53:11.550718 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 739/1534] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:11 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:11.698562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:11.709211 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0019bbbd-b531-427d-95ec-7b5fa83b61b0 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1804 microversion: 2.1 time: 0.213887 Oct 15 20:53:11.710629 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 796/1535] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:11 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1804 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:11.736261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.737313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:11.744319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.745092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.745805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.746759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.747308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.747974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.783166 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e147c750-dbbf-408d-9bfd-378f4e6f37f3 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac/ips" status: 200 len: 217 microversion: 2.1 time: 0.053699 Oct 15 20:53:11.784516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 740/1536] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 20:53:11 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac/ips => generated 217 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:11.798600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.799340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:11.806176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.806673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.807113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.807870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.808353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.808779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.823724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:11.824624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:11.831321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.832037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.832664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.833636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:11.834249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:11.834840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:11.858315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:12.018521 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c85470db-59ae-4fa1-a7df-872856e1777b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "DELETE /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 204 len: 0 microversion: 2.1 time: 0.226115 Oct 15 20:53:12.020389 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 797/1537] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:11 2022] DELETE /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 0 bytes in 229 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:12.035769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:12.040863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:12.069188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.074382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.075189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.078327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.085765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.086555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.097428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:12.121893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.138348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b277d1b6-486b-42ac-b201-7e98dbfe66c1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.318548 Oct 15 20:53:12.144617 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 741/1538] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:11 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:12.292302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:12.293613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:12.301815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:12.306851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.307595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.308038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.309426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.309895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.310337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.478723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.489540 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b305e683-87a9-4ad0-a436-622b3636ce00 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1810 microversion: 2.1 time: 0.463820 Oct 15 20:53:12.497088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 798/1539] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:12 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1810 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:12.497088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:12.497420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:12.513558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.513558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.516949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.522663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.523164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.523614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.538160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.548106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.549914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.551455 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-92600fad-af98-4439-b187-ed782a18e25c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.264388 Oct 15 20:53:12.560222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 742/1540] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:12 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:53:12.564675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:12.567549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:12.575105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.576938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.577632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.590509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:12.591283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:12.592238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:12.684937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.693636 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f39d39d5-3c7b-4ecc-a5e4-92b8f3d14f0f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1269 microversion: 2.1 time: 0.133528 Oct 15 20:53:12.697228 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 743/1541] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:12 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1269 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:12.710763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9323c412-a41a-451d-9c65-2464becd69d3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7585719d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:12.737824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:12.753577 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7ade041b-c84e-4b41-b377-ae881ae3862d tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.254629 Oct 15 20:53:12.754913 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 799/1542] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:12 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:12.800100 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9323c412-a41a-451d-9c65-2464becd69d3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.104262 Oct 15 20:53:12.800975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 744/1543] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:12 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:13.180094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:13.180722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:13.187530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:13.188019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:13.188453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:13.189227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:13.190522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:13.190817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:13.374875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:13.382989 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-03a92c5a-0795-4bf4-b532-b466bd172df9 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1474 microversion: 2.1 time: 0.207412 Oct 15 20:53:13.393526 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 800/1544] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:13 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1474 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:13.532051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:13.536242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:13.551829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:13.553994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:13.555149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:13.557238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:13.565007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:13.566303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:13.586801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:13.586801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:13.595791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:13.598588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:13.599831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:13.601644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:13.602130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:13.602570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:13.970836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:13.983859 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32fa8b67-3cea-4b21-8c2c-d311293eb63f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1810 microversion: 2.1 time: 0.458222 Oct 15 20:53:13.987448 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 745/1545] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:13 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1810 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:14.015248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:14.019208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:14.047396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:14.047396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:14.047396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:14.047396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:14.047952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:14.047952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:14.047952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:14.142585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:14.142585 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51677273-b34b-49bb-9d6d-8b97eb0b5575 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1269 microversion: 2.1 time: 0.152455 Oct 15 20:53:14.142585 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 746/1546] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:13 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1269 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:14.153484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:14.154105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:14.173856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:14.174375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:14.174861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:14.175603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:14.176080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:14.176518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:14.416478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:14.437348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:14.441256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:14.450507 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-056e3230-f5b7-4940-91d7-c7a6c4ec3a4d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.874060 Oct 15 20:53:14.451679 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 801/1547] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:13 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 876 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:14.458349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a6d81f38-a506-4b29-ba8f-dba755227031 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f322dc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:14.520136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:14.554582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4dd39a08-a7ea-4c23-983b-504ba59a88a7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1313 microversion: 2.1 time: 0.411235 Oct 15 20:53:14.562718 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a6d81f38-a506-4b29-ba8f-dba755227031 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.109799 Oct 15 20:53:14.563286 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 747/1548] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:14 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1313 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:14.563612 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 802/1549] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:14 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:14.589947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:14.590722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:14.602529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:14.603335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:14.603958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:14.604855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:14.606306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:14.607069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:14.844107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:14.859335 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77a54b44-3d82-4c86-92ab-75b1986e0ca8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1473 microversion: 2.1 time: 0.293830 Oct 15 20:53:14.860292 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 748/1550] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:14 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1473 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:15.016907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:15.017851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:15.028316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.033709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.034378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.035292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.035900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.036502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.170468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:15.171254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:15.183454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.184306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.186018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.186852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.193180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.193180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.301141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:15.311278 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e37f209-cadf-4206-9702-da3a6427aaed tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1269 microversion: 2.1 time: 0.144622 Oct 15 20:53:15.313113 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 749/1551] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:15 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1269 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:15.492107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:15.493024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:15.500106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.500915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.501629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.502756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.503513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.504290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.560674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:15.633167 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0077902-893c-48ed-91c9-72b2378073c8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1810 microversion: 2.1 time: 0.618341 Oct 15 20:53:15.642857 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 803/1552] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:15 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1810 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:15.872165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "name": "network-vif-unplugged", "status": "completed", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:15.912020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:15.917478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:15.919003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:15.964780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] Creating event network-vif-unplugged:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch2-master-1 Oct 15 20:53:16.005299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:53:16.061108 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-62001521-5660-4b0a-abe4-0eec51ef5227 req-19f46ffc-0286-425f-8ec4-9789146df5f7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.416342 Oct 15 20:53:16.077348 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 804/1553] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:15 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:16.090404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-584982f5-5a67-4436-b513-6eb923ca0995 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f322550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:16.151480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:16.215647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:16.227602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:16.235565 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3222f3e8-8fca-46b3-8ae5-4756977ea10d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1412 microversion: 2.25 time: 0.754668 Oct 15 20:53:16.239818 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 750/1554] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:15 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1412 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:16.253122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:16.274894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:16.292297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:16.295177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:16.297941 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-584982f5-5a67-4436-b513-6eb923ca0995 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1453 microversion: 2.1 time: 0.218774 Oct 15 20:53:16.298734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:16.299722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:16.300383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:16.300562 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 805/1555] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:16 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1453 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:16.301205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:16.311311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:16.315506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:16.327170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:16.329703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:16.330933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:16.336320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:16.337306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:16.338054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:16.884936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:16.913482 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6958fb3d-170f-4bf3-af0c-77d59e9b4691 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1463 microversion: 2.1 time: 0.659250 Oct 15 20:53:16.918684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 751/1556] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:16 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1463 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:16.942220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-66a549d0-e5f6-49fb-b10b-bda0737e2cee req-3a62fd88-eec8-4e9f-8bc1-7d4137923799 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-unplugged", "status": "completed", "tag": "4ed7714f-38f6-40a9-96ad-267754f29c1d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:16.997383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66a549d0-e5f6-49fb-b10b-bda0737e2cee req-3a62fd88-eec8-4e9f-8bc1-7d4137923799 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:16.997910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66a549d0-e5f6-49fb-b10b-bda0737e2cee req-3a62fd88-eec8-4e9f-8bc1-7d4137923799 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:16.998354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66a549d0-e5f6-49fb-b10b-bda0737e2cee req-3a62fd88-eec8-4e9f-8bc1-7d4137923799 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:17.067389 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-66a549d0-e5f6-49fb-b10b-bda0737e2cee req-3a62fd88-eec8-4e9f-8bc1-7d4137923799 service nova] Creating event network-vif-unplugged:4ed7714f-38f6-40a9-96ad-267754f29c1d for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:17.109098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-66a549d0-e5f6-49fb-b10b-bda0737e2cee req-3a62fd88-eec8-4e9f-8bc1-7d4137923799 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.180735 Oct 15 20:53:17.109098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 752/1557] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:16 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:17.134111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:17.138540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:17.172257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:17.183117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:17.184079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:17.188415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:17.196243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:17.223298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:17.376032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:17.386444 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3295d7f5-6e26-439c-9ed3-92a7bcd68028 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1275 microversion: 2.1 time: 0.278551 Oct 15 20:53:17.414608 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 753/1558] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:17 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1275 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:17.420605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:17.432649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:17.467174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:17.469903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:17.469903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:17.472035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:17.473681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:17.476035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:17.576528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:17.600397 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-447e0e87-8aae-4ba4-b6a8-a26a09a1571f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1473 microversion: 2.1 time: 1.294579 Oct 15 20:53:17.618537 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 806/1559] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:16 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1473 bytes in 1317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:17.934971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Calling method '.version_select at 0x7fe74f3c6700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:17.955987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:17.957063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:17.958322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:18.011597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:18.013427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:18.014046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:18.014600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:18.023261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc267451-a70e-45bd-a755-a53047ceb4bf tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1768 microversion: 2.1 time: 0.607634 Oct 15 20:53:18.026333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 754/1560] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:17 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1768 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:18.036880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:18.038720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:18.054099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:18.056129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:18.057730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:18.060189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:18.062734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:18.063495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:18.064339 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-90292e9a-b558-4e02-baad-1f9752b8da92 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/b2227fe5e2aa4eddba19afff48b1d11a?start=2022-10-14T20%3A53%3A16.938996&end=2022-10-16T20%3A53%3A16.938996" status: 200 len: 694 microversion: 2.1 time: 0.443016 Oct 15 20:53:18.065562 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 807/1561] 10.222.0.16 () {58 vars in 1412 bytes} [Sat Oct 15 20:53:17 2022] GET /compute/v2.1/os-simple-tenant-usage/b2227fe5e2aa4eddba19afff48b1d11a?start=2022-10-14T20%3A53%3A16.938996&end=2022-10-16T20%3A53%3A16.938996 => generated 694 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:18.070572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98eafb1b-278c-4233-956c-7ba09a324e30 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f43a790>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:18.828950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:18.847697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:18.850965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:18.857860 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bafc6bc4-256a-488d-b0bf-5112d08885d1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1607 microversion: 2.25 time: 0.827657 Oct 15 20:53:18.859167 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 755/1562] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:18 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1607 bytes in 832 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:53:18.871646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '.version_select at 0x7fe758597a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:18.896779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:18.901688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:18.902378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:18.943350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:18.944112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:18.946161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:18.997622 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0c0896c-7a30-4312-8681-4dfd7462a097 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/b2227fe5e2aa4eddba19afff48b1d11a?start=2022-10-14T20%3A53%3A16.938996&end=2022-10-16T20%3A53%3A16.938996" status: 200 len: 694 microversion: 2.1 time: 0.135698 Oct 15 20:53:18.998541 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 756/1563] 10.222.0.16 () {58 vars in 1412 bytes} [Sat Oct 15 20:53:18 2022] GET /compute/v2.1/os-simple-tenant-usage/b2227fe5e2aa4eddba19afff48b1d11a?start=2022-10-14T20%3A53%3A16.938996&end=2022-10-16T20%3A53%3A16.938996 => generated 694 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:19.013431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-deleted", "tag": "de10a24b-8347-41f7-9b6b-e7ae78428202"}, {"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-deleted", "tag": "4ed7714f-38f6-40a9-96ad-267754f29c1d"}, {"name": "network-changed", "server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:19.041356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.041356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.041843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.087055 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Creating event network-vif-deleted:de10a24b-8347-41f7-9b6b-e7ae78428202 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:19.087905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Creating event network-vif-deleted:4ed7714f-38f6-40a9-96ad-267754f29c1d for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:19.090477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Creating event network-changed:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch2-master-1 Oct 15 20:53:19.116101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:53:19.160893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-69f2f193-ad42-4f79-9199-49350bad5b14 req-d9f00ad3-54ed-4c4e-a0b2-4c5021321f05 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 521 microversion: 2.1 time: 0.158443 Oct 15 20:53:19.161594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 757/1564] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:18 2022] POST /compute/v2.1/os-server-external-events => generated 521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:19.178300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:19.181661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:19.196861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.197710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.198336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.199299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.203732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.204427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.365683 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98eafb1b-278c-4233-956c-7ba09a324e30 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1462 microversion: 2.1 time: 1.298289 Oct 15 20:53:19.367179 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 808/1565] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:18 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1462 bytes in 1301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:19.387119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:19.395447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:19.415190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.415646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.416103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.421836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.422344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.422980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.751640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:19.761161 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5817c4e4-a4c1-4d0c-acd6-95e489d006f8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1473 microversion: 2.1 time: 0.392776 Oct 15 20:53:19.773686 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 809/1566] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:19 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1473 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:19.784220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:19.785287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:19.794607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.795403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.796129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.797175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.798076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.798734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.812798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:19.826909 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75803282-1572-4377-96c2-be20e2e5bea7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.662026 Oct 15 20:53:19.827554 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 758/1567] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:19 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:19.839455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Calling method '.version_select at 0x7fe7585351f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:19.868723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.869821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.871951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.899036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.899908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.900553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.920197 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69ac0cd7-53e5-44ef-b718-3c6a65e6fb61 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-interface" status: 200 len: 284 microversion: 2.25 time: 0.145009 Oct 15 20:53:19.928601 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 810/1568] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 20:53:19 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-interface => generated 284 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:53:19.928601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:19.928601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:19.948203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.949800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.950253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:19.950993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:19.951457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:19.951889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.042093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.042833 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ff4c24f-5c5b-483c-869b-b94579aa52b2 tempest-TenantUsagesTestJSON-1030950698 tempest-TenantUsagesTestJSON-1030950698-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage?start=2022-10-14T20%3A53%3A16.938996&end=2022-10-16T20%3A53%3A16.938996&detailed=1" status: 200 len: 16765 microversion: 2.1 time: 0.209167 Oct 15 20:53:20.043317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.044517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.044652 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 759/1569] 10.222.0.16 () {58 vars in 1335 bytes} [Sat Oct 15 20:53:19 2022] GET /compute/v2.1/os-simple-tenant-usage?start=2022-10-14T20%3A53%3A16.938996&end=2022-10-16T20%3A53%3A16.938996&detailed=1 => generated 16765 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:53:20.046578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lazy-loading 'fault' on Instance uuid aebc2aa6-a3c4-45d7-afd8-1503dbc64cac {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:53:20.057297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-051d90db-731d-4db2-bd62-77969d9d6059 req-64de7012-7111-4023-a4ac-4000e4e32482 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-unplugged", "status": "completed", "tag": "94e6278a-0fe8-4e57-8500-38f6a69ef5b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:20.081283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-051d90db-731d-4db2-bd62-77969d9d6059 req-64de7012-7111-4023-a4ac-4000e4e32482 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.082253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-051d90db-731d-4db2-bd62-77969d9d6059 req-64de7012-7111-4023-a4ac-4000e4e32482 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.082916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-051d90db-731d-4db2-bd62-77969d9d6059 req-64de7012-7111-4023-a4ac-4000e4e32482 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.101484 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-051d90db-731d-4db2-bd62-77969d9d6059 req-64de7012-7111-4023-a4ac-4000e4e32482 service nova] Creating event network-vif-unplugged:94e6278a-0fe8-4e57-8500-38f6a69ef5b8 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:20.105654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:20.111210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-051d90db-731d-4db2-bd62-77969d9d6059 req-64de7012-7111-4023-a4ac-4000e4e32482 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.064330 Oct 15 20:53:20.112594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 760/1570] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:20.118207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3093a15e-dd79-45d7-9762-990d2e1984dc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe758597940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:20.190210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3093a15e-dd79-45d7-9762-990d2e1984dc tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 200 len: 1462 microversion: 2.1 time: 0.075984 Oct 15 20:53:20.204039 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 761/1571] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:20 2022] GET /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 1462 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:20.210479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:20.211915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:20.221656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.222633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.223677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.227593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.228746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.229689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.270397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:20.449219 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02d908d4-42ea-4e3d-aab4-027d7854eb12 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 204 len: 0 microversion: 2.1 time: 0.241575 Oct 15 20:53:20.470182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 762/1572] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:20 2022] DELETE /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:20.484458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:20.486133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:20.494441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.495261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.496701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.497676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.498348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.498990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.805747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:20.819187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c4edbbb-3372-42d5-9b41-d5007e94967f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1481 microversion: 2.1 time: 0.346929 Oct 15 20:53:20.824199 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 763/1573] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:20 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1481 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:20.834012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:20.834881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:20.843043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.844594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.845308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:20.848872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:20.849742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:20.850906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.044854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:21.053525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-83c9beab-9576-4605-b441-e1dfe3a3ce22 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 200 len: 1303 microversion: 2.1 time: 1.131209 Oct 15 20:53:21.063425 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 811/1574] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:19 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 1303 bytes in 1141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:21.070537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:21.071333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:21.078412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.079137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.079729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.080601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.081354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.081923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.085618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:21.108898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f55eb8bc-e862-4bb4-90c7-df0e1deb77d7 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1469 microversion: 2.1 time: 0.282174 Oct 15 20:53:21.121062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 764/1575] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:20 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1469 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:21.121062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:21.121062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:21.126647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.127262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.127825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.128615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.129208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.129744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.158282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:21.314369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:21.323927 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af86f715-6c30-46ad-a94d-c98966f8e348 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.212879 Oct 15 20:53:21.326928 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 765/1576] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:21 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:21.335820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e45b86c0-3b33-43ca-ab78-5c5856808954 req-c8f4d3d4-48d2-443d-abc5-4894ae282b97 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "66c3144b-d02b-4ad2-b03a-6075b8a7529b", "tag": "bcf0950c-513f-4782-8f49-23235b3bc408"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:21.356955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e45b86c0-3b33-43ca-ab78-5c5856808954 req-c8f4d3d4-48d2-443d-abc5-4894ae282b97 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.357868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e45b86c0-3b33-43ca-ab78-5c5856808954 req-c8f4d3d4-48d2-443d-abc5-4894ae282b97 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.358103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e45b86c0-3b33-43ca-ab78-5c5856808954 req-c8f4d3d4-48d2-443d-abc5-4894ae282b97 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.358502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:21.369240 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b8a0db30-325b-4bce-a86c-7cf2cf5e0ff0 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1471 microversion: 2.1 time: 0.303798 Oct 15 20:53:21.374649 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 812/1577] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:21 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1471 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:21.379477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-355c27b2-4b9c-47ad-adc6-c2ace4dff54b req-39b64a9e-6e61-442c-a694-b12a2b2d7f34 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aebc2aa6-a3c4-45d7-afd8-1503dbc64cac", "name": "network-vif-deleted", "tag": "94e6278a-0fe8-4e57-8500-38f6a69ef5b8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:21.390505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e45b86c0-3b33-43ca-ab78-5c5856808954 req-c8f4d3d4-48d2-443d-abc5-4894ae282b97 service nova] Creating event network-changed:bcf0950c-513f-4782-8f49-23235b3bc408 for instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b on nch1-master-1 Oct 15 20:53:21.395381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-355c27b2-4b9c-47ad-adc6-c2ace4dff54b req-39b64a9e-6e61-442c-a694-b12a2b2d7f34 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.396090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-355c27b2-4b9c-47ad-adc6-c2ace4dff54b req-39b64a9e-6e61-442c-a694-b12a2b2d7f34 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.396404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e45b86c0-3b33-43ca-ab78-5c5856808954 req-c8f4d3d4-48d2-443d-abc5-4894ae282b97 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067810 Oct 15 20:53:21.397490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-355c27b2-4b9c-47ad-adc6-c2ace4dff54b req-39b64a9e-6e61-442c-a694-b12a2b2d7f34 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.397645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 766/1578] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:21.410520 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-355c27b2-4b9c-47ad-adc6-c2ace4dff54b req-39b64a9e-6e61-442c-a694-b12a2b2d7f34 service nova] Creating event network-vif-deleted:94e6278a-0fe8-4e57-8500-38f6a69ef5b8 for instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac on nch1-master-1 Oct 15 20:53:21.418290 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-355c27b2-4b9c-47ad-adc6-c2ace4dff54b req-39b64a9e-6e61-442c-a694-b12a2b2d7f34 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041763 Oct 15 20:53:21.419141 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 813/1579] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:21.852511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:21.853268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:21.863486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.869502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.871507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:21.874604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:21.879604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:21.880340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.091602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:22.092306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:22.094573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:22.107861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.108337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.108773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.109533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.109992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.110416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.149706 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] HTTP exception thrown: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Request method failure captured: Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac HTTP/1.1 Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2ANl7AV4DUGTAmJCJrJoBUGOPLq38faLYUZfcViDfXVEvnA_PNoZJFdNze6Vt9ihvNbHykKXsI_OpFPpZY5X2AN7w4Xgg5TdRRgfn8pIEhKZPCdQbqV3CubFinqCtxxFbMVTmObo3tgEYJbzoNGy9bnJvpixa-cFDSYU4dED45oOM Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:53:22.170593 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersTestMultiNic-246188881 Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersTestMultiNic-246188881-project Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 8bbc891924f84d99aac22518f69e7b99 Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersTestMultiNic-246188881-project Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:53:22.172266 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'aebc2aa6-a3c4-45d7-afd8-1503dbc64cac'} Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:22.173091 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:53:22.174416 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:22.177020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Returning 404 to user: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:22.179461 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52326352-f804-4745-abf3-fd8fe526dbe8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 404 len: 111 microversion: 2.1 time: 0.092642 Oct 15 20:53:22.183617 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 814/1580] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:22 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:22.195622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:22.199513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:22.213952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.214535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.215660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.216406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.217247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.218007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.461723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:22.476694 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-84028671-2b6f-4368-9adf-4b963bbd8911 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1469 microversion: 2.1 time: 0.287468 Oct 15 20:53:22.481089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 815/1581] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:22 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1469 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:22.493767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:22.494631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:22.505672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.506451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.507035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.507741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.508204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.508635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.536022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:22.551395 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7bf2f20-a36e-41e0-b2b1-9731a4df36ad tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1481 microversion: 2.1 time: 0.705199 Oct 15 20:53:22.561066 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 767/1582] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:21 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1481 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:22.561529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:22.562161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:22.582497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.583018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.583468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.587190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:22.587961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:22.596299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:22.732170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:22.742854 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-477e94f1-095c-4318-ae7c-3f88bb40c424 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.263824 Oct 15 20:53:22.753868 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 816/1583] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:22 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:23.110636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:23.130165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4dc9482-4b5c-4881-9374-5878c8b4285f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1471 microversion: 2.1 time: 0.573586 Oct 15 20:53:23.130784 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 768/1584] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:22 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1471 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:23.504884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:23.506433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:23.522307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.530525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-30193ba9-e258-494b-b690-d0d3f9f4ab3a req-51bb4d67-3bb7-438f-a523-94a7351c0448 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c05e4f55-245d-45f2-8567-e665ccbca061", "tag": "0e64f25a-9a1d-4631-8907-36be7c15d6f3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:23.544745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.548851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.023s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:23.549947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.552174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.553388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:23.575937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30193ba9-e258-494b-b690-d0d3f9f4ab3a req-51bb4d67-3bb7-438f-a523-94a7351c0448 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.578390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30193ba9-e258-494b-b690-d0d3f9f4ab3a req-51bb4d67-3bb7-438f-a523-94a7351c0448 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.582494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30193ba9-e258-494b-b690-d0d3f9f4ab3a req-51bb4d67-3bb7-438f-a523-94a7351c0448 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:23.612382 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-30193ba9-e258-494b-b690-d0d3f9f4ab3a req-51bb4d67-3bb7-438f-a523-94a7351c0448 service nova] Creating event network-changed:0e64f25a-9a1d-4631-8907-36be7c15d6f3 for instance c05e4f55-245d-45f2-8567-e665ccbca061 on nch1-master-1 Oct 15 20:53:23.619419 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-30193ba9-e258-494b-b690-d0d3f9f4ab3a req-51bb4d67-3bb7-438f-a523-94a7351c0448 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093433 Oct 15 20:53:23.622932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 769/1585] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:23.635137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:23.636102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:23.650277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.650784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.655953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:23.657474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.659535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.660599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:23.853914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:23.875886 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5206883f-eb56-47a7-9c79-472c7219d388 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1427 microversion: 2.1 time: 0.379705 Oct 15 20:53:23.890430 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 817/1586] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:23 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1427 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:23.900828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:23.901857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:23.909378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.910892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.911528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:23.912605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:23.913299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:23.913914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.116880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:24.126123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33e111b8-7ac9-4a7c-b9f5-2f23d447edd5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1481 microversion: 2.1 time: 0.501740 Oct 15 20:53:24.133183 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 770/1587] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:23 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1481 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:24.147615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e614e7f9-3d47-4186-8c97-1ce4c6ff4f8c req-892d6bff-61db-4f19-aeb0-ee23e729cab2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0", "name": "network-vif-deleted", "tag": "954a6345-f2b2-44b7-bdf4-c5c592d58ca6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:24.218364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e614e7f9-3d47-4186-8c97-1ce4c6ff4f8c req-892d6bff-61db-4f19-aeb0-ee23e729cab2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.220119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e614e7f9-3d47-4186-8c97-1ce4c6ff4f8c req-892d6bff-61db-4f19-aeb0-ee23e729cab2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.043s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.222750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e614e7f9-3d47-4186-8c97-1ce4c6ff4f8c req-892d6bff-61db-4f19-aeb0-ee23e729cab2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.274443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e614e7f9-3d47-4186-8c97-1ce4c6ff4f8c req-892d6bff-61db-4f19-aeb0-ee23e729cab2 service nova] Creating event network-vif-deleted:954a6345-f2b2-44b7-bdf4-c5c592d58ca6 for instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 on nch1-master-1 Oct 15 20:53:24.287787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e614e7f9-3d47-4186-8c97-1ce4c6ff4f8c req-892d6bff-61db-4f19-aeb0-ee23e729cab2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.158141 Oct 15 20:53:24.289024 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 771/1588] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:24.296566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:24.302769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:24.309818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.310466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.311015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.311845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.313485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.315024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.487551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:24.503862 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b7d49810-1c53-48da-a40f-89fc0953adb7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.607672 Oct 15 20:53:24.511278 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 818/1589] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:23 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:24.565370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:24.577585 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bff924c-c89c-4fa6-9a7d-ea12ecd1a7ed tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1468 microversion: 2.1 time: 0.286846 Oct 15 20:53:24.585065 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 772/1590] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:24 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1468 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:24.609508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:24.610049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:24.616984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.617635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.617995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.619061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.619806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.620400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.727194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:24.837332 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e371058b-676d-4a00-a13a-68bdf5ca961a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "POST /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363/action" status: 202 len: 0 microversion: 2.1 time: 0.230509 Oct 15 20:53:24.840521 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 819/1591] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:53:24 2022] POST /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363/action => generated 0 bytes in 236 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:53:24.847480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:24.848103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:24.860865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.864306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.864784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.865554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.872345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.873587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.908503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:24.909556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:24.922512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.925743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.929718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:24.932939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:24.934228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:24.938793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.026263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.027408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.028378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.029117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lazy-loading 'fault' on Instance uuid 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:53:25.128421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e92c87e9b9c840e6a4bc73ee0593a2c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2227fe5e2aa4eddba19afff48b1d11a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:25.158150 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b721a464-eefd-433c-8d03-721f61d91981 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 200 len: 1303 microversion: 2.1 time: 0.259682 Oct 15 20:53:25.180948 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 820/1592] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:24 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 1303 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:25.188145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:25.204791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:25.216872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.219235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.220368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.222028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.223185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.224553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.472108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:25.482788 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b423062-738e-47c3-95dd-9ec89db84c0e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.640135 Oct 15 20:53:25.483368 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 773/1593] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:24 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:25.500083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-40ac9b93-79d0-40d6-995c-c1c77b044dac req-eb90d9c7-dcdb-4265-9ab0-49bbae4b7a39 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "94ed23bf-05fc-41bf-a0d4-547148104a6f", "name": "network-vif-unplugged", "status": "completed", "tag": "a172cd8d-3a0e-48dc-be83-b54ffefe0f32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:25.531481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-40ac9b93-79d0-40d6-995c-c1c77b044dac req-eb90d9c7-dcdb-4265-9ab0-49bbae4b7a39 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.532265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-40ac9b93-79d0-40d6-995c-c1c77b044dac req-eb90d9c7-dcdb-4265-9ab0-49bbae4b7a39 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.532866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-40ac9b93-79d0-40d6-995c-c1c77b044dac req-eb90d9c7-dcdb-4265-9ab0-49bbae4b7a39 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.550661 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-40ac9b93-79d0-40d6-995c-c1c77b044dac req-eb90d9c7-dcdb-4265-9ab0-49bbae4b7a39 service nova] Creating event network-vif-unplugged:a172cd8d-3a0e-48dc-be83-b54ffefe0f32 for instance 94ed23bf-05fc-41bf-a0d4-547148104a6f on nch2-master-1 Oct 15 20:53:25.561964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:25.567588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-40ac9b93-79d0-40d6-995c-c1c77b044dac req-eb90d9c7-dcdb-4265-9ab0-49bbae4b7a39 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.077185 Oct 15 20:53:25.570634 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 774/1594] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:25 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:25.580484 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c6141de1-9c96-44bc-9abd-02440d79b59b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1468 microversion: 2.1 time: 0.397890 Oct 15 20:53:25.584453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:25.585410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:25.594356 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 821/1595] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:25 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1468 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:25.602279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.603653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.604309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.605903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.609793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.611154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.618289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-1104895005"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:25.623818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:25.644160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.647466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.648987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.650640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:25.651520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:25.652187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:25.975772 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d3b53a0c-8d2a-402a-a470-e7a6673f3f14 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "POST /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11/action" status: 202 len: 0 microversion: 2.1 time: 0.370895 Oct 15 20:53:25.983918 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 822/1596] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:53:25 2022] POST /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11/action => generated 0 bytes in 380 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:53:25.990190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8787ec93-161e-479f-93f5-373951bd923c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2faf70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:26.097722 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8787ec93-161e-479f-93f5-373951bd923c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.111858 Oct 15 20:53:26.098235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 823/1597] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:25 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:26.185637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:26.187106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] [instance: 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:26.203349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:26.205893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:26.207872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:26.208806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:26.211016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:26.211865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:26.240651 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] HTTP exception thrown: Instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 could not be found. Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Request method failure captured: Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 HTTP/1.1 Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2gBhbKgl-qDNOyIboStJbyHls5KqkdAZhMTONtYHb3-yrDlKQPdFZ5znG9wdBntSrzM6W_7H7PRYVpL7CBlY3E2yQfdbDccw3ipFTGhxzG1CUbFTJ_apO4wkO1KfseSjNR1NFy1kT29bWnVnJOAkJYKP7SaKWTHLIkgGMcXMuSg4s Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b2227fe5e2aa4eddba19afff48b1d11a Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TenantUsagesTestJSON-358343754 Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:53:26.243588 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b2227fe5e2aa4eddba19afff48b1d11a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b2227fe5e2aa4eddba19afff48b1d11a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b2227fe5e2aa4eddba19afff48b1d11a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b2227fe5e2aa4eddba19afff48b1d11a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TenantUsagesTestJSON-358343754 Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b2227fe5e2aa4eddba19afff48b1d11a Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TenantUsagesTestJSON-358343754 Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TenantUsagesTestJSON-358343754-project Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: e92c87e9b9c840e6a4bc73ee0593a2c6 Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TenantUsagesTestJSON-358343754-project Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 could not be found. Oct 15 20:53:26.244995 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0'} Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 could not be found. Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:26.246015 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 could not be found. Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 could not be found. Oct 15 20:53:26.247450 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:26.250363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] Returning 404 to user: Instance 6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:26.251417 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0eff678e-61dd-4299-a9e7-3c51a4a1e6c1 tempest-TenantUsagesTestJSON-358343754 tempest-TenantUsagesTestJSON-358343754-project] 10.222.0.16 "GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0" status: 404 len: 111 microversion: 2.1 time: 0.070173 Oct 15 20:53:26.255813 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 824/1598] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:26 2022] GET /compute/v2.1/servers/6413b36f-069a-4f5b-b7dc-37c3a5cc9ec0 => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:26.439674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:26.460308 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38cf6d9c-89ee-41fa-9921-457233dbf0f3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.888108 Oct 15 20:53:26.462287 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 775/1599] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:25 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:26.509840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:26.510644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:26.535637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:26.536198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:26.536928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:26.539376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:26.539852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:26.540290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:27.017222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:27.036033 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-24a30636-419d-4b62-a2cf-55fca9ad8576 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.524957 Oct 15 20:53:27.036691 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 825/1600] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:26 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:27.118827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99aefc8c-ab67-4eb2-bdb4-e73f9def558f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75850fdc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:27.236076 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99aefc8c-ab67-4eb2-bdb4-e73f9def558f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.125532 Oct 15 20:53:27.236705 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 776/1601] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:27 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:27.505442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:27.505442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:27.518403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:27.518403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:27.518403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:27.518403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:27.518775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:27.518775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:27.874408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:27.901767 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3b461292-a0ac-4054-9d2e-4c5b41b41c07 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.416528 Oct 15 20:53:27.909203 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 826/1602] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:27 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:28.060334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:28.061131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:28.068719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:28.074010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:28.075461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:28.078103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:28.078671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:28.079198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:28.152667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-9ca1c899-b568-43b1-9adb-1c4360d19f02 req-75ffa75c-6a5d-4b27-974c-c763db40b570 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "66c3144b-d02b-4ad2-b03a-6075b8a7529b", "name": "network-vif-plugged", "status": "completed", "tag": "bcf0950c-513f-4782-8f49-23235b3bc408"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:28.170134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:28.179293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9ca1c899-b568-43b1-9adb-1c4360d19f02 req-75ffa75c-6a5d-4b27-974c-c763db40b570 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:28.180301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9ca1c899-b568-43b1-9adb-1c4360d19f02 req-75ffa75c-6a5d-4b27-974c-c763db40b570 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:28.181280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9ca1c899-b568-43b1-9adb-1c4360d19f02 req-75ffa75c-6a5d-4b27-974c-c763db40b570 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:28.219990 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-9ca1c899-b568-43b1-9adb-1c4360d19f02 req-75ffa75c-6a5d-4b27-974c-c763db40b570 service nova] Creating event network-vif-plugged:bcf0950c-513f-4782-8f49-23235b3bc408 for instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b on nch1-master-1 Oct 15 20:53:28.250282 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-9ca1c899-b568-43b1-9adb-1c4360d19f02 req-75ffa75c-6a5d-4b27-974c-c763db40b570 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101507 Oct 15 20:53:28.265133 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 827/1603] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:53:28.292079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3437c59b-de59-4b7e-aa99-d3b9fa90d268 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2a2700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:28.398094 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3437c59b-de59-4b7e-aa99-d3b9fa90d268 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.131343 Oct 15 20:53:28.408678 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 828/1604] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:28 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:28.436804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:28.453554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-495ddcec-99d6-43b6-af58-b8b53b90e8db tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.395070 Oct 15 20:53:28.469787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 777/1605] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:28 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:28.929455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:28.930479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:28.938949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:28.939657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:28.940289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:28.940992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:28.941647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:28.942231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:29.173151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:29.180723 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c86b87d-9ccc-46be-a1f1-4646f9045838 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1317 microversion: 2.1 time: 0.255585 Oct 15 20:53:29.201441 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 829/1606] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:28 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1317 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:29.422181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae142b45-593f-4405-a1e8-b2ce72f52c7a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe758502e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:29.478156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:29.479230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:29.481836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae142b45-593f-4405-a1e8-b2ce72f52c7a tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.069599 Oct 15 20:53:29.490195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 778/1607] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:29 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:29.499754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:29.501531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:29.504661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:29.505705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:29.506531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:29.515221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:30.208827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:30.214525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:30.215962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:30.235072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:30.236431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:30.238044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:30.239026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:30.241877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:30.243838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:30.282079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:30.299204 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-955db249-2441-417a-b0e3-e09fd9ae291f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.822302 Oct 15 20:53:30.304035 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 830/1608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:29 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:30.567982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "name": "network-vif-plugged", "status": "completed", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:30.594684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:30.596998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:30.597677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:30.635985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:30.641024 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] Creating event network-vif-plugged:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch2-master-1 Oct 15 20:53:30.656535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b93403e-6056-4f4b-b494-3fec531fa2b1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1467 microversion: 2.1 time: 0.448437 Oct 15 20:53:30.659270 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 779/1609] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:30 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1467 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:30.666623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6725f053-8542-4017-8b27-9d0c6308ac1e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe754109040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:30.689300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:53:30.716998 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-813c3b5d-e0a2-4363-a722-8c6cf0d0d63c req-b9741981-7816-4204-adb8-f20e66a69516 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.382650 Oct 15 20:53:30.720220 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 831/1610] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:30.738773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:30.739508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:30.749367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:30.750239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:30.750843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:30.751864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:30.752509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:30.753170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:30.760644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6725f053-8542-4017-8b27-9d0c6308ac1e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.098632 Oct 15 20:53:30.761913 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 780/1611] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:30 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:30.865509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestMultiNic-server-283905103", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "8a0217d3-79be-4f44-b38b-ce6f9e57046a"}, {"uuid": "6ec39d6d-bd60-4f8f-a3b8-7e255d09fa53"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:30.938817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] validate_networks() for [('8a0217d3-79be-4f44-b38b-ce6f9e57046a', None, None, None, None, None), ('6ec39d6d-bd60-4f8f-a3b8-7e255d09fa53', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:53:31.062083 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-282866b4-a3f6-4b03-92f0-ee8f0a0ef33a tempest-ServersOnMultiNodesTest-1915481637 tempest-ServersOnMultiNodesTest-1915481637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1618 microversion: 2.1 time: 0.340027 Oct 15 20:53:31.062647 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 832/1612] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:30 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1618 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:31.095999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:31.096735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:31.108656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:31.109971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:31.110423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:31.111159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:31.113493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:31.115219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:31.171939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:31.519841 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-67857d9b-4774-4038-ad27-3ff0ccb3ec61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 204 len: 0 microversion: 2.1 time: 0.427890 Oct 15 20:53:31.521812 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 833/1613] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:31 2022] DELETE /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:31.528792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:31.532852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:31.550662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:31.552122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:31.552887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:31.554179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:31.555178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:31.555841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:31.812462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:31.828700 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e2d217fa-9e91-408e-bd82-a8c609cdc4df tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.304325 Oct 15 20:53:31.830304 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 834/1614] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:31 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:31.836460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:31.837398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:31.847207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:31.847971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:31.849628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:31.850562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:31.851467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:31.852234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:31.896598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:32.077089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:32.088684 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-29d7e640-cdbd-4bff-9f15-b5564c739345 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1473 microversion: 2.1 time: 0.256302 Oct 15 20:53:32.090955 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 835/1615] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:31 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1473 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:32.096186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-114d72a6-d7b8-4d08-b15e-7d25fede1ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2a4af0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:32.185175 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-114d72a6-d7b8-4d08-b15e-7d25fede1ea6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.092602 Oct 15 20:53:32.186832 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 836/1616] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:32 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:32.355528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:53:32.356627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:53:32.357448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:53:32.358887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:53:32.359661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:53:32.385170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:32.386368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:32.387578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:32.389280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:32.390170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:32.391423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:32.419990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Getting quotas for project 1c30aa11fd244564be716550bfe5aaa6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:53:32.428473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Getting quotas for user 8bbc891924f84d99aac22518f69e7b99 and project 1c30aa11fd244564be716550bfe5aaa6. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:53:32.446449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:53:32.451439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:53:32.545210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ff59462-8049-40c2-93e5-b58548122b67 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.683706 Oct 15 20:53:32.546631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 781/1617] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:53:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 1686 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:53:32.560195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87a9ad0d-45ff-4c34-8fba-031fb6211347 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:32.561426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87a9ad0d-45ff-4c34-8fba-031fb6211347 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:32.660664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-87a9ad0d-45ff-4c34-8fba-031fb6211347 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:32.671292 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87a9ad0d-45ff-4c34-8fba-031fb6211347 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1212 microversion: 2.1 time: 0.117231 Oct 15 20:53:32.690048 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 837/1618] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:32 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1212 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:32.862906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:32.863569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:32.878559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:32.879014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:32.880993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:32.883488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:32.884384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:32.886333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:33.118450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:33.119080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:33.140046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:33.140577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:33.141024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:33.141780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:33.142248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:33.142685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:33.375994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:33.388860 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8256ffdf-2fbd-4f0a-92df-c241bd70dc5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1473 microversion: 2.1 time: 0.274088 Oct 15 20:53:33.390310 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 838/1619] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:33 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1473 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:33.399959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b50956a2-88c4-4274-b2c2-8b4988b1ca4f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f28b790>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:33.474986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b50956a2-88c4-4274-b2c2-8b4988b1ca4f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.081464 Oct 15 20:53:33.478459 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 839/1620] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:33 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:33.697416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:33.698882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:33.724887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:33.725597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:33.725884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:33.730197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:33.732598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:33.735222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:33.744980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:33.760060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27c97d8f-3f84-4a36-aea6-a482cd12f943 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.900292 Oct 15 20:53:33.768362 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 782/1621] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:32 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 912 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:33.888818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:33.897799 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b887813b-a276-4c92-80b1-e78b3f6bb7d8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1216 microversion: 2.1 time: 0.205318 Oct 15 20:53:33.898387 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 840/1622] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:33 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1216 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:34.148309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:34.149384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:34.157555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:34.158406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:34.159100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:34.160048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:34.160864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:34.161829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:34.414234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:34.414845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:34.428017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:34.428508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:34.429180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:34.430105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:34.431449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:34.432327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:34.750882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:34.763844 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17071563-bb28-4c37-94ae-8d04ce5fafa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1473 microversion: 2.1 time: 0.352591 Oct 15 20:53:34.767145 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 841/1623] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:34 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1473 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:34.784456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5a45ba82-c931-48ea-b350-05dee141c71e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2fa700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:34.851076 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5a45ba82-c931-48ea-b350-05dee141c71e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.082439 Oct 15 20:53:34.859713 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 842/1624] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:34 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:34.859713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "name": "network-vif-unplugged", "status": "completed", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:34.901573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:34.904588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:34.905366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:34.943855 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] Creating event network-vif-unplugged:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch1-master-1 Oct 15 20:53:34.967561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:53:35.000771 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-87efa6e6-8adb-4f76-8b44-92d8a046b85f req-97beff52-b86f-4cea-9e6d-2188912fcb76 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.144923 Oct 15 20:53:35.004813 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 843/1625] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:34 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:35.053441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:35.058367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:35.093682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:35.095119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:35.095793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:35.100382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:35.102471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:35.105324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:35.732916 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd7476b2-5cf5-4b72-865b-17ea4a1e764e tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1812 microversion: 2.1 time: 1.587828 Oct 15 20:53:35.734457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 783/1626] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:34 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1812 bytes in 1591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:35.772495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:35.778515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:35.806483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:35.806985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:35.807427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:35.808145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:35.809150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:35.818060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:35.818554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:35.832814 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-261abe49-ed1f-47ad-82f9-1d43211ed358 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.826512 Oct 15 20:53:35.833937 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 844/1627] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:35 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:35.962492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b7a28e54-ed18-4812-9c82-b7e2f4480b12 req-ea8047d8-f508-4e6c-99c6-44ee2328dddf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "66c3144b-d02b-4ad2-b03a-6075b8a7529b", "name": "network-vif-deleted", "tag": "bcf0950c-513f-4782-8f49-23235b3bc408"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:35.988702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b7a28e54-ed18-4812-9c82-b7e2f4480b12 req-ea8047d8-f508-4e6c-99c6-44ee2328dddf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.000850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b7a28e54-ed18-4812-9c82-b7e2f4480b12 req-ea8047d8-f508-4e6c-99c6-44ee2328dddf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.001694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b7a28e54-ed18-4812-9c82-b7e2f4480b12 req-ea8047d8-f508-4e6c-99c6-44ee2328dddf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.021838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b7a28e54-ed18-4812-9c82-b7e2f4480b12 req-ea8047d8-f508-4e6c-99c6-44ee2328dddf service nova] Creating event network-vif-deleted:bcf0950c-513f-4782-8f49-23235b3bc408 for instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b on nch1-master-1 Oct 15 20:53:36.035407 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b7a28e54-ed18-4812-9c82-b7e2f4480b12 req-ea8047d8-f508-4e6c-99c6-44ee2328dddf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.199379 Oct 15 20:53:36.040775 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 845/1628] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:36.048035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:36.055695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.065811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.066661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.067720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.068414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.069095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.126860 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d9fd1841-d2d0-4f6e-a49d-3def18f08cea tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.084443 Oct 15 20:53:36.145497 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 846/1629] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:53:36 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:36.151425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:36.152738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:36.164253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.164832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.167549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.169357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.170537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.172131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.245829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.251883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.252371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.252853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lazy-loading 'fault' on Instance uuid 66c3144b-d02b-4ad2-b03a-6075b8a7529b {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:53:36.417362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:36.434843 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-153f2c2c-76fa-4eec-b5c3-db76690efaa0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 200 len: 1307 microversion: 2.1 time: 0.287528 Oct 15 20:53:36.436013 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 847/1630] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:36 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 1307 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:36.456621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35e19a33-0869-4ef4-9184-93fa1d4f78b6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2cdca0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:36.579846 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35e19a33-0869-4ef4-9184-93fa1d4f78b6 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.142194 Oct 15 20:53:36.591981 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 848/1631] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:36 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:36.610787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:36.631217 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c4a6b9b-48d5-4c05-ae05-f0a025b87704 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1270 microversion: 2.1 time: 0.879216 Oct 15 20:53:36.640162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 784/1632] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:35 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1270 bytes in 889 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:36.862865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:36.864895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:36.890672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.897142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.898014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:36.904452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:36.909922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:36.910698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:37.412437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ef2161fe-15f8-471e-b2b3-0d71e242f594 req-014036d4-1ea7-4270-a19d-6cbc78ceaa1c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "name": "network-vif-plugged", "status": "completed", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:37.431470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef2161fe-15f8-471e-b2b3-0d71e242f594 req-014036d4-1ea7-4270-a19d-6cbc78ceaa1c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:37.432796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef2161fe-15f8-471e-b2b3-0d71e242f594 req-014036d4-1ea7-4270-a19d-6cbc78ceaa1c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:37.433539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef2161fe-15f8-471e-b2b3-0d71e242f594 req-014036d4-1ea7-4270-a19d-6cbc78ceaa1c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:37.451516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ef2161fe-15f8-471e-b2b3-0d71e242f594 req-014036d4-1ea7-4270-a19d-6cbc78ceaa1c service nova] Creating event network-vif-plugged:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:53:37.458682 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ef2161fe-15f8-471e-b2b3-0d71e242f594 req-014036d4-1ea7-4270-a19d-6cbc78ceaa1c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057761 Oct 15 20:53:37.460020 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 785/1633] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:37.492102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:37.494781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 66c3144b-d02b-4ad2-b03a-6075b8a7529b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:37.513385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:37.515124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:37.515920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:37.516887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:37.519687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:37.521257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:37.579291 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b could not be found. Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b HTTP/1.1 Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:53:37.583968 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b could not be found. Oct 15 20:53:37.585937 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '66c3144b-d02b-4ad2-b03a-6075b8a7529b'} Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b could not be found. Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:37.586788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b could not be found. Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b could not be found. Oct 15 20:53:37.588043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:37.597191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance 66c3144b-d02b-4ad2-b03a-6075b8a7529b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:37.600329 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac175191-dd49-475e-9c5f-f5174592cdb3 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b" status: 404 len: 111 microversion: 2.1 time: 0.138656 Oct 15 20:53:37.601772 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 786/1634] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:37 2022] GET /compute/v2.1/servers/66c3144b-d02b-4ad2-b03a-6075b8a7529b => generated 111 bytes in 140 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:37.620270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e207d6e3-5368-40b7-91cf-0fa397cec290 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7584a9040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:37.733986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:37.748737 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-33d22fd0-3060-4c94-9b1c-8effc6a46abd tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.887447 Oct 15 20:53:37.759363 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 849/1635] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:36 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:37.778695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:37.782806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:37.797101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:37.798283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:37.801138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:37.803661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:37.804394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:37.805943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:37.918698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:38.203672 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-508ffc23-cf6f-4a08-b189-c35eb3d7a4c2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 204 len: 0 microversion: 2.1 time: 0.442908 Oct 15 20:53:38.205109 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 850/1636] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:37 2022] DELETE /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 0 bytes in 445 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:38.214847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:38.216508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:38.236189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:38.237700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:38.239355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:38.240328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:38.241444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:38.242148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:39.080451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:39.103933 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff9bd10f-10e7-40fb-8aac-e7ad7eb58065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.896364 Oct 15 20:53:39.105123 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 851/1637] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:38 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 899 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:39.128199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:39.132579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:39.171099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:39.171871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:39.172513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:39.175400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:39.176043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:39.182145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.234556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:40.253762 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f18ebd1-e8ee-4ac3-90b3-dcf12a6be669 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1471 microversion: 2.1 time: 1.137634 Oct 15 20:53:40.255397 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 852/1638] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:39 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1471 bytes in 1149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:40.264314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:40.268353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:40.279423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.280369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.281101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.282065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.282754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.283405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.432244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e207d6e3-5368-40b7-91cf-0fa397cec290 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 2.828075 Oct 15 20:53:40.432805 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 787/1639] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:37 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 2830 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:40.444857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:40.458067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:40.476929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.480047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.481225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.521698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] Creating event network-changed:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch1-master-1 Oct 15 20:53:40.538859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 20:53:40.591885 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5c87c467-c46c-4c24-b7b4-3d527171c8ef req-70330c94-776a-4341-bd93-7c3b970f29ab service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.154047 Oct 15 20:53:40.595904 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 788/1640] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:40.613508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7247b0f0-0975-42a1-86c8-08fe3b1f39e4", "device": "/dev/xvdb"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:40.621979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:40.641721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.642237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.642689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.647384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.647892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.648352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.919288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:40.930758 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cbd7026d-7e97-44fc-b864-224d79a0eab3 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.671174 Oct 15 20:53:40.931382 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 853/1641] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:40 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:40.943603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:40.947780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:40.963420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.967003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.967863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:40.968815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:40.970145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:40.970897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:41.359569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:41.404904 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36dc6f18-cb6f-4d59-8137-b6c67ff01eb0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "POST /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.806911 Oct 15 20:53:41.406108 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 789/1642] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 20:53:40 2022] POST /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments => generated 194 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:41.525827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:41.535413 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-792cd506-4e4c-4b18-ba5d-acc574f750b8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.602731 Oct 15 20:53:41.542405 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 854/1643] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:40 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:41.559743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:41.560742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:41.573856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:41.574699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:41.575646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:41.577617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:41.582356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:41.583027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:41.776842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15054c11-9580-410d-8500-1a8034c018ae tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Action: 'create', calling method: .version_select at 0x7fe7584e0670>, body: {"keypair": {"name": "tempest-keypair-1640437525", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCBfWCABD7S2uSjWd6PLRw8Kx3xs2QbMmJPeMI4O4yeTyHAoJHn9Ndu/C9oNci7VeERKEs6cGWgfA4+HtsU7d93x2Bi4JC5EFegj9RiNgzBf8eeFbuHcH6WBkC67I0/RUg=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:41.847879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-15054c11-9580-410d-8500-1a8034c018ae tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Getting quotas for project e67382792a7d4ccf8100febea9052e1b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:53:41.870811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:41.877130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-15054c11-9580-410d-8500-1a8034c018ae tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Getting quotas for user 999bc03f52824573bb3c8bdc6a1179cf and project e67382792a7d4ccf8100febea9052e1b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:53:41.891748 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-068b2db5-9033-4d0f-9640-1ce3c853e874 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1429 microversion: 2.1 time: 0.340086 Oct 15 20:53:41.897084 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 855/1644] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:41 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1429 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:41.915249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8cd94d6b-6065-4863-b123-183e56f06074 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2661f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:41.958469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15054c11-9580-410d-8500-1a8034c018ae tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.551515 Oct 15 20:53:41.959074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 790/1645] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:53:41 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:41.990351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-181a1d67-7b4e-43f2-8f09-15acf5a68872 req-11818e7d-8c25-487b-a1fe-ac82f23487bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c10c829d-ef62-4d1c-98e9-422615414c5e", "tag": "2557963e-5bb5-430b-9efd-87827090ab6c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:42.019582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-181a1d67-7b4e-43f2-8f09-15acf5a68872 req-11818e7d-8c25-487b-a1fe-ac82f23487bd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.020356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-181a1d67-7b4e-43f2-8f09-15acf5a68872 req-11818e7d-8c25-487b-a1fe-ac82f23487bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.021195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-181a1d67-7b4e-43f2-8f09-15acf5a68872 req-11818e7d-8c25-487b-a1fe-ac82f23487bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.044907 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8cd94d6b-6065-4863-b123-183e56f06074 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.145862 Oct 15 20:53:42.046169 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 856/1646] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:41 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:42.058895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-181a1d67-7b4e-43f2-8f09-15acf5a68872 req-11818e7d-8c25-487b-a1fe-ac82f23487bd service nova] Creating event network-changed:2557963e-5bb5-430b-9efd-87827090ab6c for instance c10c829d-ef62-4d1c-98e9-422615414c5e on nch1-master-1 Oct 15 20:53:42.061616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:42.062459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:42.074096 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-181a1d67-7b4e-43f2-8f09-15acf5a68872 req-11818e7d-8c25-487b-a1fe-ac82f23487bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095833 Oct 15 20:53:42.077691 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 791/1647] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:42.078887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.081972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.082618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.086001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.086714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.087321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.569221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:42.570543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:42.576175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:42.611258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.619625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.622251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.631454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.632322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.636784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.650861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:42.657107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:42.678711 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d24ea6c7-bb1a-495b-b058-45adb1641811 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1467 microversion: 2.1 time: 0.631153 Oct 15 20:53:42.682929 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 857/1648] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:42 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1467 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:42.790362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-3b6a2dbb-0b01-48ec-9448-a761f9b96548 req-55e6d369-76e6-490c-a551-e777dd0c56dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed61048c-5a28-4c6d-becc-59c01ab7089d", "name": "network-vif-deleted", "tag": "c1c0e9d4-bc1c-433c-bbab-d38dd8f326f7"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:42.804608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:42.843171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3b6a2dbb-0b01-48ec-9448-a761f9b96548 req-55e6d369-76e6-490c-a551-e777dd0c56dc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.845696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3b6a2dbb-0b01-48ec-9448-a761f9b96548 req-55e6d369-76e6-490c-a551-e777dd0c56dc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.850768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3b6a2dbb-0b01-48ec-9448-a761f9b96548 req-55e6d369-76e6-490c-a551-e777dd0c56dc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.905519 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-3b6a2dbb-0b01-48ec-9448-a761f9b96548 req-55e6d369-76e6-490c-a551-e777dd0c56dc service nova] Creating event network-vif-deleted:c1c0e9d4-bc1c-433c-bbab-d38dd8f326f7 for instance ed61048c-5a28-4c6d-becc-59c01ab7089d on nch2-master-1 Oct 15 20:53:42.917256 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-3b6a2dbb-0b01-48ec-9448-a761f9b96548 req-55e6d369-76e6-490c-a551-e777dd0c56dc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.130830 Oct 15 20:53:42.917874 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 858/1649] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:42.940022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:42.940742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:42.985778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:42.990232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.993233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:42.994575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:42.995704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:42.998221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:43.000245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:43.091764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:43.092785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:43.095479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:43.096140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lazy-loading 'fault' on Instance uuid ed61048c-5a28-4c6d-becc-59c01ab7089d {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:53:43.248946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:43.257895 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fbd8b233-81c5-4fe5-8304-c805ad88097e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 200 len: 1306 microversion: 2.1 time: 0.328086 Oct 15 20:53:43.273576 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 859/1650] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:42 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 1306 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:43.273576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-085d6176-0fba-4db6-a736-bd3c25e81f3f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f382a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:43.360978 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-085d6176-0fba-4db6-a736-bd3c25e81f3f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.094145 Oct 15 20:53:43.369369 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 860/1651] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:43 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:43.717026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:43.719718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:43.751948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:43.753450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:43.754132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:43.768236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:43.769468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:43.770374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:44.802173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:44.813537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e087148-7aa2-440e-89cc-7718eb502b4a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1460 microversion: 2.1 time: 1.105465 Oct 15 20:53:44.814962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 861/1652] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:43 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1460 bytes in 1107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:44.819777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:44.826044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: ed61048c-5a28-4c6d-becc-59c01ab7089d] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:44.841700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:44.842487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:44.845363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:44.846406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:44.847109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:44.854288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:44.855543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:44.864473 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f9258b0-e032-44f1-bae4-ab6fe1fcfdda tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 2.300300 Oct 15 20:53:44.865080 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 792/1653] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:42 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 2302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:44.895733 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance ed61048c-5a28-4c6d-becc-59c01ab7089d could not be found. Oct 15 20:53:44.899449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54ab3cd5-457c-4124-8c77-606901142d06 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75848f820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d HTTP/1.1 Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:53:44.901618 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance ed61048c-5a28-4c6d-becc-59c01ab7089d could not be found. Oct 15 20:53:44.903067 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ed61048c-5a28-4c6d-becc-59c01ab7089d'} Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ed61048c-5a28-4c6d-becc-59c01ab7089d could not be found. Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:44.903960 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ed61048c-5a28-4c6d-becc-59c01ab7089d could not be found. Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ed61048c-5a28-4c6d-becc-59c01ab7089d could not be found. Oct 15 20:53:44.905810 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:44.909184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance ed61048c-5a28-4c6d-becc-59c01ab7089d could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:44.910403 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-435b85dc-75fd-4642-939d-a92dc64030e5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d" status: 404 len: 111 microversion: 2.1 time: 0.094191 Oct 15 20:53:44.911856 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 862/1654] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:44 2022] GET /compute/v2.1/servers/ed61048c-5a28-4c6d-becc-59c01ab7089d => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:44.921952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:44.927655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:44.938880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:44.939904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:44.943256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:44.944324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:44.945791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:44.952603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.004283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54ab3cd5-457c-4124-8c77-606901142d06 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.118902 Oct 15 20:53:45.009222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 793/1655] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:44 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:45.023290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:45.025347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:45.044176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.047171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.047889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.048837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.050134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.051718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.123491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:45.323315 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6916be8-ed81-436a-ba0d-04477afa4d4f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 204 len: 0 microversion: 2.1 time: 0.310962 Oct 15 20:53:45.329068 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 794/1656] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:45 2022] DELETE /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:45.350269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:45.363474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:45.387906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.393825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.397383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.404198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.405562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.408341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.542039 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-11b08c75-04a6-4b68-9790-df353e7a4410 tempest-MigrationsAdminTest-425200727 tempest-MigrationsAdminTest-425200727-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1611 microversion: 2.1 time: 0.628825 Oct 15 20:53:45.546265 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 863/1657] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:44 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1611 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:45.594175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:45.595058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:45.624937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.628821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.630721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.633333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.635598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.636168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.715817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:45.841665 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b44e6a30-fa71-416a-9d7e-d249e86056ab tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 204 len: 0 microversion: 2.1 time: 0.251633 Oct 15 20:53:45.842390 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 864/1658] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:45 2022] DELETE /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 20:53:45.854282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:45.855250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:45.862624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.863307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.863926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:45.864786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:45.865525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:45.866107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:46.057664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:46.067439 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3fea2b4b-4cc5-49e4-962c-2a4929b79a91 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1466 microversion: 2.1 time: 0.220742 Oct 15 20:53:46.073844 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 865/1659] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:45 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1466 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:46.082320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:46.083255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:46.092586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:46.095109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:46.095867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:46.096840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:46.097818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:46.098492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:46.099129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:46.339947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:46.377119 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5b358ba-1a11-42ae-af22-885ea0c2f563 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.298506 Oct 15 20:53:46.379558 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 866/1660] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:46 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:46.390491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f509d4e3-6c7b-4072-9754-3b331cfa3616 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2cd280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:46.415649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:46.437517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4624833e-155f-4dcc-8a8d-a51d58ecbbb5 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1472 microversion: 2.1 time: 1.101312 Oct 15 20:53:46.440564 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 795/1661] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:45 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1472 bytes in 1105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:46.456038 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f509d4e3-6c7b-4072-9754-3b331cfa3616 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.075027 Oct 15 20:53:46.456963 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 867/1662] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:46 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:46.623583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-06af0e60-b089-4cce-9896-8b3ea54ea2c7 req-d3f6df40-093b-4172-8fa2-eb062ce54f6f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "name": "network-vif-plugged", "status": "completed", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:46.649551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-06af0e60-b089-4cce-9896-8b3ea54ea2c7 req-d3f6df40-093b-4172-8fa2-eb062ce54f6f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:46.657782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-06af0e60-b089-4cce-9896-8b3ea54ea2c7 req-d3f6df40-093b-4172-8fa2-eb062ce54f6f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:46.658741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-06af0e60-b089-4cce-9896-8b3ea54ea2c7 req-d3f6df40-093b-4172-8fa2-eb062ce54f6f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:46.682038 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-06af0e60-b089-4cce-9896-8b3ea54ea2c7 req-d3f6df40-093b-4172-8fa2-eb062ce54f6f service nova] Creating event network-vif-plugged:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch1-master-1 Oct 15 20:53:46.697957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-06af0e60-b089-4cce-9896-8b3ea54ea2c7 req-d3f6df40-093b-4172-8fa2-eb062ce54f6f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078361 Oct 15 20:53:46.698701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 796/1663] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:47.092018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:47.093678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:47.100862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:47.101607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:47.102387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:47.103267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:47.104796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:47.105707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:47.405816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:47.406538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:47.423727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:47.424358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:47.424572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:47.425319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:47.431098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:47.431977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:48.057791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:48.066944 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-867baa6c-7dc7-4e8f-be4a-ab63ba354e3e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1466 microversion: 2.1 time: 0.980078 Oct 15 20:53:48.078151 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 868/1664] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:47 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1466 bytes in 992 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:48.083189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-802164d0-caf4-4839-8034-e99aa0f0fdae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c9c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:48.169238 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-802164d0-caf4-4839-8034-e99aa0f0fdae tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.089353 Oct 15 20:53:48.181006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:48.189555 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4a67722-a72c-41d2-a0a1-fb899bfdad98 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.788760 Oct 15 20:53:48.190267 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 869/1665] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:48 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:48.195474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:48.196829 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 797/1666] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:47 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:48.197691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:48.207749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:48.208425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:48.209025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:48.210361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:48.210745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:48.211341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:48.470301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:48.479917 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-210a0218-a90f-4cbd-8dca-1ad96b47d986 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1472 microversion: 2.1 time: 0.288217 Oct 15 20:53:48.483610 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 870/1667] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:48 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1472 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:48.808906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9a4d91fb-e925-44d0-a863-112815d298f7 req-743c7d55-cd24-48b6-b01e-e84a679c66b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c10c829d-ef62-4d1c-98e9-422615414c5e", "tag": "7c2c4743-524f-46ad-b187-3b4184a1040b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:48.871811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9a4d91fb-e925-44d0-a863-112815d298f7 req-743c7d55-cd24-48b6-b01e-e84a679c66b9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:48.882992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9a4d91fb-e925-44d0-a863-112815d298f7 req-743c7d55-cd24-48b6-b01e-e84a679c66b9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:48.885121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9a4d91fb-e925-44d0-a863-112815d298f7 req-743c7d55-cd24-48b6-b01e-e84a679c66b9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:48.955303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9a4d91fb-e925-44d0-a863-112815d298f7 req-743c7d55-cd24-48b6-b01e-e84a679c66b9 service nova] Creating event network-changed:7c2c4743-524f-46ad-b187-3b4184a1040b for instance c10c829d-ef62-4d1c-98e9-422615414c5e on nch1-master-1 Oct 15 20:53:48.993927 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9a4d91fb-e925-44d0-a863-112815d298f7 req-743c7d55-cd24-48b6-b01e-e84a679c66b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.188470 Oct 15 20:53:48.994544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 798/1668] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:49.078642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-66bde1b4-648e-41f7-ab61-91e44442bd13 req-fcfd80cc-ef46-43b3-8e2c-4ae6fe9200e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d36ee2d3-b039-4811-959c-9cb6bf2b0363", "name": "network-vif-deleted", "tag": "38cbbc7f-0ebf-4d16-88a9-c59b3f079aed"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:49.102373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:49.103349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:49.109412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-66bde1b4-648e-41f7-ab61-91e44442bd13 req-fcfd80cc-ef46-43b3-8e2c-4ae6fe9200e5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.113457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-66bde1b4-648e-41f7-ab61-91e44442bd13 req-fcfd80cc-ef46-43b3-8e2c-4ae6fe9200e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.115454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.123944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-66bde1b4-648e-41f7-ab61-91e44442bd13 req-fcfd80cc-ef46-43b3-8e2c-4ae6fe9200e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.128631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.129258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.130247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.130798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.131301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.151650 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-66bde1b4-648e-41f7-ab61-91e44442bd13 req-fcfd80cc-ef46-43b3-8e2c-4ae6fe9200e5 service nova] Creating event network-vif-deleted:38cbbc7f-0ebf-4d16-88a9-c59b3f079aed for instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 on nch1-master-1 Oct 15 20:53:49.173470 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-66bde1b4-648e-41f7-ab61-91e44442bd13 req-fcfd80cc-ef46-43b3-8e2c-4ae6fe9200e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099435 Oct 15 20:53:49.179929 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 871/1669] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:49.200445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e3d1933c-d512-4736-9b12-f2368b797980 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2cd670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:49.328426 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e3d1933c-d512-4736-9b12-f2368b797980 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.147011 Oct 15 20:53:49.339120 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 872/1670] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:49 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:49.351331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:49.352526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:49.371263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.372358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.377314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.378613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.379391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.380057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.550562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:49.562088 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e111394-db47-4ae9-9378-8c30e3ab06bc tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 200 len: 1297 microversion: 2.1 time: 0.464108 Oct 15 20:53:49.573741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 799/1671] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:49 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 1297 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:53:49.580406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:49.580406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:49.613726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.614173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.614623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.620273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:49.620764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:49.621244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:49.678320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:49.692889 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f7318d57-219b-4a76-a618-ab4c7733772a tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.349908 Oct 15 20:53:49.705073 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 873/1672] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:49 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:50.344064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b2cc9bd0-69f6-41e5-ba11-6f1ed7963389 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2cd940>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:50.344747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:50.361299 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81db2085-3e6b-4a3a-8a88-7b0c3827ffd1 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 200 len: 1472 microversion: 2.1 time: 0.792907 Oct 15 20:53:50.362214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 800/1673] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:49 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 1472 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:50.462669 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b2cc9bd0-69f6-41e5-ba11-6f1ed7963389 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.122272 Oct 15 20:53:50.466584 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 874/1674] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:50 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:50.495841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachSCSIVolumeTestJSON-server-428714873", "imageRef": "91ba1ddf-88c3-4a2e-b0ac-65e5e6273090", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "f47f4457-aa7a-4880-a096-341f0b0b6c39"}], "security_groups": [{"name": "tempest-securitygroup--384989454"}], "key_name": "tempest-keypair-1640437525", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:50.593476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:50.594599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: d36ee2d3-b039-4811-959c-9cb6bf2b0363] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:50.611782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:50.613446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:50.615512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:50.625834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:50.626412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:50.626867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:50.686225 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] HTTP exception thrown: Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 could not be found. Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Request method failure captured: Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 HTTP/1.1 Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyrxLt9ZkRGBZEC262Xudbc4kffeptUv_lXweM-AbzOgTA--5IzJeazLLXDuegKkd17i1HT8bNeetv0IXo1FIUSXopfcoPvaNxD6gFon6e8zjH7XSGtH8Uh6ycdR8hHJFlbVtzOSITJGkIgH80sQdi_CjHEAN8I7EWSLXG9fMC0qEE Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:53:50.689895 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_94e961feee714587bb2ffe545d286c5c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-MigrationsAdminTest-389983916 Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-MigrationsAdminTest-389983916-project Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: e32a1c53b4be44188b07c768405961ed Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-MigrationsAdminTest-389983916-project Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 could not be found. Oct 15 20:53:50.691345 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'd36ee2d3-b039-4811-959c-9cb6bf2b0363'} Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 could not be found. Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:50.692336 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 could not be found. Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 could not be found. Oct 15 20:53:50.693730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:53:50.696921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Returning 404 to user: Instance d36ee2d3-b039-4811-959c-9cb6bf2b0363 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:50.699345 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c2ca2e0-c78d-4827-b0dd-26088b75b096 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363" status: 404 len: 111 microversion: 2.1 time: 0.115067 Oct 15 20:53:50.718826 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 875/1675] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:50 2022] GET /compute/v2.1/servers/d36ee2d3-b039-4811-959c-9cb6bf2b0363 => generated 111 bytes in 135 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:50.734139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:50.734650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:50.748830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:50.758583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:50.759704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:50.760772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:50.761480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:50.763692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:50.840573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] validate_networks() for [('f47f4457-aa7a-4880-a096-341f0b0b6c39', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:53:50.859758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:51.088459 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3a3582f8-2110-438e-b9fe-bd3bfa76e15b tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 204 len: 0 microversion: 2.1 time: 0.363407 Oct 15 20:53:51.089509 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 876/1676] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:50 2022] DELETE /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:51.100055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:51.101666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:51.119564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.120476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.121256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.125169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.126017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.126670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.469180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:51.491292 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fff8cae7-70d5-46b7-b77f-d684bc8c38b5 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.400113 Oct 15 20:53:51.493662 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 877/1677] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:51 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:51.499948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:51.501431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:51.525404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.526212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.527183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.528934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.529683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.530278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.574577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:53:51.849485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:53:51.850972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:53:51.852010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:53:51.853417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:53:51.854017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:53:51.900145 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d52fc7b-986b-4c6d-82b6-1531856595e6 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1858 microversion: 2.1 time: 0.404722 Oct 15 20:53:51.900897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.900897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.901680 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 878/1678] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:51 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1858 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:51.903510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.906595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.907093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:51.907882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:51.908198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.910734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.924986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.925516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.925953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.926660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:51.927111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:51.927534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:51.967744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Getting quotas for project e67382792a7d4ccf8100febea9052e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:53:51.984983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Getting quotas for user 999bc03f52824573bb3c8bdc6a1179cf and project e67382792a7d4ccf8100febea9052e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:53:52.041430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:53:52.046778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='91ba1ddf-88c3-4a2e-b0ac-65e5e6273090',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:53:52.158274 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-265e09c0-acc1-498b-9176-ffd8fe372528 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.668470 Oct 15 20:53:52.160217 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 801/1679] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:53:50 2022] POST /compute/v2.1/servers => generated 405 bytes in 1671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:53:52.165994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6d6c0417-1707-4aa6-af4f-99f01a21d94a req-b1ff9d53-c36b-4742-ab41-6bf6cdec5785 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d0f01cf-e928-45bc-9c9c-3f34fd9d0945", "name": "network-vif-deleted", "tag": "4144980c-d9d6-49c0-b7b0-6e5557bc8c1d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:52.178560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d6c0417-1707-4aa6-af4f-99f01a21d94a req-b1ff9d53-c36b-4742-ab41-6bf6cdec5785 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:52.179426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d6c0417-1707-4aa6-af4f-99f01a21d94a req-b1ff9d53-c36b-4742-ab41-6bf6cdec5785 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:52.180031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d6c0417-1707-4aa6-af4f-99f01a21d94a req-b1ff9d53-c36b-4742-ab41-6bf6cdec5785 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:52.194232 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6d6c0417-1707-4aa6-af4f-99f01a21d94a req-b1ff9d53-c36b-4742-ab41-6bf6cdec5785 service nova] Creating event network-vif-deleted:4144980c-d9d6-49c0-b7b0-6e5557bc8c1d for instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 on nch2-master-1 Oct 15 20:53:52.202143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6d6c0417-1707-4aa6-af4f-99f01a21d94a req-b1ff9d53-c36b-4742-ab41-6bf6cdec5785 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039666 Oct 15 20:53:52.203258 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 802/1680] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:52.209668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:52.210382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:52.218980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:52.220008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:52.220697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:52.221800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:52.222631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:52.223285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:52.239895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 could not be found. Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 HTTP/1.1 Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:53:52.241950 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 could not be found. Oct 15 20:53:52.243375 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4d0f01cf-e928-45bc-9c9c-3f34fd9d0945'} Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 could not be found. Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:52.244320 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 could not be found. Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 could not be found. Oct 15 20:53:52.245655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:52.250097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance 4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:52.251372 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4cd2012-8f3c-4436-ac61-e14295cad10d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945" status: 404 len: 111 microversion: 2.1 time: 0.046818 Oct 15 20:53:52.252451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 803/1681] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:52 2022] GET /compute/v2.1/servers/4d0f01cf-e928-45bc-9c9c-3f34fd9d0945 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:52.257355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13c75d3f-7964-4f52-b831-1999ca7ee05f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75846e820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:52.324879 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13c75d3f-7964-4f52-b831-1999ca7ee05f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.070839 Oct 15 20:53:52.351872 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 804/1682] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:52 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:52.366297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:52.371269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:52.391606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:52.395302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:52.395764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:52.405741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:52.406799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:52.407268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:52.478074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lazy-loading 'flavor' on Instance uuid bec91401-323d-4a36-88b7-88fe24aa813f {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:53:52.516085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Going to try to live migrate instance to nch1-master-1 {{(pid=138688) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:53:52.939985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:52.959482 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea8d9865-70d3-4827-91be-4a6a6c0f06ff tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1535 microversion: 2.1 time: 1.056468 Oct 15 20:53:52.964321 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 879/1683] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:51 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1535 bytes in 1062 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:52.983574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f4c56147-1301-486e-9c0a-c0ce50e55240 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:52.993362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f4c56147-1301-486e-9c0a-c0ce50e55240 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:53.205690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f4c56147-1301-486e-9c0a-c0ce50e55240 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:53.216959 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f4c56147-1301-486e-9c0a-c0ce50e55240 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1245 microversion: 2.1 time: 0.251439 Oct 15 20:53:53.221231 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 880/1684] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:52 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1245 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:53.226464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:53.234519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:53.256382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:53.259235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:53.260887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:53.263131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:53.268218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:53.268896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:53.350245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:53.460663 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-10853a39-b542-4245-9acd-c155817b42db tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805" status: 204 len: 0 microversion: 2.1 time: 0.237577 Oct 15 20:53:53.467165 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 881/1685] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:53 2022] DELETE /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:53.477480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:53.477480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:53.486259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:53.486755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:53.487832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:53.488565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:53.489050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:53.489661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:53.918184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:53.928258 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87984ecd-af85-4f83-b005-cabf08fb2f5f tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.459076 Oct 15 20:53:53.929031 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 882/1686] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:53 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:53.958087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b516743d-40c0-4d48-b50d-50cace635bf3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f3223a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:54.086004 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b516743d-40c0-4d48-b50d-50cace635bf3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1453 microversion: 2.1 time: 0.146446 Oct 15 20:53:54.086703 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 883/1687] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:53 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1453 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:54.095425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:54.098683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:54.116694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.117405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.119713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.120994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.124490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.125341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.213442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:53:54.451465 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-96dad11c-381c-463e-811a-97c7239a2b61 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 204 len: 0 microversion: 2.1 time: 0.362512 Oct 15 20:53:54.457448 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 884/1688] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:53:54 2022] DELETE /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:53:54.472139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:54.473493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:54.487643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.488150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.488625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.489586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.490044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.490481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.610697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:54.620593 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-50f271eb-8f5a-481f-a34d-5b3c6591f5f2 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 200 len: 1493 microversion: 2.1 time: 0.160100 Oct 15 20:53:54.621418 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 885/1689] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:54 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 1493 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:54.626006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:54.626616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:54.634581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.635432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.645758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.647919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.648416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.648855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.885992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:54.904846 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-59124348-48b7-459b-b17e-1a07dc81c07c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1297 microversion: 2.1 time: 0.280616 Oct 15 20:53:54.907891 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 886/1690] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:54 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1297 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:54.914649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:54.915559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:54.927377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.928900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.929804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:54.931050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:54.931746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:54.932340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:55.250412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:55.265723 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0dc12627-6205-477e-be31-d6a30bca190d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805" status: 200 len: 1474 microversion: 2.1 time: 0.355216 Oct 15 20:53:55.266335 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 887/1691] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:54 2022] GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805 => generated 1474 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:55.290753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-53aed4e9-848d-4867-bb71-2ff0aef8812a req-31d30f68-04f0-48c5-86bf-6f059ee7724f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f9dc34bb-2691-4bb1-90c9-5a07b21df3f3", "name": "network-vif-deleted", "tag": "5151f426-4525-4c7a-9807-8f23b0714eb8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:53:55.312558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-53aed4e9-848d-4867-bb71-2ff0aef8812a req-31d30f68-04f0-48c5-86bf-6f059ee7724f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:55.313446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-53aed4e9-848d-4867-bb71-2ff0aef8812a req-31d30f68-04f0-48c5-86bf-6f059ee7724f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:55.314054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-53aed4e9-848d-4867-bb71-2ff0aef8812a req-31d30f68-04f0-48c5-86bf-6f059ee7724f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:55.329832 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-53aed4e9-848d-4867-bb71-2ff0aef8812a req-31d30f68-04f0-48c5-86bf-6f059ee7724f service nova] Creating event network-vif-deleted:5151f426-4525-4c7a-9807-8f23b0714eb8 for instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 on nch1-master-1 Oct 15 20:53:55.336509 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-53aed4e9-848d-4867-bb71-2ff0aef8812a req-31d30f68-04f0-48c5-86bf-6f059ee7724f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066405 Oct 15 20:53:55.337810 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 888/1692] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:53:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:53:55.344959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:55.345917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:55.353263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:55.354001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:55.354602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:55.355496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:55.356246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:55.356931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:57.919997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:53:57.939411 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d0580590-edfb-4f90-80c5-11f0c1249456 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 2.600168 Oct 15 20:53:57.940204 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 889/1693] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:55 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 2602 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:53:57.953083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa8b8a57-8543-418e-83f0-a9f55028d1f4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f27e5e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:59.249860 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7bdba03-2bd3-47bd-a91c-40bc2a79d585 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/action" status: 202 len: 0 microversion: 2.1 time: 6.903219 Oct 15 20:53:59.254765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 805/1694] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:53:52 2022] POST /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/action => generated 0 bytes in 6909 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:53:59.261413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:59.263340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: f9dc34bb-2691-4bb1-90c9-5a07b21df3f3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:59.297817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:59.302423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:59.304511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:59.312684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:59.314604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:59.317711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:59.364701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] HTTP exception thrown: Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 could not be found. Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Request method failure captured: Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 HTTP/1.1 Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyrxLt9ZkRGBZEC262Xudbc4kffeptUv_lXweM-AbzOgTA--5IzJeazLLXDuegKkd17i1HT8bNeetv0IXo1FIUSXopfcoPvaNxD6gFon6e8zjH7XSGtH8Uh6ycdR8hHJFlbVtzOSITJGkIgH80sQdi_CjHEAN8I7EWSLXG9fMC0qEE Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:53:59.367962 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_94e961feee714587bb2ffe545d286c5c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MigrationsAdminTest-389983916 Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MigrationsAdminTest-389983916-project Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e32a1c53b4be44188b07c768405961ed Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MigrationsAdminTest-389983916-project Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 could not be found. Oct 15 20:53:59.369664 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f9dc34bb-2691-4bb1-90c9-5a07b21df3f3'} Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 could not be found. Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:53:59.370501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 could not be found. Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 could not be found. Oct 15 20:53:59.372739 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:53:59.376544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Returning 404 to user: Instance f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:53:59.376544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca58d676-de97-4d31-98a0-b138bb1b185c tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3" status: 404 len: 111 microversion: 2.1 time: 0.119396 Oct 15 20:53:59.389525 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 806/1695] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:53:59 2022] GET /compute/v2.1/servers/f9dc34bb-2691-4bb1-90c9-5a07b21df3f3 => generated 111 bytes in 134 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:53:59.412693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:53:59.426366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:53:59.452345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:59.460299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:59.477860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:53:59.481421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:53:59.486242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:53:59.487766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:00.481771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid c05e4f55-245d-45f2-8567-e665ccbca061 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:00.520504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78ccb9d0-46ec-4e0f-bccf-9f8c4648f17e tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 2345 microversion: 2.25 time: 1.129128 Oct 15 20:54:00.521500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 807/1696] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:53:59 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 2345 bytes in 1131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:00.530857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:00.535740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:00.555971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:00.558248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:00.559596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:00.563377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:00.564105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:00.565845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:01.548571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:01.579293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b03a687f-4a25-49f1-9f75-5988baf33737 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 1.050764 Oct 15 20:54:01.585265 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 808/1697] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:00 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 1063 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:01.610308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:01.613534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: c683117d-d7b3-4e84-beb1-c7ff7c809805] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:01.666655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:01.667264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:01.667781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:01.671834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:01.679172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:01.682348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:01.755848 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance c683117d-d7b3-4e84-beb1-c7ff7c809805 could not be found. Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805 HTTP/1.1 Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:54:01.763819 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c683117d-d7b3-4e84-beb1-c7ff7c809805 could not be found. Oct 15 20:54:01.766246 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c683117d-d7b3-4e84-beb1-c7ff7c809805'} Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c683117d-d7b3-4e84-beb1-c7ff7c809805 could not be found. Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:01.767334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c683117d-d7b3-4e84-beb1-c7ff7c809805 could not be found. Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c683117d-d7b3-4e84-beb1-c7ff7c809805 could not be found. Oct 15 20:54:01.769312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:01.771759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance c683117d-d7b3-4e84-beb1-c7ff7c809805 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:01.773558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fa231c9c-f063-41c7-8ed7-1c7f0c3ccf21 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805" status: 404 len: 111 microversion: 2.1 time: 0.183896 Oct 15 20:54:01.773558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 809/1698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:01 2022] GET /compute/v2.1/servers/c683117d-d7b3-4e84-beb1-c7ff7c809805 => generated 111 bytes in 186 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:01.790676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8dcf718f-01cd-4b78-8846-e9213bb205f7 req-828e1c9f-6ae4-49df-afe0-8b6158b7bee9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c683117d-d7b3-4e84-beb1-c7ff7c809805", "name": "network-vif-deleted", "tag": "146333fb-b86a-486a-91fe-a1828bdef9d9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:01.826844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8dcf718f-01cd-4b78-8846-e9213bb205f7 req-828e1c9f-6ae4-49df-afe0-8b6158b7bee9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:01.827323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8dcf718f-01cd-4b78-8846-e9213bb205f7 req-828e1c9f-6ae4-49df-afe0-8b6158b7bee9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:01.828127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8dcf718f-01cd-4b78-8846-e9213bb205f7 req-828e1c9f-6ae4-49df-afe0-8b6158b7bee9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:01.862396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8dcf718f-01cd-4b78-8846-e9213bb205f7 req-828e1c9f-6ae4-49df-afe0-8b6158b7bee9 service nova] Creating event network-vif-deleted:146333fb-b86a-486a-91fe-a1828bdef9d9 for instance c683117d-d7b3-4e84-beb1-c7ff7c809805 on nch1-master-1 Oct 15 20:54:01.873086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8dcf718f-01cd-4b78-8846-e9213bb205f7 req-828e1c9f-6ae4-49df-afe0-8b6158b7bee9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088427 Oct 15 20:54:01.876389 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 810/1699] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:01.888661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:01.889872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:01.910882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:01.911742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:01.912918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:01.913922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:01.914615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:01.915280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:01.966865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:01.970905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:02.004841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=16 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:02.527522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:02.538764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24acb3c8-f9dc-44a7-8540-00021ca06ba8 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.660752 Oct 15 20:54:02.556526 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 811/1700] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:01 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 672 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:02.566583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:02.570791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:02.600909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:02.601470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:02.608142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:02.609006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:02.609536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:02.615494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:03.212655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:03.814224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:03.826943 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6a16dbf-8ad8-441a-832a-70b052160c01 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 1.275260 Oct 15 20:54:03.827546 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 812/1701] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:02 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 1277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:03.846196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a28d83d7-bc3b-4173-bd2b-75d9902e5011 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '.version_select at 0x7fe7584d1af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:03.908219 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a28d83d7-bc3b-4173-bd2b-75d9902e5011 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-253147738" status: 202 len: 0 microversion: 2.1 time: 0.066524 Oct 15 20:54:03.909608 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 813/1702] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 20:54:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-253147738 => generated 0 bytes in 69 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:03.920127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-add36c9d-f3e4-4936-8211-212990e397d4 req-e673d5ea-5787-40eb-91a6-4bd55bf26bf5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e801c019-e5a3-4208-bed9-ca567e0996fb", "name": "network-vif-unplugged", "status": "completed", "tag": "9e53d8b9-9a15-4432-8b47-46a0152c1409"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:03.948212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-add36c9d-f3e4-4936-8211-212990e397d4 req-e673d5ea-5787-40eb-91a6-4bd55bf26bf5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:03.949000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-add36c9d-f3e4-4936-8211-212990e397d4 req-e673d5ea-5787-40eb-91a6-4bd55bf26bf5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:03.952246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-add36c9d-f3e4-4936-8211-212990e397d4 req-e673d5ea-5787-40eb-91a6-4bd55bf26bf5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:03.967406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-add36c9d-f3e4-4936-8211-212990e397d4 req-e673d5ea-5787-40eb-91a6-4bd55bf26bf5 service nova] Creating event network-vif-unplugged:9e53d8b9-9a15-4432-8b47-46a0152c1409 for instance e801c019-e5a3-4208-bed9-ca567e0996fb on nch2-master-1 Oct 15 20:54:03.974338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-add36c9d-f3e4-4936-8211-212990e397d4 req-e673d5ea-5787-40eb-91a6-4bd55bf26bf5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.062295 Oct 15 20:54:03.975271 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 814/1703] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:03 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:03.984734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:03.990101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:03.992369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:03.993025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:03.994230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:03.994861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:03.995524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.044080 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83ebebd2-5a79-45e2-851f-f10b0f4c5422 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.063683 Oct 15 20:54:04.046221 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 815/1704] 10.222.0.16 () {60 vars in 1217 bytes} [Sat Oct 15 20:54:03 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:54:04.091385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-470c2251-95af-44db-a063-402f8e2756a3 req-aee396ce-a767-4f9c-a175-e7a60dc92c75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e801c019-e5a3-4208-bed9-ca567e0996fb", "name": "network-vif-deleted", "tag": "9e53d8b9-9a15-4432-8b47-46a0152c1409"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:04.125426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-470c2251-95af-44db-a063-402f8e2756a3 req-aee396ce-a767-4f9c-a175-e7a60dc92c75 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.127359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-470c2251-95af-44db-a063-402f8e2756a3 req-aee396ce-a767-4f9c-a175-e7a60dc92c75 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.128002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-470c2251-95af-44db-a063-402f8e2756a3 req-aee396ce-a767-4f9c-a175-e7a60dc92c75 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.179452 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-470c2251-95af-44db-a063-402f8e2756a3 req-aee396ce-a767-4f9c-a175-e7a60dc92c75 service nova] Creating event network-vif-deleted:9e53d8b9-9a15-4432-8b47-46a0152c1409 for instance e801c019-e5a3-4208-bed9-ca567e0996fb on nch2-master-1 Oct 15 20:54:04.194532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-470c2251-95af-44db-a063-402f8e2756a3 req-aee396ce-a767-4f9c-a175-e7a60dc92c75 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.144854 Oct 15 20:54:04.197468 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 816/1705] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:04.225969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:04.227078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: e801c019-e5a3-4208-bed9-ca567e0996fb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:04.245514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.247565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.248811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.252315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.253016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.253737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.298351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance e801c019-e5a3-4208-bed9-ca567e0996fb could not be found. Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb HTTP/1.1 Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:54:04.301636 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e801c019-e5a3-4208-bed9-ca567e0996fb could not be found. Oct 15 20:54:04.303804 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e801c019-e5a3-4208-bed9-ca567e0996fb'} Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e801c019-e5a3-4208-bed9-ca567e0996fb could not be found. Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:04.304602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e801c019-e5a3-4208-bed9-ca567e0996fb could not be found. Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e801c019-e5a3-4208-bed9-ca567e0996fb could not be found. Oct 15 20:54:04.307312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:04.312548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance e801c019-e5a3-4208-bed9-ca567e0996fb could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:04.315261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c4d23e25-0b9a-4fc5-ae34-32ec496ea7f0 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb" status: 404 len: 111 microversion: 2.1 time: 0.114115 Oct 15 20:54:04.317358 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 817/1706] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:04 2022] GET /compute/v2.1/servers/e801c019-e5a3-4208-bed9-ca567e0996fb => generated 111 bytes in 119 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:04.334160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:04.336747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:04.357846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.358350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.362407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.366004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.366515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.366963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.380001 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa8b8a57-8543-418e-83f0-a9f55028d1f4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1462 microversion: 2.1 time: 6.435094 Oct 15 20:54:04.395316 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 890/1707] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:53:57 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1462 bytes in 6451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:04.408690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:04.410718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:04.427968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.428895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.429581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.430524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.431228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.431874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.825676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:04.843391 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-189a8e95-a3cf-408b-ac39-af01d2e439ff tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.446121 Oct 15 20:54:04.858403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:04.865476 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 891/1708] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:04 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:04.872188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ff71c513-3cfa-4dab-9562-7f50d7a183c5 req-5407042b-0f57-4e89-9d02-2e54700183d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82e74d13-2be4-47d0-93fa-fb24eb81c212", "tag": "328d28c6-9ece-4b0b-b5f5-e870e5285550"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:04.885235 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37f719ef-2f25-4063-9a72-8ba2cd3ad715 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.566196 Oct 15 20:54:04.894073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 818/1709] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:04 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:04.896399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ff71c513-3cfa-4dab-9562-7f50d7a183c5 req-5407042b-0f57-4e89-9d02-2e54700183d2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:04.897266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ff71c513-3cfa-4dab-9562-7f50d7a183c5 req-5407042b-0f57-4e89-9d02-2e54700183d2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:04.897995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ff71c513-3cfa-4dab-9562-7f50d7a183c5 req-5407042b-0f57-4e89-9d02-2e54700183d2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:04.912069 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ff71c513-3cfa-4dab-9562-7f50d7a183c5 req-5407042b-0f57-4e89-9d02-2e54700183d2 service nova] Creating event network-changed:328d28c6-9ece-4b0b-b5f5-e870e5285550 for instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 on nch2-master-1 Oct 15 20:54:04.919155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-985a1f7f-35ee-4bb5-9cd8-3b0bb3f26e20 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:04.920736 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ff71c513-3cfa-4dab-9562-7f50d7a183c5 req-5407042b-0f57-4e89-9d02-2e54700183d2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052935 Oct 15 20:54:04.921921 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 892/1710] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:04.928275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a0f44b75-4d0f-4db0-9496-0bba10706764 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f27e430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:04.999611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=4,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:05.034547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-985a1f7f-35ee-4bb5-9cd8-3b0bb3f26e20 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/dfac27d3-d366-4ce8-a977-90fbd824648a" status: 204 len: 0 microversion: 2.1 time: 0.137981 Oct 15 20:54:05.034916 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 819/1711] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 20:54:04 2022] DELETE /compute/v2.1/os-server-groups/dfac27d3-d366-4ce8-a977-90fbd824648a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:05.047431 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a0f44b75-4d0f-4db0-9496-0bba10706764 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 200 len: 1462 microversion: 2.1 time: 0.123903 Oct 15 20:54:05.048021 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 893/1712] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:04 2022] GET /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 1462 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:05.065906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:05.068440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:05.069097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:05.070399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:05.083780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:05.085836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:05.087531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:05.088971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:05.089668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:05.090448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:05.091449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:05.093505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:05.093780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:05.094473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:05.095469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:05.095896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:05.193576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:05.498229 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c1328dfa-1398-4ba8-b98e-247fd3f1ce25 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 204 len: 0 microversion: 2.1 time: 0.447877 Oct 15 20:54:05.500636 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 894/1713] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:05 2022] DELETE /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:05.533574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:05.534544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:05.558893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:05.561768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:05.562479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:05.564010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:05.564010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:05.569975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:05.681378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:05.975192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:05.995420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:05.995901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0eaebff5-0d5a-4dfa-a38f-b0c11b2d4673 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.945182 Oct 15 20:54:05.996492 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 820/1714] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:05 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 961 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:06.013076 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-13a546a0-4cc4-4256-b643-d294fa620d5b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1467 microversion: 2.1 time: 0.491231 Oct 15 20:54:06.026460 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 895/1715] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:05 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1467 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:06.039287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:06.040028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:06.044344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:06.053559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.053838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:06.054658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.055321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.056888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.057630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.058262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.069849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.070338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.070774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.071485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.071940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.083440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.139890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:06.258999 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4cb9ec3-4493-48c7-b8b5-c8ca07b16e5d tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 204 len: 0 microversion: 2.1 time: 0.231989 Oct 15 20:54:06.274713 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 821/1716] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:06 2022] DELETE /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:06.286844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:06.287987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:06.297467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.298375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.298885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.299812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.300342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.301094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.430119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:06.438635 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7551ff10-8dc9-4263-88b0-4eb61ab99310 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.408915 Oct 15 20:54:06.439231 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 896/1717] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:06 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:06.448245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c10c829d-ef62-4d1c-98e9-422615414c5e", "name": "network-vif-plugged", "status": "completed", "tag": "7c2c4743-524f-46ad-b187-3b4184a1040b"}, {"server_uuid": "c10c829d-ef62-4d1c-98e9-422615414c5e", "name": "network-vif-plugged", "status": "completed", "tag": "2557963e-5bb5-430b-9efd-87827090ab6c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:06.458465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.458983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.459439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.473818 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] Creating event network-vif-plugged:7c2c4743-524f-46ad-b187-3b4184a1040b for instance c10c829d-ef62-4d1c-98e9-422615414c5e on nch1-master-1 Oct 15 20:54:06.474362 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] Creating event network-vif-plugged:2557963e-5bb5-430b-9efd-87827090ab6c for instance c10c829d-ef62-4d1c-98e9-422615414c5e on nch1-master-1 Oct 15 20:54:06.482124 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-41e69b40-c467-43ca-90b7-de2e5b6039cd req-225d29d8-1c67-4982-84aa-c53a5c14f76f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.041138 Oct 15 20:54:06.483221 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 897/1718] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:06 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:06.488396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c9fc8ea5-4c70-43a4-83f8-28d2a11abc43 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2fa5e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:06.524630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:06.536698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-084836be-9bd1-4f0b-b9e4-4375dbc3d20f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.255212 Oct 15 20:54:06.541612 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 822/1719] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:06 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:06.547916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:06.548827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:06.564002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.564829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.565490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.566413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.567051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.567645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.791803 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c9fc8ea5-4c70-43a4-83f8-28d2a11abc43 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?changes-since=2022-10-15T20%3A51%3A22Z" status: 200 len: 2803 microversion: 2.1 time: 0.307070 Oct 15 20:54:06.793446 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 898/1720] 10.222.0.16 () {58 vars in 1199 bytes} [Sat Oct 15 20:54:06 2022] GET /compute/v2.1/images?changes-since=2022-10-15T20%3A51%3A22Z => generated 2803 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:06.806137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:06.807212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:06.818048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.818878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.820427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.821571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:06.822221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:06.822812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:06.972876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:06.989566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:07.006409 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b2d1b89-a41a-4e3e-a746-ce3b38ccf41b tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 200 len: 1474 microversion: 2.1 time: 0.461955 Oct 15 20:54:07.007624 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 823/1721] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:06 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 1474 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:07.023056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a02d2cad-c453-42f6-aabc-ca1baa1d0574", "device": "/dev/xvdb"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:07.034731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:07.055483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:07.057486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:07.058841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:07.061819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:07.062486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:07.064417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:07.172027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=5 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:07.184085 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b3aef3ec-ebb0-47ac-8902-953bf9e81958 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 204 len: 0 microversion: 2.1 time: 0.387773 Oct 15 20:54:07.188218 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 899/1722] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:06 2022] DELETE /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:07.511296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98372d69-2e0c-451a-ab3b-7123a720892e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Action: 'create', calling method: .version_select at 0x7fe74f9c9c10>, body: {"keypair": {"name": "tempest-keypair-1573082579", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCPEj75Ddl5ew5dH7FsCHm02C0mIjHgL/XAc74FE9V8sILmjUtWXDvMWVbYTSXmURie5hTo1uONwUxc3UkVrJBE7GxhDn8C5pu1Bc5nr1OfI375T1K/mMaZbKcnYuszq8g=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:07.536969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-98372d69-2e0c-451a-ab3b-7123a720892e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Getting quotas for project e08b85b4f6cf47a88fe24ead8e6d45e9. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:07.550867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-98372d69-2e0c-451a-ab3b-7123a720892e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Getting quotas for user ec8dcd7615cc43b3866034d5af34a9a1 and project e08b85b4f6cf47a88fe24ead8e6d45e9. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:07.628328 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98372d69-2e0c-451a-ab3b-7123a720892e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.436524 Oct 15 20:54:07.631333 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 900/1723] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:07 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:07.668435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ddd3872-b079-4c68-a2cf-b3bd40e79a10 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f27e1f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:07.787884 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ddd3872-b079-4c68-a2cf-b3bd40e79a10 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?name=tempest-ListImageFiltersTestJSON-image-54329338" status: 200 len: 483 microversion: 2.1 time: 0.153415 Oct 15 20:54:07.789184 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 901/1724] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Oct 15 20:54:07 2022] GET /compute/v2.1/images?name=tempest-ListImageFiltersTestJSON-image-54329338 => generated 483 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:07.796107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:07.797894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:07.806597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:07.807524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:07.807934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:07.808652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:07.809287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:07.810369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.055420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:08.073210 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fa0b2432-0fe5-49c8-8685-52d82dc5a52b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.282666 Oct 15 20:54:08.073793 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 902/1725] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:07 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:08.108047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:08.108724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:08.126055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:08.126636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:08.127041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.127835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:08.128410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:08.128999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.344696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:08.377146 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5c7f0bc3-05e8-4b2d-9194-02295dd97f09 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1468 microversion: 2.1 time: 0.273011 Oct 15 20:54:08.379028 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 903/1726] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:08 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1468 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:08.393135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:08.398667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:08.413448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:08.415742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:08.417259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.436110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:08.438553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:08.440811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.658319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:08.670374 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-750589fd-537e-4e1e-927a-e580f1f1991b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1333 microversion: 2.1 time: 0.288447 Oct 15 20:54:08.671264 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 904/1727] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:08 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1333 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:08.676579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:08.677630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:08.705148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:08.706017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:08.706495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.707224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:08.707679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:08.708106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:08.924400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:08.940307 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f0152935-98a9-4cb3-be62-09bfb16330f0 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.267769 Oct 15 20:54:08.941123 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 905/1728] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:08 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:08.950508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f6636e4-d780-480b-9b59-29775db14593 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f322c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.071120 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f6636e4-d780-480b-9b59-29775db14593 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?server=9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 958 microversion: 2.1 time: 0.123509 Oct 15 20:54:09.079933 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 906/1729] 10.222.0.16 () {58 vars in 1209 bytes} [Sat Oct 15 20:54:08 2022] GET /compute/v2.1/images?server=9be47b55-0549-4005-b37e-52219a6e7c11 => generated 958 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:09.085533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.087074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:09.096431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.096938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.097398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.098318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.098936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.100020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.150697 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e373edf6-b5f3-4ac4-b88a-a1bbfc27940e tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "POST /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments" status: 200 len: 194 microversion: 2.25 time: 2.140716 Oct 15 20:54:09.171576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 824/1730] 10.222.0.16 () {62 vars in 1364 bytes} [Sat Oct 15 20:54:07 2022] POST /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments => generated 194 bytes in 2163 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:54:09.185629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-48ce3e1e-672a-4ffa-a7d9-57429880d59e req-499f8817-695e-4e7f-9623-91426930eb9c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "name": "network-vif-unplugged", "status": "completed", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:09.222709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-48ce3e1e-672a-4ffa-a7d9-57429880d59e req-499f8817-695e-4e7f-9623-91426930eb9c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.224789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-48ce3e1e-672a-4ffa-a7d9-57429880d59e req-499f8817-695e-4e7f-9623-91426930eb9c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.228427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-48ce3e1e-672a-4ffa-a7d9-57429880d59e req-499f8817-695e-4e7f-9623-91426930eb9c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.272046 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-48ce3e1e-672a-4ffa-a7d9-57429880d59e req-499f8817-695e-4e7f-9623-91426930eb9c service nova] Creating event network-vif-unplugged:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:54:09.280511 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-48ce3e1e-672a-4ffa-a7d9-57429880d59e req-499f8817-695e-4e7f-9623-91426930eb9c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.104792 Oct 15 20:54:09.281716 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 825/1731] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:09 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:09.288437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-188c20da-74f6-4b53-8314-f144edd84e23 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe7584dbb80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.362457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-188c20da-74f6-4b53-8314-f144edd84e23 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 485 microversion: 2.1 time: 0.079195 Oct 15 20:54:09.364153 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 826/1732] 10.222.0.16 () {58 vars in 1317 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 485 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:09.368511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:09.369727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.370570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:09.378849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.379344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.379982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.380992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.381741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.382025 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7b22a4d4-46ea-4599-acfe-cd31b9f7ea95 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 200 len: 1474 microversion: 2.1 time: 0.300321 Oct 15 20:54:09.382580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.393074 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 907/1733] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 1474 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:09.393074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70844b98-41f0-4261-b6bd-30d853f5d62b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2faf70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.502235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70844b98-41f0-4261-b6bd-30d853f5d62b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 485 microversion: 2.1 time: 0.116909 Oct 15 20:54:09.513207 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 908/1734] 10.222.0.16 () {58 vars in 1303 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 485 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:09.519879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.521956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:09.532956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.534484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.535106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.536413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.537477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.538443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.738074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:09.750889 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-08fc02b5-1fac-473b-acea-3c4240af9fc1 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1468 microversion: 2.1 time: 0.245468 Oct 15 20:54:09.751959 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 909/1735] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1468 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:09.758411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-397da5ac-1211-4a48-b410-d5e6cc4edd05 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f3223a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.818979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:09.836378 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8956fe90-b948-4064-89ab-f775be6e9e25 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.470706 Oct 15 20:54:09.837340 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 827/1736] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:09.842903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.843520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:09.862476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.863259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.863871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.864781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:09.865430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:09.866019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:09.944767 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-397da5ac-1211-4a48-b410-d5e6cc4edd05 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?status=ACTIVE" status: 200 len: 3707 microversion: 2.1 time: 0.190607 Oct 15 20:54:09.947512 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 910/1737] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/images?status=ACTIVE => generated 3707 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:09.978383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:09.980432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:10.009590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:10.011912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:10.013662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:10.016609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:10.020778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:10.021704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:10.351418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:10.365633 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d919598a-5337-47e1-8d7d-edd899caeb39 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.401241 Oct 15 20:54:10.381773 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 911/1738] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:10.389168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7cbeb95-f759-4a30-924b-381df6f3342c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f9c91f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:10.502799 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d7cbeb95-f759-4a30-924b-381df6f3342c tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?type=snapshot" status: 200 len: 1431 microversion: 2.1 time: 0.117790 Oct 15 20:54:10.504824 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 912/1739] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:54:10 2022] GET /compute/v2.1/images?type=snapshot => generated 1431 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:10.510558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:10.511439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:10.522159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:10.523033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:10.523726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:10.526915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:10.527425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:10.527850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:11.050717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:11.061573 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2bc66c93-e64e-4818-a895-8ef7ab6e3304 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 200 len: 1474 microversion: 2.1 time: 0.555241 Oct 15 20:54:11.080944 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 913/1740] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:10 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 1474 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:11.093936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f0272437-2c65-4969-b136-58a934b1d463 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f382940>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:11.216148 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f0272437-2c65-4969-b136-58a934b1d463 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images?limit=1" status: 200 len: 622 microversion: 2.1 time: 0.129548 Oct 15 20:54:11.221005 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 914/1741] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:11 2022] GET /compute/v2.1/images?limit=1 => generated 622 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:11.238203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:11.242902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:11.265873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:11.270362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:11.271598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:11.274335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:11.276016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:11.278962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:11.286685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:11.297471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7fbc8698-149e-474a-9d8b-f82223bf3ef6 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1667 microversion: 2.1 time: 1.458916 Oct 15 20:54:11.303844 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 828/1742] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:09 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1667 bytes in 1466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:11.317579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:11.330637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:11.349670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:11.354130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:11.355579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:11.357306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:11.360789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:11.362869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:11.506141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:11.520064 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a10b401f-2296-40aa-bb0d-3b90e3b6338e tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1426 microversion: 2.1 time: 0.296522 Oct 15 20:54:11.548782 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 915/1743] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:11 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1426 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:11.571793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4a9c28f6-b681-44b1-bca6-6eb142f9f78f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f382d30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:11.853249 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4a9c28f6-b681-44b1-bca6-6eb142f9f78f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?changes-since=2022-10-15T20%3A51%3A17Z" status: 200 len: 7016 microversion: 2.1 time: 0.295252 Oct 15 20:54:11.858646 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 916/1744] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Oct 15 20:54:11 2022] GET /compute/v2.1/images/detail?changes-since=2022-10-15T20%3A51%3A17Z => generated 7016 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:11.867411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:11.867978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:11.878922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:11.879566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:11.880006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:11.889773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:11.890293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:11.891922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:12.020810 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef41cfe3-d27a-48fa-b253-453a0688fb16 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e/ips" status: 200 len: 178 microversion: 2.1 time: 0.156050 Oct 15 20:54:12.025810 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 917/1745] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 20:54:11 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e/ips => generated 178 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:12.033137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:12.037098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:12.076909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:12.077602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:12.086404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:12.087566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:12.088618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:12.089339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:12.413110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:12.428586 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4f5a564-7914-4385-abdd-072906c41904 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.400171 Oct 15 20:54:12.432440 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 918/1746] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:12 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:12.456340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-3e067248-3843-4fe0-8b0b-b1add892469e req-dab6a798-c468-4f14-b741-1eb72a973b01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76141eca-81bf-4771-87e9-fcd4007ff701", "name": "network-vif-deleted", "tag": "9585a427-e471-4df6-a4bc-2e935e877931"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:12.498718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3e067248-3843-4fe0-8b0b-b1add892469e req-dab6a798-c468-4f14-b741-1eb72a973b01 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:12.500205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3e067248-3843-4fe0-8b0b-b1add892469e req-dab6a798-c468-4f14-b741-1eb72a973b01 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:12.501256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3e067248-3843-4fe0-8b0b-b1add892469e req-dab6a798-c468-4f14-b741-1eb72a973b01 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:12.532669 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-3e067248-3843-4fe0-8b0b-b1add892469e req-dab6a798-c468-4f14-b741-1eb72a973b01 service nova] Creating event network-vif-deleted:9585a427-e471-4df6-a4bc-2e935e877931 for instance 76141eca-81bf-4771-87e9-fcd4007ff701 on nch2-master-1 Oct 15 20:54:12.547685 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-3e067248-3843-4fe0-8b0b-b1add892469e req-dab6a798-c468-4f14-b741-1eb72a973b01 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.112022 Oct 15 20:54:12.551359 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 919/1747] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:12.572536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b06f5e8d-2ea8-40a3-93e9-b41310f1174b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2351f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:12.735494 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b06f5e8d-2ea8-40a3-93e9-b41310f1174b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?name=tempest-ListImageFiltersTestJSON-image-54329338" status: 200 len: 663 microversion: 2.1 time: 0.173605 Oct 15 20:54:12.736853 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 920/1748] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 20:54:12 2022] GET /compute/v2.1/images/detail?name=tempest-ListImageFiltersTestJSON-image-54329338 => generated 663 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:12.758742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:12.759387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:12.810637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:12.811458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:12.812547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:12.814923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:12.817181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:12.818611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:12.891338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:12.955794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:12.957917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:12.988260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1baec436-3eaa-47ab-b682-c85aca1b0b86 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 1.674875 Oct 15 20:54:13.004810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 829/1749] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:11 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 1692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:13.015613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:13.041717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:13.071624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.072510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.073746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.075146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.077557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.079098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.214469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.217524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.218048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.222139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lazy-loading 'fault' on Instance uuid 76141eca-81bf-4771-87e9-fcd4007ff701 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:13.232467 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-154c6cef-823a-4b0a-98db-7e7ed845c7ec tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 204 len: 0 microversion: 2.1 time: 0.493977 Oct 15 20:54:13.235115 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 921/1750] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:12 2022] DELETE /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 0 bytes in 497 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:13.250499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:13.251626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:13.266832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.269060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.269863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.273122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.275379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.277399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.395533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.398336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.398795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.399266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lazy-loading 'fault' on Instance uuid 0e84ed5b-8560-4a2c-a144-382839509e8f {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:13.486450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:13.509373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb7841a7-fd38-4ea0-bbfc-f088a4e0187f tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1302 microversion: 2.1 time: 0.268674 Oct 15 20:54:13.522898 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 922/1751] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:13 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1302 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:13.522898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-29c843cc-d81d-412f-a6e8-7598d3c0d475 req-4c3f1de5-ceea-462e-a3c5-79ae7876920a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e84ed5b-8560-4a2c-a144-382839509e8f", "name": "network-vif-deleted", "tag": "5531cb3d-9d57-432f-9e05-8b2e5b9f4831"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:13.554631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-29c843cc-d81d-412f-a6e8-7598d3c0d475 req-4c3f1de5-ceea-462e-a3c5-79ae7876920a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:13.572606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-29c843cc-d81d-412f-a6e8-7598d3c0d475 req-4c3f1de5-ceea-462e-a3c5-79ae7876920a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:13.573853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-29c843cc-d81d-412f-a6e8-7598d3c0d475 req-4c3f1de5-ceea-462e-a3c5-79ae7876920a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:13.614010 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-29c843cc-d81d-412f-a6e8-7598d3c0d475 req-4c3f1de5-ceea-462e-a3c5-79ae7876920a service nova] Creating event network-vif-deleted:5531cb3d-9d57-432f-9e05-8b2e5b9f4831 for instance 0e84ed5b-8560-4a2c-a144-382839509e8f on nch1-master-1 Oct 15 20:54:13.628660 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-29c843cc-d81d-412f-a6e8-7598d3c0d475 req-4c3f1de5-ceea-462e-a3c5-79ae7876920a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.111591 Oct 15 20:54:13.630187 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 923/1752] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:13.649439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c96ea1c-b815-49f6-ad5c-166e0c921136 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f27eca0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:14.005361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:14.103128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:14.109200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:14.116624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b6f1cd3-1cce-49ed-9b5c-81195d1c9535 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 200 len: 1309 microversion: 2.1 time: 1.110192 Oct 15 20:54:14.142003 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 830/1753] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:13 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 1309 bytes in 1136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.147694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:14.159769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:14.162371 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c96ea1c-b815-49f6-ad5c-166e0c921136 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1465 microversion: 2.1 time: 0.529965 Oct 15 20:54:14.162962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 924/1754] 10.222.0.16 () {58 vars in 1338 bytes} [Sat Oct 15 20:54:13 2022] GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1465 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.183664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:14.185141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:14.189737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.190236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.190691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.191385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.191863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.192266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.205441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.205908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.206307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.207019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.213617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.214164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.476755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:14.489222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2798262-35a5-421a-bd05-bd5c4244fbc9 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1673 microversion: 2.1 time: 0.345366 Oct 15 20:54:14.490128 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 831/1755] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:14 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1673 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.496326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:14.503825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e3ab5eb1-885c-4a4b-b01b-7f6b3445e37c req-07ac51fd-d3f2-4ba9-8c60-2bf8bcb7c839 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "22368920-f970-409f-a301-afe70a4ee4c8", "name": "network-vif-unplugged", "status": "completed", "tag": "113c86a4-f077-45e8-aafc-023b1a21c6ef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:14.515062 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da41bc06-a1a2-43db-a937-da9e83d6d7e1 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.336278 Oct 15 20:54:14.516404 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 925/1756] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:14 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.521512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:14.522392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:14.529696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.530371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.530974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.531899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.532512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.533441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.534475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e3ab5eb1-885c-4a4b-b01b-7f6b3445e37c req-07ac51fd-d3f2-4ba9-8c60-2bf8bcb7c839 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.535013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e3ab5eb1-885c-4a4b-b01b-7f6b3445e37c req-07ac51fd-d3f2-4ba9-8c60-2bf8bcb7c839 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.535466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e3ab5eb1-885c-4a4b-b01b-7f6b3445e37c req-07ac51fd-d3f2-4ba9-8c60-2bf8bcb7c839 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.550450 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e3ab5eb1-885c-4a4b-b01b-7f6b3445e37c req-07ac51fd-d3f2-4ba9-8c60-2bf8bcb7c839 service nova] Creating event network-vif-unplugged:113c86a4-f077-45e8-aafc-023b1a21c6ef for instance 22368920-f970-409f-a301-afe70a4ee4c8 on nch2-master-1 Oct 15 20:54:14.555992 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e3ab5eb1-885c-4a4b-b01b-7f6b3445e37c req-07ac51fd-d3f2-4ba9-8c60-2bf8bcb7c839 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.063718 Oct 15 20:54:14.556822 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 832/1757] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:14.563539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f548df1-4ddc-40ad-9727-4a0bb91cb2cb tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe75850f670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:14.650871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f548df1-4ddc-40ad-9727-4a0bb91cb2cb tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1465 microversion: 2.1 time: 0.092635 Oct 15 20:54:14.652721 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 833/1758] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Oct 15 20:54:14 2022] GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1465 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.658294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-958827595", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "f88dd690-f671-4f8e-8831-3ff9fe36c6b1"}], "security_groups": [{"name": "tempest-securitygroup--708482109"}], "key_name": "tempest-keypair-1573082579", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:14.702050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:14.715002 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fd976f44-b798-4ddf-ae7b-1f47ef368936 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.197101 Oct 15 20:54:14.716464 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 926/1759] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:14 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.721747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:14.722589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:14.729423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.730191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.730810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.733365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.734003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.734633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.773697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.774451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.775044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.775950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lazy-loading 'fault' on Instance uuid 0e84ed5b-8560-4a2c-a144-382839509e8f {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:14.804258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] validate_networks() for [('f88dd690-f671-4f8e-8831-3ff9fe36c6b1', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:54:14.831382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:14.839610 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4e24e647-ad2f-4a8b-a48a-4d6896ccbc16 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 200 len: 1302 microversion: 2.1 time: 0.121625 Oct 15 20:54:14.840170 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 927/1760] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:14 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 1302 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:14.845866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bec91401-323d-4a36-88b7-88fe24aa813f", "name": "network-vif-unplugged", "status": "completed", "tag": "e75505e8-a2ad-4084-9f6f-4d72bd372f6f"}, {"name": "network-changed", "server_uuid": "bec91401-323d-4a36-88b7-88fe24aa813f", "tag": "e75505e8-a2ad-4084-9f6f-4d72bd372f6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:14.856719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:14.857485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:14.858080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:14.874819 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] Creating event network-vif-unplugged:e75505e8-a2ad-4084-9f6f-4d72bd372f6f for instance bec91401-323d-4a36-88b7-88fe24aa813f on nch2-master-1 Oct 15 20:54:14.875581 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] Creating event network-changed:e75505e8-a2ad-4084-9f6f-4d72bd372f6f for instance bec91401-323d-4a36-88b7-88fe24aa813f on nch2-master-1 Oct 15 20:54:14.883902 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-fb8c35d8-43f6-4f4a-b8c2-62c2a711d945 req-cf693687-b847-4421-a398-66772b491e51 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 352 microversion: 2.1 time: 0.041712 Oct 15 20:54:14.885014 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 928/1761] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:14 2022] POST /compute/v2.1/os-server-external-events => generated 352 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:14.893076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e13981c-c741-440c-9c78-bd74b883bff3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2fa820>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.078823 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e13981c-c741-440c-9c78-bd74b883bff3 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?status=ACTIVE" status: 200 len: 7791 microversion: 2.1 time: 0.191834 Oct 15 20:54:15.080240 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 929/1762] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 20:54:14 2022] GET /compute/v2.1/images/detail?status=ACTIVE => generated 7791 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:15.114899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b6d112ce-8f3b-4c63-9dc4-45e1df8dee67 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f27eb80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.317904 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b6d112ce-8f3b-4c63-9dc4-45e1df8dee67 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?type=snapshot" status: 200 len: 4371 microversion: 2.1 time: 0.207607 Oct 15 20:54:15.328443 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 930/1763] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/images/detail?type=snapshot => generated 4371 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:15.346239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.352891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 76141eca-81bf-4771-87e9-fcd4007ff701] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:15.389153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.389985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.390519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.391433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.391903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.392337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.411917 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance 76141eca-81bf-4771-87e9-fcd4007ff701 could not be found. Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 HTTP/1.1 Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:15.413893 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 76141eca-81bf-4771-87e9-fcd4007ff701 could not be found. Oct 15 20:54:15.415679 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '76141eca-81bf-4771-87e9-fcd4007ff701'} Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 76141eca-81bf-4771-87e9-fcd4007ff701 could not be found. Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:15.416611 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 76141eca-81bf-4771-87e9-fcd4007ff701 could not be found. Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 76141eca-81bf-4771-87e9-fcd4007ff701 could not be found. Oct 15 20:54:15.418207 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:15.422003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance 76141eca-81bf-4771-87e9-fcd4007ff701 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:15.423823 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3d7cb681-b95d-4f11-aef6-3f035df4e52f tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701" status: 404 len: 111 microversion: 2.1 time: 0.094249 Oct 15 20:54:15.424952 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 931/1764] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/servers/76141eca-81bf-4771-87e9-fcd4007ff701 => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:15.429967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-16096752-b76a-4b12-a72a-bab026d5df52 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f279e50>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.496274 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-16096752-b76a-4b12-a72a-bab026d5df52 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/77903f56-0555-4c00-82ae-bbf42e11a2a9" status: 200 len: 784 microversion: 2.1 time: 0.068429 Oct 15 20:54:15.501529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 932/1765] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/images/77903f56-0555-4c00-82ae-bbf42e11a2a9 => generated 784 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:15.510134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.511451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:15.521174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.521993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.522618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.524325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.524985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.525647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.596758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:54:15.598009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:54:15.598687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:54:15.599294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:54:15.602794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:54:15.614756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.615354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.616062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.616871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.617533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.618051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.627081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:15.637131 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d51b1e89-c2dd-43a4-ad65-7002c2aeab7e tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1433 microversion: 2.1 time: 0.133216 Oct 15 20:54:15.637824 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 933/1766] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1433 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:15.642674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Getting quotas for project e08b85b4f6cf47a88fe24ead8e6d45e9. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:15.643413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.644367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:15.648623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Getting quotas for user ec8dcd7615cc43b3866034d5af34a9a1 and project e08b85b4f6cf47a88fe24ead8e6d45e9. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:15.655149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.655727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.656235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.657014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.658471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.658985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.664102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:54:15.666591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:54:15.718012 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-254f6b7d-6d33-4446-be59-2779066428a4 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.063433 Oct 15 20:54:15.718684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 834/1767] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:54:14 2022] POST /compute/v2.1/servers => generated 405 bytes in 1065 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:54:15.724138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d95f4c2-c643-423a-a663-8433620786b7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe758406af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.779660 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d95f4c2-c643-423a-a663-8433620786b7 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/images/detail?limit=1" status: 200 len: 1609 microversion: 2.1 time: 0.059919 Oct 15 20:54:15.780487 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 835/1768] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/images/detail?limit=1 => generated 1609 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:15.786063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.786854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:15.797651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.798152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.798597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.799313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.799775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.800209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.893259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:15.908187 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb09e154-b68c-47f6-a95e-49f491b5c0fc tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1673 microversion: 2.1 time: 0.269016 Oct 15 20:54:15.921060 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 934/1769] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1673 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:15.921060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-36d6e400-2604-4775-920a-52eecdfe10ee tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.921060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-36d6e400-2604-4775-920a-52eecdfe10ee tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:15.965309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:15.974764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d893be4b-5e06-4116-93b6-9ad754eb6009 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.192851 Oct 15 20:54:15.975927 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 836/1770] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:15.981908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:15.982526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:15.988501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.988994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.989492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:15.990193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:15.990662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:15.991091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.036797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-36d6e400-2604-4775-920a-52eecdfe10ee tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:16.064948 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-36d6e400-2604-4775-920a-52eecdfe10ee tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1241 microversion: 2.1 time: 0.153064 Oct 15 20:54:16.076580 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 935/1771] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1241 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:16.092561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-62712a16-1d0e-47d7-9ef1-16e35e825941 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f279550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:16.238061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:16.249763 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-00737b39-4e67-427e-a342-294be91ee682 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.271744 Oct 15 20:54:16.262067 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 837/1772] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:15 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:16.271167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6c8e9963-c95a-4564-89dc-97bd863b9bd0 req-9f4f73c8-cc02-43f9-a78d-d8b14b9a1d0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "22368920-f970-409f-a301-afe70a4ee4c8", "name": "network-vif-deleted", "tag": "113c86a4-f077-45e8-aafc-023b1a21c6ef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:16.301397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c8e9963-c95a-4564-89dc-97bd863b9bd0 req-9f4f73c8-cc02-43f9-a78d-d8b14b9a1d0a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.302151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c8e9963-c95a-4564-89dc-97bd863b9bd0 req-9f4f73c8-cc02-43f9-a78d-d8b14b9a1d0a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.302946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c8e9963-c95a-4564-89dc-97bd863b9bd0 req-9f4f73c8-cc02-43f9-a78d-d8b14b9a1d0a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.321218 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6c8e9963-c95a-4564-89dc-97bd863b9bd0 req-9f4f73c8-cc02-43f9-a78d-d8b14b9a1d0a service nova] Creating event network-vif-deleted:113c86a4-f077-45e8-aafc-023b1a21c6ef for instance 22368920-f970-409f-a301-afe70a4ee4c8 on nch2-master-1 Oct 15 20:54:16.327313 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6c8e9963-c95a-4564-89dc-97bd863b9bd0 req-9f4f73c8-cc02-43f9-a78d-d8b14b9a1d0a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067695 Oct 15 20:54:16.328395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 838/1773] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:16.339769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:16.341211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: 0e84ed5b-8560-4a2c-a144-382839509e8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:16.349970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.350604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.351099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.352495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.352958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.353411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.371420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] HTTP exception thrown: Instance 0e84ed5b-8560-4a2c-a144-382839509e8f could not be found. Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Request method failure captured: Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f HTTP/1.1 Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxyrxLt9ZkRGBZEC262Xudbc4kffeptUv_lXweM-AbzOgTA--5IzJeazLLXDuegKkd17i1HT8bNeetv0IXo1FIUSXopfcoPvaNxD6gFon6e8zjH7XSGtH8Uh6ycdR8hHJFlbVtzOSITJGkIgH80sQdi_CjHEAN8I7EWSLXG9fMC0qEE Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:54:16.374826 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_94e961feee714587bb2ffe545d286c5c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MigrationsAdminTest-389983916 Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MigrationsAdminTest-389983916-project Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e32a1c53b4be44188b07c768405961ed Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MigrationsAdminTest-389983916-project Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 0e84ed5b-8560-4a2c-a144-382839509e8f could not be found. Oct 15 20:54:16.376522 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0e84ed5b-8560-4a2c-a144-382839509e8f'} Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0e84ed5b-8560-4a2c-a144-382839509e8f could not be found. Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:16.377620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0e84ed5b-8560-4a2c-a144-382839509e8f could not be found. Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0e84ed5b-8560-4a2c-a144-382839509e8f could not be found. Oct 15 20:54:16.379948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:16.381887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Returning 404 to user: Instance 0e84ed5b-8560-4a2c-a144-382839509e8f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:16.381887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffbb93b1-2ce9-4a5e-ae56-0fcc9aa969de tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f" status: 404 len: 111 microversion: 2.1 time: 0.047637 Oct 15 20:54:16.381887 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 839/1774] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:16 2022] GET /compute/v2.1/servers/0e84ed5b-8560-4a2c-a144-382839509e8f => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:16.389684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:16.390306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:16.410168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.410672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.411323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.412164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.412912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.413720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.610217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid c05e4f55-245d-45f2-8567-e665ccbca061 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:16.635672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d559183-aff8-4b8a-8083-2d6770d96e60 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 2423 microversion: 2.25 time: 0.250229 Oct 15 20:54:16.636735 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 840/1775] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:16 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 2423 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:16.667372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:16.668317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:16.676454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.677423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.678248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.679163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.679802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.680372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.730362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:16.784471 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-62712a16-1d0e-47d7-9ef1-16e35e825941 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a" status: 204 len: 0 microversion: 2.1 time: 0.704042 Oct 15 20:54:16.785145 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 936/1776] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:16 2022] DELETE /compute/v2.1/images/00cdd6af-5ed0-449d-b15a-104fc343c17a => generated 0 bytes in 708 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:16.792499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:16.793323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:16.809987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.815654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.816440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.818313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.821829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.822527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.850038 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8df4a6c9-302a-4925-90d1-675c97314d7a tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 204 len: 0 microversion: 2.1 time: 0.197557 Oct 15 20:54:16.851954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 841/1777] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:16 2022] DELETE /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:16.862926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": "auto"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:16.866979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:16.874983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.875494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.875935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.876700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:16.877160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:16.877589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:16.909465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'flavor' on Instance uuid c05e4f55-245d-45f2-8567-e665ccbca061 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:16.927821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Going to try to live migrate instance to nch2-master-1 {{(pid=138688) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:54:16.944494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:16.953680 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7f16246d-a10d-4c58-ab0b-0c37670ca788 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1304 microversion: 2.1 time: 0.167206 Oct 15 20:54:16.954801 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 937/1778] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:16 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1304 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:16.959896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-51fca1be-d012-4ccd-9b6a-e672c63fe559 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f2fa3a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:17.518959 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-51fca1be-d012-4ccd-9b6a-e672c63fe559 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896" status: 204 len: 0 microversion: 2.1 time: 0.562874 Oct 15 20:54:17.519583 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 938/1779] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:16 2022] DELETE /compute/v2.1/images/bd2d36b1-b444-4617-964f-35e777081896 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:17.524453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:17.525075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:17.531554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.532034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.532651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.533388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.533846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.534271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.667656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:17.675512 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-242b5aec-c8a8-4e9c-8896-d7a4fcea7e87 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1468 microversion: 2.1 time: 0.154545 Oct 15 20:54:17.676168 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 939/1780] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:17 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1468 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:17.680769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d245d5de-6cf3-4c08-a03e-a21673f951ff req-852496f8-d98b-4b8d-b8e7-761f466214c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c10c829d-ef62-4d1c-98e9-422615414c5e", "name": "network-vif-deleted", "tag": "2557963e-5bb5-430b-9efd-87827090ab6c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:17.690251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d245d5de-6cf3-4c08-a03e-a21673f951ff req-852496f8-d98b-4b8d-b8e7-761f466214c6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.690736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d245d5de-6cf3-4c08-a03e-a21673f951ff req-852496f8-d98b-4b8d-b8e7-761f466214c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.691173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d245d5de-6cf3-4c08-a03e-a21673f951ff req-852496f8-d98b-4b8d-b8e7-761f466214c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.702935 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d245d5de-6cf3-4c08-a03e-a21673f951ff req-852496f8-d98b-4b8d-b8e7-761f466214c6 service nova] Creating event network-vif-deleted:2557963e-5bb5-430b-9efd-87827090ab6c for instance c10c829d-ef62-4d1c-98e9-422615414c5e on nch1-master-1 Oct 15 20:54:17.707749 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d245d5de-6cf3-4c08-a03e-a21673f951ff req-852496f8-d98b-4b8d-b8e7-761f466214c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030418 Oct 15 20:54:17.708315 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 940/1781] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:17.713346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:17.714038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:17.720123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.720605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.721083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.721759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.722215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.722639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.786594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:17.794751 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9967e0a4-4e32-43dd-89a0-39af2f91df8e tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1610 microversion: 2.1 time: 0.085255 Oct 15 20:54:17.795322 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 941/1782] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:17 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1610 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:17.799890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:17.800501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:17.807665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.808185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.808669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.810036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.810520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.810942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.942481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:17.950807 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e6505248-5ca1-452b-905d-3893b17ef4a9 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.153850 Oct 15 20:54:17.951356 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 942/1783] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:17 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:17.955741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:17.956379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:17.962004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.962482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.962914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:17.963598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:17.964052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:17.964471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.037479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:18.045941 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-18e6eb1c-b9d0-4cfc-b3aa-b1644c1ff2b5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1293 microversion: 2.1 time: 0.093603 Oct 15 20:54:18.046506 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 943/1784] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:17 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1293 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:18.053376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:18.054263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:18.060737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.061365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.061818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.062527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.062983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.063403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.206757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:18.215335 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c5cae47f-2df0-4da3-8946-d6463d08b0c2 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.167498 Oct 15 20:54:18.216036 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 944/1785] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:18 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:18.220430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ee747fa5-3df7-41c7-ad83-960811cf6a1b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f20baf0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:18.703657 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ee747fa5-3df7-41c7-ad83-960811cf6a1b tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e" status: 204 len: 0 microversion: 2.1 time: 0.486489 Oct 15 20:54:18.704235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 945/1786] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:18 2022] DELETE /compute/v2.1/images/b9c86776-0529-4b97-aa36-07daf5b4db8e => generated 0 bytes in 488 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:18.709759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:18.710367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:18.718374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.718981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.719414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.720400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.721080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.721702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.766319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.767925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.768602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.769840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lazy-loading 'fault' on Instance uuid 22368920-f970-409f-a301-afe70a4ee4c8 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:18.863221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:18.865542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:18.874111 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec297ac6-5192-4329-8b0b-5dd47bff7f32 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1309 microversion: 2.1 time: 0.168339 Oct 15 20:54:18.874735 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 946/1787] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:18 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1309 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:18.879688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:18.880295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:18.889357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.890311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.890753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.891472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.892616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.893075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.969141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e32a1c53b4be44188b07c768405961ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '94e961feee714587bb2ffe545d286c5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:18.977459 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-86f49317-8c80-4c68-b08b-2ec5f90111e8 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 200 len: 1426 microversion: 2.1 time: 0.101685 Oct 15 20:54:18.978022 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 947/1788] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:18 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 1426 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:18.982665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:18.983217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:18.989325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.989806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.990239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:18.990932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:18.991382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:18.991800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.018395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:19.087919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=3,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:19.095280 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4cafdc24-ba29-49c3-81a2-c46204a2cfc9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 204 len: 0 microversion: 2.1 time: 0.116129 Oct 15 20:54:19.095857 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 948/1789] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:18 2022] DELETE /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:19.100931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:19.101915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:19.108865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.109372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.109805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.110502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.110957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.111379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.143149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.143625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.144059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.144504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lazy-loading 'fault' on Instance uuid c10c829d-ef62-4d1c-98e9-422615414c5e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:19.186559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bbc891924f84d99aac22518f69e7b99', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c30aa11fd244564be716550bfe5aaa6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:19.195359 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-94b86d85-e67c-4be1-8970-f47041dcecbe tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 200 len: 1302 microversion: 2.1 time: 0.098285 Oct 15 20:54:19.209468 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 949/1790] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:19 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 1302 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:19.209468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:19.209468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:19.209468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.210529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.210529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.211275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.211923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.213291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.385419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:19.394844 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31aa85c7-0d3d-477c-891d-39cd22aebc0f tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.197821 Oct 15 20:54:19.409682 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 950/1791] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:19 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:19.417564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:19.418457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:19.426557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.427301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.427957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.428913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.429514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.430322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.509624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:19.521463 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5d463166-f342-4fd4-ad29-c12074a3a986 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1299 microversion: 2.1 time: 0.110391 Oct 15 20:54:19.522864 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 951/1792] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:19 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1299 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:19.528213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:19.530993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:19.542387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.542891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.544087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.544919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.545408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.546125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.705734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:19.714080 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-57d63e77-3fd8-42bd-a258-763ba756b589 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1474 microversion: 2.1 time: 0.189732 Oct 15 20:54:19.725218 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 952/1793] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:19 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1474 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:19.729214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:19.730027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:19.736734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.737264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.737706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.738405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.738863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.739292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.877891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:19.887252 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dcedc42b-7512-4ff0-b969-cf065f11925b tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.171129 Oct 15 20:54:19.888385 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 953/1794] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:19 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:19.893512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c10c829d-ef62-4d1c-98e9-422615414c5e", "name": "network-vif-deleted", "tag": "7c2c4743-524f-46ad-b187-3b4184a1040b"}, {"server_uuid": "e597eee9-ea21-44d4-8f17-5726355a6114", "name": "network-vif-deleted", "tag": "a1988c6d-abca-4574-9f81-2e94f32f51da"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:19.907752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.908585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.909468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.925389 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] Creating event network-vif-deleted:7c2c4743-524f-46ad-b187-3b4184a1040b for instance c10c829d-ef62-4d1c-98e9-422615414c5e on nch1-master-1 Oct 15 20:54:19.925696 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] Creating event network-vif-deleted:a1988c6d-abca-4574-9f81-2e94f32f51da for instance e597eee9-ea21-44d4-8f17-5726355a6114 on nch1-master-1 Oct 15 20:54:19.932748 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-154abad0-d62b-4844-a12d-d104dc3d517e req-b12b9ee6-b42d-4984-9920-23820b919c8a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.043211 Oct 15 20:54:19.933617 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 954/1795] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:19 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:19.940385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:19.941146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:19.947789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.948204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.948638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.949375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.949822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.950427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.992295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:19.992811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:19.993265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:19.993716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lazy-loading 'fault' on Instance uuid 22368920-f970-409f-a301-afe70a4ee4c8 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:20.062368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:20.071100 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7171adea-edc6-49ca-8dc2-cc136d736b89 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 200 len: 1309 microversion: 2.1 time: 0.136052 Oct 15 20:54:20.071684 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 955/1796] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:19 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 1309 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:20.076750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:20.077694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] [instance: e597eee9-ea21-44d4-8f17-5726355a6114] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:20.084277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.084901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.085661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.087639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.088204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.088803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.104940 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] HTTP exception thrown: Instance e597eee9-ea21-44d4-8f17-5726355a6114 could not be found. Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Request method failure captured: Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 HTTP/1.1 Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSxyrxLt9ZkRGBZEC262Xudbc4kffeptUv_lXweM-AbzOgTA--5IzJeazLLXDuegKkd17i1HT8bNeetv0IXo1FIUSXopfcoPvaNxD6gFon6e8zjH7XSGtH8Uh6ycdR8hHJFlbVtzOSITJGkIgH80sQdi_CjHEAN8I7EWSLXG9fMC0qEE Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:20.107123 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_94e961feee714587bb2ffe545d286c5c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/94e961feee714587bb2ffe545d286c5c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-MigrationsAdminTest-389983916 Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 94e961feee714587bb2ffe545d286c5c Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-MigrationsAdminTest-389983916 Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-MigrationsAdminTest-389983916-project Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: e32a1c53b4be44188b07c768405961ed Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-MigrationsAdminTest-389983916-project Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance e597eee9-ea21-44d4-8f17-5726355a6114 could not be found. Oct 15 20:54:20.108751 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'e597eee9-ea21-44d4-8f17-5726355a6114'} Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e597eee9-ea21-44d4-8f17-5726355a6114 could not be found. Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:20.109633 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e597eee9-ea21-44d4-8f17-5726355a6114 could not be found. Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e597eee9-ea21-44d4-8f17-5726355a6114 could not be found. Oct 15 20:54:20.111022 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.112232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] Returning 404 to user: Instance e597eee9-ea21-44d4-8f17-5726355a6114 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:20.112232 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2d81a060-2baf-4cec-9cf7-4895cafd6665 tempest-MigrationsAdminTest-389983916 tempest-MigrationsAdminTest-389983916-project] 10.222.0.16 "GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114" status: 404 len: 111 microversion: 2.1 time: 0.035810 Oct 15 20:54:20.112232 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 956/1797] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:20 2022] GET /compute/v2.1/servers/e597eee9-ea21-44d4-8f17-5726355a6114 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:20.223798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:20.224413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:20.231398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.232465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.233132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.234011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.234652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.235241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.251768 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] HTTP exception thrown: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Request method failure captured: Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e HTTP/1.1 Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2ANl7AV4DUGTAmJCJrJoBUGOPLq38faLYUZfcViDfXVEvnA_PNoZJFdNze6Vt9ihvNbHykKXsI_OpFPpZY5X2AN7w4Xgg5TdRRgfn8pIEhKZPCdQbqV3CubFinqCtxxFbMVTmObo3tgEYJbzoNGy9bnJvpixa-cFDSYU4dED45oOM Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:20.253437 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 8bbc891924f84d99aac22518f69e7b99 Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:20.255435 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'c10c829d-ef62-4d1c-98e9-422615414c5e'} Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:20.259864 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:20.261847 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:20.264529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Returning 404 to user: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:20.264529 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d828456-9b70-4e62-abf5-3540b54032ac tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 404 len: 111 microversion: 2.1 time: 0.036069 Oct 15 20:54:20.264529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 957/1798] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:20 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:54:20.431625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:20.432411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:20.441706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.444430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.445296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.446416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.447515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.448518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.687385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:20.696450 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-10dbc7be-803b-45bd-be97-0ac03dff798a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.278641 Oct 15 20:54:20.711061 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 958/1799] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:20 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:20.727406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:20.728989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:20.743558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.744256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.744798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:20.746351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:20.748959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:20.750124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:21.110970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:21.142725 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-be3fa1b9-b074-489d-a7e0-956740e5da6d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1366 microversion: 2.1 time: 0.425657 Oct 15 20:54:21.143332 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 959/1800] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:20 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1366 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:21.172626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:21.173464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:21.182846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:21.183649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:21.185817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:21.187175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:21.187898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:21.188631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:21.566382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:21.578407 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-184ce50f-87d7-456d-a6e1-89c8c3bd548e tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1474 microversion: 2.1 time: 0.415807 Oct 15 20:54:21.585086 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 960/1801] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:21 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1474 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:21.593988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:21.594624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:21.603789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:21.604318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:21.604768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:21.605559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:21.606181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:21.606924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.084861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:22.095714 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6f327f25-6bfe-425e-8db0-d8e00d97ee79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.508115 Oct 15 20:54:22.097081 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 961/1802] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:21 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:22.102284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:22.102893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 22368920-f970-409f-a301-afe70a4ee4c8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:22.109354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.110135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.110748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.111660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.112754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.113230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.129282 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance 22368920-f970-409f-a301-afe70a4ee4c8 could not be found. Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 HTTP/1.1 Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:22.131320 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 22368920-f970-409f-a301-afe70a4ee4c8 could not be found. Oct 15 20:54:22.133015 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '22368920-f970-409f-a301-afe70a4ee4c8'} Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 22368920-f970-409f-a301-afe70a4ee4c8 could not be found. Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:22.134003 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 22368920-f970-409f-a301-afe70a4ee4c8 could not be found. Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 22368920-f970-409f-a301-afe70a4ee4c8 could not be found. Oct 15 20:54:22.135581 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:22.138393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance 22368920-f970-409f-a301-afe70a4ee4c8 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:22.139278 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9e43229c-aa9d-4c68-ab04-838f7a945978 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8" status: 404 len: 111 microversion: 2.1 time: 0.041161 Oct 15 20:54:22.139860 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 962/1803] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:22 2022] GET /compute/v2.1/servers/22368920-f970-409f-a301-afe70a4ee4c8 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:22.147479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-40b7bb3e-d4f7-4e10-8042-5d9db81fa573 req-c25bd768-ef04-463a-b2f9-bd54f316643e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "062c4651-90be-430a-a69b-372fac258ebb", "tag": "e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:22.158161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-40b7bb3e-d4f7-4e10-8042-5d9db81fa573 req-c25bd768-ef04-463a-b2f9-bd54f316643e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.158650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-40b7bb3e-d4f7-4e10-8042-5d9db81fa573 req-c25bd768-ef04-463a-b2f9-bd54f316643e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.159088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-40b7bb3e-d4f7-4e10-8042-5d9db81fa573 req-c25bd768-ef04-463a-b2f9-bd54f316643e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.171826 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-40b7bb3e-d4f7-4e10-8042-5d9db81fa573 req-c25bd768-ef04-463a-b2f9-bd54f316643e service nova] Creating event network-changed:e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8 for instance 062c4651-90be-430a-a69b-372fac258ebb on nch1-master-1 Oct 15 20:54:22.177518 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-40b7bb3e-d4f7-4e10-8042-5d9db81fa573 req-c25bd768-ef04-463a-b2f9-bd54f316643e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036206 Oct 15 20:54:22.179139 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 963/1804] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:22.185868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:22.186598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:22.200067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.201178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.201687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.202379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.203572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.204231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.225347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:22.565928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:22.604780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-63e7ff4d-f2c0-4d86-810a-57fe66c19676 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.423385 Oct 15 20:54:22.606021 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 964/1805] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:22 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:22.613397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f75c0b02-6ee2-4ecf-9706-a776d3dea2d7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:22.687211 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f75c0b02-6ee2-4ecf-9706-a776d3dea2d7 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/98fcc0af-9b5b-4ce5-ab02-53a563fa5866" status: 204 len: 0 microversion: 2.1 time: 0.079224 Oct 15 20:54:22.687672 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 965/1806] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 20:54:22 2022] DELETE /compute/v2.1/os-server-groups/98fcc0af-9b5b-4ce5-ab02-53a563fa5866 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:22.695547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:22.696511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:22.737223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.737947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.743696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.749927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:22.752168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:22.754102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:22.984295 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4862d949-14d5-4a4c-b15d-52f9b65775ff tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/action" status: 202 len: 0 microversion: 2.25 time: 6.130567 Oct 15 20:54:22.985341 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 842/1807] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:54:16 2022] POST /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/action => generated 0 bytes in 6133 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:22.996658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:22.998866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:23.009149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.009832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.010286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.012119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.013018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.015082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.096690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.098572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.100137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.103854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lazy-loading 'fault' on Instance uuid 94ed23bf-05fc-41bf-a0d4-547148104a6f {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:23.280790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:23.304628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:23.315221 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9046647a-e1fa-41aa-8756-fd5e162c84b8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1366 microversion: 2.1 time: 0.626240 Oct 15 20:54:23.316350 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 966/1808] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:22 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1366 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:23.330422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:23.331280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:23.340262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.344067 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33ac890a-a4c2-4801-9363-ccfa40563e54 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 200 len: 1308 microversion: 2.1 time: 0.347743 Oct 15 20:54:23.344731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 843/1809] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:22 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 1308 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:23.345255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.345997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.346986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.349406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.350376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.363060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:23.364239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:23.380868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.382000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.383891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.385030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.391191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.391936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.469779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:23.659678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4fb19c82-16e3-4de4-aa9e-39ebbe093bf2 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 204 len: 0 microversion: 2.1 time: 0.341208 Oct 15 20:54:23.661907 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 967/1810] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:23 2022] DELETE /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:23.674474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:23.674474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:23.692702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.695439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.696237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:23.698705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:23.702606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:23.703185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.053285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.062664 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-90811a62-4cea-4259-9aee-03a0aa7358b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.398880 Oct 15 20:54:24.073580 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 968/1811] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:23 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:24.085781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.086686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.094197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.094944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.096071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.096999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.097703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.098345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.242340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.263837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.275074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.276596 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4e805fdb-8067-459d-a7b9-12683906a0c1 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.921323 Oct 15 20:54:24.290891 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 844/1812] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:23 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 945 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:24.303677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3b4ba103-11cd-47f5-8e52-f16aa76e08b3 req-4f00e6a2-bc64-4d29-98fd-ebe52543e3a8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82e74d13-2be4-47d0-93fa-fb24eb81c212", "name": "network-vif-plugged", "status": "completed", "tag": "328d28c6-9ece-4b0b-b5f5-e870e5285550"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:24.321360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3b4ba103-11cd-47f5-8e52-f16aa76e08b3 req-4f00e6a2-bc64-4d29-98fd-ebe52543e3a8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.321837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3b4ba103-11cd-47f5-8e52-f16aa76e08b3 req-4f00e6a2-bc64-4d29-98fd-ebe52543e3a8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.322261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3b4ba103-11cd-47f5-8e52-f16aa76e08b3 req-4f00e6a2-bc64-4d29-98fd-ebe52543e3a8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.336352 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3b4ba103-11cd-47f5-8e52-f16aa76e08b3 req-4f00e6a2-bc64-4d29-98fd-ebe52543e3a8 service nova] Creating event network-vif-plugged:328d28c6-9ece-4b0b-b5f5-e870e5285550 for instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 on nch2-master-1 Oct 15 20:54:24.343054 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3b4ba103-11cd-47f5-8e52-f16aa76e08b3 req-4f00e6a2-bc64-4d29-98fd-ebe52543e3a8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050280 Oct 15 20:54:24.347334 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 845/1813] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:24.354875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.355590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.366713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.367210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.367646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.368355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.369186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.369623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.429330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.453995 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f34db0d-8ad0-48b0-b9c6-f1653d9a1c44 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.372442 Oct 15 20:54:24.454586 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 969/1814] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:24 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:24.466521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.467434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.477652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.479777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.480646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.481909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.482989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.483642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.748469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.759472 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a1063b63-f00f-45d3-ae07-0ee4d7f491f3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1366 microversion: 2.1 time: 0.303832 Oct 15 20:54:24.762571 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 970/1815] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:24 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1366 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:24.769944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.771045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 94ed23bf-05fc-41bf-a0d4-547148104a6f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.781486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.782537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.783603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.784315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.785359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.786178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.795167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:24.810135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-00e5c22f-42c2-452f-ad68-89f8b6e83504 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1473 microversion: 2.1 time: 0.460236 Oct 15 20:54:24.812365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 846/1816] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:24 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1473 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:24.822262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7f590fb2-f584-4c39-9d6e-8f7624701661 req-b7303b92-1bfc-445f-982e-f016011d1418 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "94ed23bf-05fc-41bf-a0d4-547148104a6f", "name": "network-vif-deleted", "tag": "a172cd8d-3a0e-48dc-be83-b54ffefe0f32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:24.823642 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] HTTP exception thrown: Instance 94ed23bf-05fc-41bf-a0d4-547148104a6f could not be found. Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Request method failure captured: Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f HTTP/1.1 Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx1EB-yAD7UG_Dap49GTD2s8yrVWivZQT83hJXr53R1MYLFh5JetTyf34JrS79KToWBGhFfABggxbY8VMHrP8asFDj5BfINoG6xOIk-6HEEXs6vjMOkFIpHoxQNeHZMks1TEz8PS3VrWi2WkmPlVxbgjJFh50JYX42qGEPiKTB5dloA Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e5c50f30f706437ea737a3d5d20ea8a1 Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ListImageFiltersTestJSON-505006441 Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:24.828463 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ListImageFiltersTestJSON-505006441 Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e5c50f30f706437ea737a3d5d20ea8a1 Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ListImageFiltersTestJSON-505006441 Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ListImageFiltersTestJSON-505006441-project Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 3e76b25814474b9db035954b3a437c96 Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ListImageFiltersTestJSON-505006441-project Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 94ed23bf-05fc-41bf-a0d4-547148104a6f could not be found. Oct 15 20:54:24.830231 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '94ed23bf-05fc-41bf-a0d4-547148104a6f'} Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 94ed23bf-05fc-41bf-a0d4-547148104a6f could not be found. Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:24.831254 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 94ed23bf-05fc-41bf-a0d4-547148104a6f could not be found. Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 94ed23bf-05fc-41bf-a0d4-547148104a6f could not be found. Oct 15 20:54:24.832645 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.834304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Returning 404 to user: Instance 94ed23bf-05fc-41bf-a0d4-547148104a6f could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:24.838015 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-51a22903-6cfd-45ba-9e03-0d1af4cce595 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f" status: 404 len: 111 microversion: 2.1 time: 0.072533 Oct 15 20:54:24.839767 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 971/1817] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:24 2022] GET /compute/v2.1/servers/94ed23bf-05fc-41bf-a0d4-547148104a6f => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:24.855206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.857281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.858200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f590fb2-f584-4c39-9d6e-8f7624701661 req-b7303b92-1bfc-445f-982e-f016011d1418 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.858767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f590fb2-f584-4c39-9d6e-8f7624701661 req-b7303b92-1bfc-445f-982e-f016011d1418 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.859306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f590fb2-f584-4c39-9d6e-8f7624701661 req-b7303b92-1bfc-445f-982e-f016011d1418 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.871347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.873644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.875039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.875260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.875489 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7f590fb2-f584-4c39-9d6e-8f7624701661 req-b7303b92-1bfc-445f-982e-f016011d1418 service nova] Creating event network-vif-deleted:a172cd8d-3a0e-48dc-be83-b54ffefe0f32 for instance 94ed23bf-05fc-41bf-a0d4-547148104a6f on nch2-master-1 Oct 15 20:54:24.876516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.876516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.882816 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7f590fb2-f584-4c39-9d6e-8f7624701661 req-b7303b92-1bfc-445f-982e-f016011d1418 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066461 Oct 15 20:54:24.884035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 847/1818] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:24.892150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.892938 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] HTTP exception thrown: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.893661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Request method failure captured: Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e HTTP/1.1 Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2ANl7AV4DUGTAmJCJrJoBUGOPLq38faLYUZfcViDfXVEvnA_PNoZJFdNze6Vt9ihvNbHykKXsI_OpFPpZY5X2AN7w4Xgg5TdRRgfn8pIEhKZPCdQbqV3CubFinqCtxxFbMVTmObo3tgEYJbzoNGy9bnJvpixa-cFDSYU4dED45oOM Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:24.894674 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 8bbc891924f84d99aac22518f69e7b99 Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.896338 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'c10c829d-ef62-4d1c-98e9-422615414c5e'} Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:24.897295 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.898939 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.900503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Returning 404 to user: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:24.900503 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-80d7cb5d-2cd6-41d5-af84-9defea3e606b tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 404 len: 111 microversion: 2.1 time: 0.055367 Oct 15 20:54:24.900503 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 972/1819] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:24 2022] DELETE /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:24.905270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.905897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: c10c829d-ef62-4d1c-98e9-422615414c5e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.908416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.909937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.910502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.911744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.912219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.912434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.912710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.913098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.913224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.913949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.914473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.915049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.931460 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] HTTP exception thrown: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Request method failure captured: Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e HTTP/1.1 Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2ANl7AV4DUGTAmJCJrJoBUGOPLq38faLYUZfcViDfXVEvnA_PNoZJFdNze6Vt9ihvNbHykKXsI_OpFPpZY5X2AN7w4Xgg5TdRRgfn8pIEhKZPCdQbqV3CubFinqCtxxFbMVTmObo3tgEYJbzoNGy9bnJvpixa-cFDSYU4dED45oOM Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:24.933148 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 8bbc891924f84d99aac22518f69e7b99 Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.935435 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'c10c829d-ef62-4d1c-98e9-422615414c5e'} Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:24.936354 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. Oct 15 20:54:24.938209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.939951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Returning 404 to user: Instance c10c829d-ef62-4d1c-98e9-422615414c5e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:24.939951 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c2072fa5-0486-4295-a191-ef2f7baafd97 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e" status: 404 len: 111 microversion: 2.1 time: 0.033552 Oct 15 20:54:24.939951 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 973/1820] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:24 2022] GET /compute/v2.1/servers/c10c829d-ef62-4d1c-98e9-422615414c5e => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:24.944131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.945315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.952665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.953277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.953723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.954438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.954900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.955493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.962817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:54:24.976506 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] HTTP exception thrown: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Request method failure captured: Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac HTTP/1.1 Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2ANl7AV4DUGTAmJCJrJoBUGOPLq38faLYUZfcViDfXVEvnA_PNoZJFdNze6Vt9ihvNbHykKXsI_OpFPpZY5X2AN7w4Xgg5TdRRgfn8pIEhKZPCdQbqV3CubFinqCtxxFbMVTmObo3tgEYJbzoNGy9bnJvpixa-cFDSYU4dED45oOM Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:24.978372 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 8bbc891924f84d99aac22518f69e7b99 Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:24.979886 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'aebc2aa6-a3c4-45d7-afd8-1503dbc64cac'} Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:24.980833 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:24.983334 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:24.984863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Returning 404 to user: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:24.984863 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e24f6663-1aa5-4c34-a3f9-aaa691e6a065 tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "DELETE /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 404 len: 111 microversion: 2.1 time: 0.039611 Oct 15 20:54:24.984863 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 974/1821] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:24 2022] DELETE /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:24.989591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:24.990211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] [instance: aebc2aa6-a3c4-45d7-afd8-1503dbc64cac] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:24.996601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.997698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.998144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:24.998955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:24.999401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:24.999828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.016200 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] HTTP exception thrown: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Request method failure captured: Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac HTTP/1.1 Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx2ANl7AV4DUGTAmJCJrJoBUGOPLq38faLYUZfcViDfXVEvnA_PNoZJFdNze6Vt9ihvNbHykKXsI_OpFPpZY5X2AN7w4Xgg5TdRRgfn8pIEhKZPCdQbqV3CubFinqCtxxFbMVTmObo3tgEYJbzoNGy9bnJvpixa-cFDSYU4dED45oOM Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:25.018036 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1c30aa11fd244564be716550bfe5aaa6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 1c30aa11fd244564be716550bfe5aaa6 Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersTestMultiNic-246188881 Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 8bbc891924f84d99aac22518f69e7b99 Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersTestMultiNic-246188881-project Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:25.019657 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'aebc2aa6-a3c4-45d7-afd8-1503dbc64cac'} Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:25.020667 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. Oct 15 20:54:25.022742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.024367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] Returning 404 to user: Instance aebc2aa6-a3c4-45d7-afd8-1503dbc64cac could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:25.024367 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9736b22a-dee4-4b6a-908c-21bfc46843be tempest-ServersTestMultiNic-246188881 tempest-ServersTestMultiNic-246188881-project] 10.222.0.16 "GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac" status: 404 len: 111 microversion: 2.1 time: 0.035347 Oct 15 20:54:25.025781 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 975/1822] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:24 2022] GET /compute/v2.1/servers/aebc2aa6-a3c4-45d7-afd8-1503dbc64cac => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:25.046232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:25.088808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:25.089621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:25.112015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.113622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.115810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.131091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.132808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.135478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.139568 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f411155e-a067-462d-93a7-43fd0af471ac tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 204 len: 0 microversion: 2.1 time: 0.252964 Oct 15 20:54:25.144845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 848/1823] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:24 2022] DELETE /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:25.152015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:25.153961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:25.163484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.164583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.165635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.166775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.167407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.168027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.456970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:25.472431 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2853c67b-a42b-4704-8d66-86a3a6f75250 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.387842 Oct 15 20:54:25.474629 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 976/1824] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:25 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:25.629721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:25.642286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Action: 'update', calling method: .version_select at 0x7fe74f279e50>, body: {"status": "disabled", "disabled_reason": "maintenance"} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:25.646539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.647186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.647711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.656769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.657406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.657894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.659492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:25.665728 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] HTTP exception thrown: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Request method failure captured: Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb HTTP/1.1 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 56 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4AYtljHvCCcgzHGzTWDA4AvUU5TD4RP7vxdlecwjcKJBG5c3wsatn0Qz07WxYi85P4-TUH5YS9y1xR82QOlnxeMWf6rw9T2Kc7ddAKKROTzblKoHo1cd-S_XD-qUrxGqarhsN0nLE_7rWhIhepz9ozH34DIfsrgD0bEkbIzE07BSw Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Openstack-Nova-Api-Version: 2.53 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 752227df24174003a236d784f047db75 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:54:25.678438 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/752227df24174003a236d784f047db75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_752227df24174003a236d784f047db75"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/752227df24174003a236d784f047db75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/752227df24174003a236d784f047db75"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 752227df24174003a236d784f047db75 Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: d8f9414680164912ba902c7a6f2a2f83 Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: {"status": "disabled", "disabled_reason": "maintenance"} Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:54:25.680242 ncdv-master-1 devstack@n-api.service[138689]: exception: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '4baee2f8-54c4-4f55-b2ea-3d41130f46fb', 'body': {'status': 'disabled', 'disabled_reason': 'maintenance'}} Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6207, in service_get_by_id Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 436, in update Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6209, in service_get_by_id Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.681179 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 441, in update Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:25.682608 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:25.683986 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:25.683986 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:25.683986 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:25.683986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Returning 404 to user: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:25.683986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d2864983-d19c-4d04-b9a3-dbb94077f89c tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb" status: 404 len: 110 microversion: 2.53 time: 0.204669 Oct 15 20:54:25.686610 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 977/1825] 10.222.0.16 () {62 vars in 1309 bytes} [Sat Oct 15 20:54:25 2022] PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb => generated 110 bytes in 211 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:54:25.694008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:25.694630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:25.702033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.702534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.702977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.703699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.704167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.704664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.913565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:25.923309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:25.924799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:25.926010 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9447e86f-7715-44da-a092-0df9cb44d945 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.238063 Oct 15 20:54:25.937067 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 978/1826] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:25 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:25.937067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:25.937067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:25.945071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.945071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.945071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:25.945071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:25.945531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:25.945531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.043701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:26.063610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:26.073350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bcb2176-82be-4bb3-9264-e85d7eaaaae1 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1473 microversion: 2.1 time: 0.926292 Oct 15 20:54:26.087208 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 849/1827] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:25 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1473 bytes in 941 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:26.093807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:26.100023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Action: 'update', calling method: .version_select at 0x7fe75846e310>, body: {"status": "disabled"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:26.104743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.105498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.106056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.108969 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b2fdb35-849d-4e05-a93d-221bd9c60cfe tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.181280 Oct 15 20:54:26.109905 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 979/1828] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:25 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:26.115173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.115798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:26.116690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.117731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.118585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.122170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.122802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.123507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.124226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.124687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.125148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.145958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] HTTP exception thrown: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Request method failure captured: Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb HTTP/1.1 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 22 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4AYtljHvCCcgzHGzTWDA4AvUU5TD4RP7vxdlecwjcKJBG5c3wsatn0Qz07WxYi85P4-TUH5YS9y1xR82QOlnxeMWf6rw9T2Kc7ddAKKROTzblKoHo1cd-S_XD-qUrxGqarhsN0nLE_7rWhIhepz9ozH34DIfsrgD0bEkbIzE07BSw Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.53 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 752227df24174003a236d784f047db75 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:54:26.151125 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/752227df24174003a236d784f047db75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_752227df24174003a236d784f047db75"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/752227df24174003a236d784f047db75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/752227df24174003a236d784f047db75"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 752227df24174003a236d784f047db75 Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d8f9414680164912ba902c7a6f2a2f83 Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: {"status": "disabled"} Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: exception: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.152876 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4baee2f8-54c4-4f55-b2ea-3d41130f46fb', 'body': {'status': 'disabled'}} Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6207, in service_get_by_id Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 436, in update Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6209, in service_get_by_id Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.154999 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 441, in update Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:26.156626 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:26.158351 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.158351 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.172103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Returning 404 to user: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:26.174446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f31fc320-5b2e-4ffa-a627-7c7b3b5f0b5d tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb" status: 404 len: 110 microversion: 2.53 time: 0.084956 Oct 15 20:54:26.175579 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 850/1829] 10.222.0.16 () {62 vars in 1309 bytes} [Sat Oct 15 20:54:26 2022] PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb => generated 110 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:54:26.183055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.186666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:26.207629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.208336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.209064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.209944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.210489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.210983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.259430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:DELETE=2,nova_api:SELECT=92,nova_api:UPDATE=3,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:26.263944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.264539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.265092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.265588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lazy-loading 'fault' on Instance uuid 0028073f-e418-41fb-830e-0c49db2dee33 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:26.298854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:26.307050 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a7def548-a899-4ac0-aca5-70d8ea588158 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1366 microversion: 2.1 time: 0.195997 Oct 15 20:54:26.307620 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 980/1830] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:26.312874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5e8d8320-dda7-4001-ad09-063e71da40a2 req-65ab868d-5caa-4598-b3a7-ab20264203b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0028073f-e418-41fb-830e-0c49db2dee33", "name": "network-vif-deleted", "tag": "41a31112-fc5d-4e74-9274-0be89a46346e"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:26.327291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5e8d8320-dda7-4001-ad09-063e71da40a2 req-65ab868d-5caa-4598-b3a7-ab20264203b6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.327912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5e8d8320-dda7-4001-ad09-063e71da40a2 req-65ab868d-5caa-4598-b3a7-ab20264203b6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.328541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5e8d8320-dda7-4001-ad09-063e71da40a2 req-65ab868d-5caa-4598-b3a7-ab20264203b6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.328984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ddcfe1b0eb5431dbbe4777b8dd77436', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b107efbbeb064e1fbf3c8bf31da12cbe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:26.339828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6def95b1-e2ab-47b3-9f22-53a8d1b71936 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 200 len: 1307 microversion: 2.1 time: 0.162667 Oct 15 20:54:26.340827 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 851/1831] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 1307 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:26.347333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Action: 'update', calling method: .version_select at 0x7fe7584c0e50>, body: {"status": "enabled"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:26.352503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.353780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.355874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.359780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5e8d8320-dda7-4001-ad09-063e71da40a2 req-65ab868d-5caa-4598-b3a7-ab20264203b6 service nova] Creating event network-vif-deleted:41a31112-fc5d-4e74-9274-0be89a46346e for instance 0028073f-e418-41fb-830e-0c49db2dee33 on nch1-master-1 Oct 15 20:54:26.363892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.364391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.364993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.366467 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5e8d8320-dda7-4001-ad09-063e71da40a2 req-65ab868d-5caa-4598-b3a7-ab20264203b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057705 Oct 15 20:54:26.367235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 981/1832] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:26.371080 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] HTTP exception thrown: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Request method failure captured: Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb HTTP/1.1 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 21 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4AYtljHvCCcgzHGzTWDA4AvUU5TD4RP7vxdlecwjcKJBG5c3wsatn0Qz07WxYi85P4-TUH5YS9y1xR82QOlnxeMWf6rw9T2Kc7ddAKKROTzblKoHo1cd-S_XD-qUrxGqarhsN0nLE_7rWhIhepz9ozH34DIfsrgD0bEkbIzE07BSw Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.53 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 752227df24174003a236d784f047db75 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:54:26.373267 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/752227df24174003a236d784f047db75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_752227df24174003a236d784f047db75"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/752227df24174003a236d784f047db75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/752227df24174003a236d784f047db75"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 752227df24174003a236d784f047db75 Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217 Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d8f9414680164912ba902c7a6f2a2f83 Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: {"status": "enabled"} Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: exception: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.374933 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4baee2f8-54c4-4f55-b2ea-3d41130f46fb', 'body': {'status': 'enabled'}} Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6207, in service_get_by_id Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 436, in update Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6209, in service_get_by_id Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.375872 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 441, in update Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:26.377237 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:26.379201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. Oct 15 20:54:26.379201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.379201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Returning 404 to user: Service 4baee2f8-54c4-4f55-b2ea-3d41130f46fb could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:26.379201 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51babd27-4b5e-4e57-85e5-7a6f83751f16 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb" status: 404 len: 110 microversion: 2.53 time: 0.031072 Oct 15 20:54:26.379201 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 852/1833] 10.222.0.16 () {62 vars in 1309 bytes} [Sat Oct 15 20:54:26 2022] PUT /compute/v2.1/os-services/4baee2f8-54c4-4f55-b2ea-3d41130f46fb => generated 110 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:54:26.385483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.389752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.390400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.390818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.391546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.392023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.392441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.416602 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3cdb1f4a-08d2-4c71-be8b-97a79d383c03 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.035552 Oct 15 20:54:26.417227 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 982/1834] 10.222.0.16 () {60 vars in 1178 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:26.432086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.435917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.436431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.436867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.437675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.438138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.438560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.465107 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-833630fe-088e-4dd5-97f9-c8180be564b2 tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.037567 Oct 15 20:54:26.466330 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 853/1835] 10.222.0.16 () {60 vars in 1211 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services?xxx=nova-compute => generated 1242 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:26.484965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.488535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.489120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.489793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.490626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.491157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.491639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.500505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.501844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:26.510351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.511214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.511975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.513650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.514534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.514961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.520858 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a3bba4fb-8f60-4eb2-a6eb-7cbccf7ef71a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.039695 Oct 15 20:54:26.521492 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 983/1836] 10.222.0.16 () {60 vars in 1178 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:26.536291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.540500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.541473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.542156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.543095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.544097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.544529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.570892 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ebc4fb93-c921-465f-9a22-483c2744bc3b tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx" status: 200 len: 16 microversion: 2.53 time: 0.038658 Oct 15 20:54:26.573088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 984/1837] 10.222.0.16 () {60 vars in 1237 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx => generated 16 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:26.586632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.590370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.590955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.591416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.592649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.592779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.593190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.624021 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-96e2c6ce-3b25-4fb2-87f0-2cc7b1c8aa2a tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.042078 Oct 15 20:54:26.632716 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 985/1838] 10.222.0.16 () {60 vars in 1178 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:26.639146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.643179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.643670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.644104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.645021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.645499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.645928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.661111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:26.674919 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-34b39f6a-512b-4a91-b764-745e9ea2e3ae tempest-ServicesAdminNegativeV253TestJSON-274862217 tempest-ServicesAdminNegativeV253TestJSON-274862217-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler" status: 200 len: 16 microversion: 2.53 time: 0.040726 Oct 15 20:54:26.675621 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 986/1839] 10.222.0.16 () {60 vars in 1239 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler => generated 16 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:26.789097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1d7e3dc-5f89-4575-8e47-201b157bf61d tempest-ServicesAdminNegativeV253TestJSON-1573593321 tempest-ServicesAdminNegativeV253TestJSON-1573593321-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.792988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d1d7e3dc-5f89-4575-8e47-201b157bf61d tempest-ServicesAdminNegativeV253TestJSON-1573593321 tempest-ServicesAdminNegativeV253TestJSON-1573593321-project] Policy check for os_compute_api:os-services:list failed with credentials {'is_admin': False, 'user_id': '93a6541758aa4eefb5c5c2407a899943', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b53a4be470b7451b88037a82e4657a12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1d7e3dc-5f89-4575-8e47-201b157bf61d tempest-ServicesAdminNegativeV253TestJSON-1573593321 tempest-ServicesAdminNegativeV253TestJSON-1573593321-project] Request method failure captured: Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-services HTTP/1.1 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4A564ax8npxgdLDpjMtELZWxqMie7vSiS8CT7_6R3R_WHtkTWPE_WdcTWK4KVBsUJAhtssXntz5WhOcjUSUYG8IHzVwZsUtaIb0IUIGOiY9qsOtNIpLtX8UZXVQweumA79-Qo-j5bIxucmyucu0js0yfCohXjFviRXCYkK0HLx1_c Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Openstack-Nova-Api-Version: 2.53 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b53a4be470b7451b88037a82e4657a12 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-1573593321 Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:26.806736 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b53a4be470b7451b88037a82e4657a12"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b53a4be470b7451b88037a82e4657a12"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b53a4be470b7451b88037a82e4657a12"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b53a4be470b7451b88037a82e4657a12"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-1573593321 Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b53a4be470b7451b88037a82e4657a12 Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-1573593321 Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServicesAdminNegativeV253TestJSON-1573593321-project Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 93a6541758aa4eefb5c5c2407a899943 Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-1573593321-project Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: exception: Policy doesn't allow os_compute_api:os-services:list to be performed. Oct 15 20:54:26.808376 ncdv-master-1 devstack@n-api.service[138689]: action_args: {} Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 384, in index Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi context.can(services_policies.BASE_POLICY_NAME % 'list', target={}) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:54:26.810685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-services:list to be performed. Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-services:list to be performed. Oct 15 20:54:26.812715 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:26.814159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1d7e3dc-5f89-4575-8e47-201b157bf61d tempest-ServicesAdminNegativeV253TestJSON-1573593321 tempest-ServicesAdminNegativeV253TestJSON-1573593321-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-services:list to be performed. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:26.814159 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d1d7e3dc-5f89-4575-8e47-201b157bf61d tempest-ServicesAdminNegativeV253TestJSON-1573593321 tempest-ServicesAdminNegativeV253TestJSON-1573593321-project] 10.222.0.16 "GET /compute/v2.1/os-services" status: 403 len: 112 microversion: 2.53 time: 0.124654 Oct 15 20:54:26.814159 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 987/1840] 10.222.0.16 () {60 vars in 1178 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/os-services => generated 112 bytes in 126 msecs (HTTP/1.1 403) 9 headers in 381 bytes (2 switches on core 0) Oct 15 20:54:26.930133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:26.941520 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a88ed0be-81e0-4322-9f13-f41b89de8d79 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1717 microversion: 2.1 time: 0.445690 Oct 15 20:54:26.942767 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 854/1841] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1717 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:26.955795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:26.956511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:26.964817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.965491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.965930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:26.966730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:26.967190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:26.967614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.098644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:27.099550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:27.108693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.113478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.114263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.115230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.115916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.116624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.146340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:27.164040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:27.167754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:27.171596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3415813a-12f2-4992-811b-e0fb80e0fecc tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.221237 Oct 15 20:54:27.175608 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 988/1842] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:26 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:27.194182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:27.199372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:27.210344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.210942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.211390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.212280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.212763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.213227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.230559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:27.422621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:27.431653 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4df4939d-fa16-40d2-94e9-4f225c8ec417 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1370 microversion: 2.1 time: 0.254765 Oct 15 20:54:27.432514 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 989/1843] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:27 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1370 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:27.437992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-662d6a2d-66fe-4b8d-8679-cc0bc8fed393 req-d51aad1c-c3bd-4238-aa60-8c50c7ab6982 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "062c4651-90be-430a-a69b-372fac258ebb", "name": "network-vif-plugged", "status": "completed", "tag": "e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:27.448444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-662d6a2d-66fe-4b8d-8679-cc0bc8fed393 req-d51aad1c-c3bd-4238-aa60-8c50c7ab6982 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.449667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-662d6a2d-66fe-4b8d-8679-cc0bc8fed393 req-d51aad1c-c3bd-4238-aa60-8c50c7ab6982 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.451453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-662d6a2d-66fe-4b8d-8679-cc0bc8fed393 req-d51aad1c-c3bd-4238-aa60-8c50c7ab6982 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.464319 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-662d6a2d-66fe-4b8d-8679-cc0bc8fed393 req-d51aad1c-c3bd-4238-aa60-8c50c7ab6982 service nova] Creating event network-vif-plugged:e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8 for instance 062c4651-90be-430a-a69b-372fac258ebb on nch1-master-1 Oct 15 20:54:27.470939 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-662d6a2d-66fe-4b8d-8679-cc0bc8fed393 req-d51aad1c-c3bd-4238-aa60-8c50c7ab6982 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036862 Oct 15 20:54:27.471769 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 990/1844] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:27.479775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:27.480928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:27.489865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.491226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.491921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.492959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.493857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.494530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.616150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:27.627619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5dfa6151-68f6-43ed-b92a-e1be2a844ac4 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1473 microversion: 2.1 time: 0.533375 Oct 15 20:54:27.628234 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 855/1845] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:27 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1473 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:27.645543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:27.646171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] [instance: 0028073f-e418-41fb-830e-0c49db2dee33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:27.657639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.658131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.658619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.659291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.662013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.662467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:27.698854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] HTTP exception thrown: Instance 0028073f-e418-41fb-830e-0c49db2dee33 could not be found. Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Request method failure captured: Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 HTTP/1.1 Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0WHnLvc1BGzpbGMpoOdp3_s1TZiO3Nd00xY4tMqbhNiTAtBedqSdsaW_39NtQOxU61GiVg4TB0xvUcRwxfS3EP1lHdS3mq9Ft-UGD8F_IG8LyoMPVybgWIftNq4eoqyCKZ6Jldm5MOIjP4uE5VL3TcJ8eParvfc6SnB7Xcw7qGxpk Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:54:27.701214 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b107efbbeb064e1fbf3c8bf31da12cbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b107efbbeb064e1fbf3c8bf31da12cbe Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-45324467 Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1ddcfe1b0eb5431dbbe4777b8dd77436 Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersOnMultiNodesTest-45324467-project Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 0028073f-e418-41fb-830e-0c49db2dee33 could not be found. Oct 15 20:54:27.702725 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0028073f-e418-41fb-830e-0c49db2dee33'} Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0028073f-e418-41fb-830e-0c49db2dee33 could not be found. Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:27.703735 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0028073f-e418-41fb-830e-0c49db2dee33 could not be found. Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0028073f-e418-41fb-830e-0c49db2dee33 could not be found. Oct 15 20:54:27.707449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:27.710920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] Returning 404 to user: Instance 0028073f-e418-41fb-830e-0c49db2dee33 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:27.710920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe9c050a-051f-43c4-b848-55d741769b29 tempest-ServersOnMultiNodesTest-45324467 tempest-ServersOnMultiNodesTest-45324467-project] 10.222.0.16 "GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33" status: 404 len: 111 microversion: 2.1 time: 0.073577 Oct 15 20:54:27.710920 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 856/1846] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:27 2022] GET /compute/v2.1/servers/0028073f-e418-41fb-830e-0c49db2dee33 => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:54:27.863370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:27.876159 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17eaa1a4-0c5d-4585-80e2-1876182f6324 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1366 microversion: 2.1 time: 0.402398 Oct 15 20:54:27.879349 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 991/1847] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:27 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1366 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:27.967963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:27.968588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:27.992160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:27.996047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:27.997319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:28.005556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:28.009077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:28.010616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:28.208976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:28.209871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:28.245301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:28.253813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:28.254887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:28.257264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:28.258205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:28.258843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:28.442411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:28.453272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-891858e0-5a1f-4332-863d-a3496758127f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1707 microversion: 2.1 time: 0.490176 Oct 15 20:54:28.481584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 857/1848] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:27 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1707 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:28.487769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:28.488392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:28.503038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:28.503859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:28.504538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:28.506249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:28.508852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:28.509990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:28.947377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:28.957770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c455a9af-8aab-4674-94f1-0f8093ce8cab tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1555 microversion: 2.1 time: 0.475073 Oct 15 20:54:28.958360 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 858/1849] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:28 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1555 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:28.971329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da2bb39a-092c-4aeb-ab9b-012efb580246 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '.version_select at 0x7fe75845e310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:28.984336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2bb39a-092c-4aeb-ab9b-012efb580246 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:28.985199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2bb39a-092c-4aeb-ab9b-012efb580246 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:28.988680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2bb39a-092c-4aeb-ab9b-012efb580246 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.002280 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da2bb39a-092c-4aeb-ab9b-012efb580246 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 2319 microversion: 2.1 time: 0.040379 Oct 15 20:54:29.015723 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 859/1850] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:54:28 2022] GET /compute/v2.1/os-migrations => generated 2319 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:29.023238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:29.024468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:29.031244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:29.041585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.042351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.043365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.045018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.045751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.046344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.047794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:29.051979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:29.054748 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fd854b0e-2458-4e33-96d3-93dbf7b4d4e7 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.851552 Oct 15 20:54:29.058105 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 992/1851] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:28 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 856 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:29.063350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:29.067924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:29.078227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.079071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.079733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.080673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.081385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.082069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.251621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:29.260552 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bdaed6e9-f577-4d0b-9ee8-0f22ef16778f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1366 microversion: 2.1 time: 0.201271 Oct 15 20:54:29.277646 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 993/1852] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:29 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1366 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:29.292329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:29.299059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:29.306676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.310907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.318506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:29.319045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.320210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.320916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.321632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.341678 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb7238e8-d041-4418-b573-cfc159611dfe tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1431 microversion: 2.1 time: 0.320654 Oct 15 20:54:29.344545 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 860/1853] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:29 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1431 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:29.358533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:29.359485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:29.371660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.373500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.374165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.375103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.375570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.376004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.463982 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-34c9830a-ef6e-458e-90fc-dfd1f61a99f5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.184978 Oct 15 20:54:29.464927 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 994/1854] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:54:29 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-interface => generated 284 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:29.470789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-031f6a69-ec10-40c3-9c41-6f680588ba3b req-30f6e9ed-ee43-4b0c-983f-a8a73a161b96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9be47b55-0549-4005-b37e-52219a6e7c11", "name": "network-vif-deleted", "tag": "26a1de8d-3364-49cb-9ceb-59656f3de4af"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:29.503874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-031f6a69-ec10-40c3-9c41-6f680588ba3b req-30f6e9ed-ee43-4b0c-983f-a8a73a161b96 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:29.504860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-031f6a69-ec10-40c3-9c41-6f680588ba3b req-30f6e9ed-ee43-4b0c-983f-a8a73a161b96 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:29.509677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-031f6a69-ec10-40c3-9c41-6f680588ba3b req-30f6e9ed-ee43-4b0c-983f-a8a73a161b96 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:29.540710 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-031f6a69-ec10-40c3-9c41-6f680588ba3b req-30f6e9ed-ee43-4b0c-983f-a8a73a161b96 service nova] Creating event network-vif-deleted:26a1de8d-3364-49cb-9ceb-59656f3de4af for instance 9be47b55-0549-4005-b37e-52219a6e7c11 on nch2-master-1 Oct 15 20:54:29.549171 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-031f6a69-ec10-40c3-9c41-6f680588ba3b req-30f6e9ed-ee43-4b0c-983f-a8a73a161b96 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082188 Oct 15 20:54:29.550057 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 995/1855] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:30.005579 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da3c8d31-4476-4df2-91bd-b00c9591547a tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1858 microversion: 2.1 time: 0.659581 Oct 15 20:54:30.009443 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 861/1856] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:29 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1858 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:30.037474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:30.038810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:30.052197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:30.052710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:30.053177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:30.053904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:30.054362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:30.054789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:30.099777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:30.102476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:30.120081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:30.121797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:30.123931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:30.125181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:30.125908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:30.127265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:30.540880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:30.550781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:30.558219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:30.561681 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e758aac5-e9ab-4754-a21b-cb977c3df620 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.478872 Oct 15 20:54:30.563500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 862/1857] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:30 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:30.595329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:30.601404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:30.615070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:30.615885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:30.616547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:30.618127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:30.620390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:30.621129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.197769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:31.207423 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-061d0abc-4851-4c48-b70b-eaebb97bf480 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1555 microversion: 2.1 time: 0.626177 Oct 15 20:54:31.208010 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 863/1858] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:30 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1555 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:31.223303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:31.226031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:31.245695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.247226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.247881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.248794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.249512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.250114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.263098 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17f9ab2f-807a-4c62-a3ad-aca28ea3ba23 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1858 microversion: 2.1 time: 1.228564 Oct 15 20:54:31.264204 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 996/1859] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:30 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1858 bytes in 1234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:31.272569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:31.273099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:31.289914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.290178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.293841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.298125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.299932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.300606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.316958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.320784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.321527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.329745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lazy-loading 'fault' on Instance uuid 9be47b55-0549-4005-b37e-52219a6e7c11 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:31.462204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3e76b25814474b9db035954b3a437c96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5c50f30f706437ea737a3d5d20ea8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:31.470279 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e7157823-cf93-4b4e-b5bf-f4e6eeb9ac7f tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 200 len: 1307 microversion: 2.1 time: 0.256808 Oct 15 20:54:31.475842 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 864/1860] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:31 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 1307 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:31.522713 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8073e6bb-4b6a-44f8-b902-82e5983aa9fe tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.257332 Oct 15 20:54:31.531246 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 997/1861] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:54:31 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-interface => generated 284 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:31.598186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:31.598716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:31.613168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.613676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.614124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.614852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.615309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.615745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.717814 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d9df9aa5-e6be-4df7-94a2-1a959ab793a1 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] 10.222.0.16 "DELETE /compute/v2.1/images/" status: 404 len: 112 microversion: - time: 0.166426 Oct 15 20:54:31.719044 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 865/1862] 10.222.0.16 () {58 vars in 1128 bytes} [Sat Oct 15 20:54:31 2022] DELETE /compute/v2.1/images/ => generated 112 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 15 20:54:31.736152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bab46d6a-b300-486c-9e51-20ea436e251b tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Calling method '.version_select at 0x7fe7583efdc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:31.845940 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-bab46d6a-b300-486c-9e51-20ea436e251b tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] HTTP exception thrown: Image not found. Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bab46d6a-b300-486c-9e51-20ea436e251b tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Request method failure captured: Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/-1 HTTP/1.1 Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4HIsAPepJ3fsM6cpfQEJeInIHDL6sB6xTtfHjGQn_abWRij_cwiNB1NP9z3IpeJmkLWB34x8Ki8R372XDwUiOWUtS6x-02BB-VKUUoqD1htTYpdlTwoDrQfz4epcpgjgEp8SLQ0LSmP-Qe9Z1gNBzkvUgcgg_VgAbeXuRhR0wHu9I Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:54:31.900721 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 74a3aef2aae044f088fca1b096db33fb Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '-1'} Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:31.902430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image -1 to delete Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.904035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image -1 could not be found. Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:31.908162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:54:31.910894 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:54:31.916175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bab46d6a-b300-486c-9e51-20ea436e251b tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:31.917395 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bab46d6a-b300-486c-9e51-20ea436e251b tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] 10.222.0.16 "DELETE /compute/v2.1/images/-1" status: 404 len: 62 microversion: 2.1 time: 0.193278 Oct 15 20:54:31.919796 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 866/1863] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 20:54:31 2022] DELETE /compute/v2.1/images/-1 => generated 62 bytes in 195 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 15 20:54:31.936236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ceae9459-8afa-4821-a997-84d83d427c4b req-f2ad4c45-eef8-4e80-9564-aa288b547824 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82e74d13-2be4-47d0-93fa-fb24eb81c212", "tag": "328d28c6-9ece-4b0b-b5f5-e870e5285550"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:31.948466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:31.958834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:31.962107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ceae9459-8afa-4821-a997-84d83d427c4b req-f2ad4c45-eef8-4e80-9564-aa288b547824 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:31.962260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:31.962938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ceae9459-8afa-4821-a997-84d83d427c4b req-f2ad4c45-eef8-4e80-9564-aa288b547824 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:31.963487 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-274b142d-167f-44d8-a0b1-43ddfd540d1c tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.369298 Oct 15 20:54:31.963966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ceae9459-8afa-4821-a997-84d83d427c4b req-f2ad4c45-eef8-4e80-9564-aa288b547824 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:31.980679 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 998/1864] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:31 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:31.981278 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ceae9459-8afa-4821-a997-84d83d427c4b req-f2ad4c45-eef8-4e80-9564-aa288b547824 service nova] Creating event network-changed:328d28c6-9ece-4b0b-b5f5-e870e5285550 for instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 on nch2-master-1 Oct 15 20:54:31.990280 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ceae9459-8afa-4821-a997-84d83d427c4b req-f2ad4c45-eef8-4e80-9564-aa288b547824 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068696 Oct 15 20:54:31.991242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 867/1865] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:32.009607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f303d1c3-c8ef-4ab1-82dd-e847565109d3 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Calling method '.version_select at 0x7fe74f1525e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.060604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.063551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:32.082914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:32.085372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:32.087715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:32.088032 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-f303d1c3-c8ef-4ab1-82dd-e847565109d3 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] HTTP exception thrown: Image not found. Oct 15 20:54:32.089725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:32.090366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:32.091508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f303d1c3-c8ef-4ab1-82dd-e847565109d3 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Request method failure captured: Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/images/5c1f4249-66ad-441d-9c80-ff7a3b75695j HTTP/1.1 Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4HIsAPepJ3fsM6cpfQEJeInIHDL6sB6xTtfHjGQn_abWRij_cwiNB1NP9z3IpeJmkLWB34x8Ki8R372XDwUiOWUtS6x-02BB-VKUUoqD1htTYpdlTwoDrQfz4epcpgjgEp8SLQ0LSmP-Qe9Z1gNBzkvUgcgg_VgAbeXuRhR0wHu9I Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:32.093163 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 74a3aef2aae044f088fca1b096db33fb Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: exception: Image not found. Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '5c1f4249-66ad-441d-9c80-ff7a3b75695j'} Oct 15 20:54:32.095007 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 5c1f4249-66ad-441d-9c80-ff7a3b75695j to delete Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.096123 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 5c1f4249-66ad-441d-9c80-ff7a3b75695j could not be found. Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:32.097657 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f303d1c3-c8ef-4ab1-82dd-e847565109d3 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Returning 404 to user: Image not found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:32.101110 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f303d1c3-c8ef-4ab1-82dd-e847565109d3 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] 10.222.0.16 "DELETE /compute/v2.1/images/5c1f4249-66ad-441d-9c80-ff7a3b75695j" status: 404 len: 62 microversion: 2.1 time: 0.112798 Oct 15 20:54:32.102798 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 999/1866] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:31 2022] DELETE /compute/v2.1/images/5c1f4249-66ad-441d-9c80-ff7a3b75695j => generated 62 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:54:32.109573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e1598efd-644b-4228-8876-5faaaa6c07e7 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Calling method '.version_select at 0x7fe74f235670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.131963 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-e1598efd-644b-4228-8876-5faaaa6c07e7 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] HTTP exception thrown: Image not found. Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e1598efd-644b-4228-8876-5faaaa6c07e7 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Request method failure captured: Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/images/95c07837-b097-497f-bba6-5cd191ed22891 HTTP/1.1 Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4HIsAPepJ3fsM6cpfQEJeInIHDL6sB6xTtfHjGQn_abWRij_cwiNB1NP9z3IpeJmkLWB34x8Ki8R372XDwUiOWUtS6x-02BB-VKUUoqD1htTYpdlTwoDrQfz4epcpgjgEp8SLQ0LSmP-Qe9Z1gNBzkvUgcgg_VgAbeXuRhR0wHu9I Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:32.133790 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 74a3aef2aae044f088fca1b096db33fb Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: exception: Image not found. Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '95c07837-b097-497f-bba6-5cd191ed22891'} Oct 15 20:54:32.135410 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 95c07837-b097-497f-bba6-5cd191ed22891 to delete Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.136328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 95c07837-b097-497f-bba6-5cd191ed22891 could not be found. Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:32.137890 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e1598efd-644b-4228-8876-5faaaa6c07e7 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Returning 404 to user: Image not found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:32.139198 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e1598efd-644b-4228-8876-5faaaa6c07e7 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] 10.222.0.16 "DELETE /compute/v2.1/images/95c07837-b097-497f-bba6-5cd191ed22891" status: 404 len: 62 microversion: 2.1 time: 0.034669 Oct 15 20:54:32.140200 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1000/1867] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:54:32 2022] DELETE /compute/v2.1/images/95c07837-b097-497f-bba6-5cd191ed22891 => generated 62 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 15 20:54:32.146734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0387a8a5-be20-4af4-b379-c2b4e937e02c tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Calling method '.version_select at 0x7fe74f1529d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.183772 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-0387a8a5-be20-4af4-b379-c2b4e937e02c tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] HTTP exception thrown: Image not found. Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0387a8a5-be20-4af4-b379-c2b4e937e02c tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Request method failure captured: Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/images/tempest-invalid-1648243001 HTTP/1.1 Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4HIsAPepJ3fsM6cpfQEJeInIHDL6sB6xTtfHjGQn_abWRij_cwiNB1NP9z3IpeJmkLWB34x8Ki8R372XDwUiOWUtS6x-02BB-VKUUoqD1htTYpdlTwoDrQfz4epcpgjgEp8SLQ0LSmP-Qe9Z1gNBzkvUgcgg_VgAbeXuRhR0wHu9I Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:32.186367 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 74a3aef2aae044f088fca1b096db33fb Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: exception: Image not found. Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'tempest-invalid-1648243001'} Oct 15 20:54:32.188069 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image tempest-invalid-1648243001 to delete Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.189153 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image tempest-invalid-1648243001 could not be found. Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:32.190607 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.193304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0387a8a5-be20-4af4-b379-c2b4e937e02c tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Returning 404 to user: Image not found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:32.195353 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0387a8a5-be20-4af4-b379-c2b4e937e02c tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] 10.222.0.16 "DELETE /compute/v2.1/images/tempest-invalid-1648243001" status: 404 len: 62 microversion: 2.1 time: 0.053867 Oct 15 20:54:32.196730 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1001/1868] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 20:54:32 2022] DELETE /compute/v2.1/images/tempest-invalid-1648243001 => generated 62 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 15 20:54:32.208985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef70c011-e704-4586-bd03-213272f1a1e4 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Calling method '.version_select at 0x7fe74f1529d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.239161 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-ef70c011-e704-4586-bd03-213272f1a1e4 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] HTTP exception thrown: Image not found. Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef70c011-e704-4586-bd03-213272f1a1e4 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Request method failure captured: Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/images/786764d7-d02a-4cc4-b1e3-d71b527bd4bd HTTP/1.1 Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4HIsAPepJ3fsM6cpfQEJeInIHDL6sB6xTtfHjGQn_abWRij_cwiNB1NP9z3IpeJmkLWB34x8Ki8R372XDwUiOWUtS6x-02BB-VKUUoqD1htTYpdlTwoDrQfz4epcpgjgEp8SLQ0LSmP-Qe9Z1gNBzkvUgcgg_VgAbeXuRhR0wHu9I Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:32.241051 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b8151ebb1b14413381064a8a8d4352c8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b8151ebb1b14413381064a8a8d4352c8 Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352 Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 74a3aef2aae044f088fca1b096db33fb Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1955815352-project Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: exception: Image not found. Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '786764d7-d02a-4cc4-b1e3-d71b527bd4bd'} Oct 15 20:54:32.242512 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 786764d7-d02a-4cc4-b1e3-d71b527bd4bd to delete Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.243508 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 786764d7-d02a-4cc4-b1e3-d71b527bd4bd could not be found. Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:32.244813 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef70c011-e704-4586-bd03-213272f1a1e4 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] Returning 404 to user: Image not found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:32.246238 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef70c011-e704-4586-bd03-213272f1a1e4 tempest-ImagesDeleteNegativeTestJSON-1955815352 tempest-ImagesDeleteNegativeTestJSON-1955815352-project] 10.222.0.16 "DELETE /compute/v2.1/images/786764d7-d02a-4cc4-b1e3-d71b527bd4bd" status: 404 len: 62 microversion: 2.1 time: 0.040087 Oct 15 20:54:32.249347 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1002/1869] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:32 2022] DELETE /compute/v2.1/images/786764d7-d02a-4cc4-b1e3-d71b527bd4bd => generated 62 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:54:32.405952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lazy-loading 'services' on Instance uuid bec91401-323d-4a36-88b7-88fe24aa813f {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:32.498123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.499373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] [instance: 9be47b55-0549-4005-b37e-52219a6e7c11] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:32.509711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:32.513202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:32.516359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:32.517408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:32.521022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:32.521759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:32.548467 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] HTTP exception thrown: Instance 9be47b55-0549-4005-b37e-52219a6e7c11 could not be found. Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Request method failure captured: Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 HTTP/1.1 Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx1EB-yAD7UG_Dap49GTD2s8yrVWivZQT83hJXr53R1MYLFh5JetTyf34JrS79KToWBGhFfABggxbY8VMHrP8asFDj5BfINoG6xOIk-6HEEXs6vjMOkFIpHoxQNeHZMks1TEz8PS3VrWi2WkmPlVxbgjJFh50JYX42qGEPiKTB5dloA Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e5c50f30f706437ea737a3d5d20ea8a1 Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ListImageFiltersTestJSON-505006441 Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:54:32.559146 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e5c50f30f706437ea737a3d5d20ea8a1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ListImageFiltersTestJSON-505006441 Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e5c50f30f706437ea737a3d5d20ea8a1 Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ListImageFiltersTestJSON-505006441 Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ListImageFiltersTestJSON-505006441-project Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 3e76b25814474b9db035954b3a437c96 Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ListImageFiltersTestJSON-505006441-project Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 9be47b55-0549-4005-b37e-52219a6e7c11 could not be found. Oct 15 20:54:32.561648 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '9be47b55-0549-4005-b37e-52219a6e7c11'} Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9be47b55-0549-4005-b37e-52219a6e7c11 could not be found. Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:54:32.562662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9be47b55-0549-4005-b37e-52219a6e7c11 could not be found. Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9be47b55-0549-4005-b37e-52219a6e7c11 could not be found. Oct 15 20:54:32.564696 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:54:32.566307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Returning 404 to user: Instance 9be47b55-0549-4005-b37e-52219a6e7c11 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:54:32.569453 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cd49088c-5550-4027-b5ae-9b3a64062cb9 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11" status: 404 len: 111 microversion: 2.1 time: 0.076210 Oct 15 20:54:32.570227 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1003/1870] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:32 2022] GET /compute/v2.1/servers/9be47b55-0549-4005-b37e-52219a6e7c11 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:54:32.579009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c7a100e-164a-4b54-a5d1-970fe0449814 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f239b80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.598324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:32.615479 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-42fd8340-3da4-400c-af8d-5b87b1eb343f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "DELETE /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments/7247b0f0-0975-42a1-86c8-08fe3b1f39e4" status: 202 len: 0 microversion: 2.1 time: 0.620449 Oct 15 20:54:32.616292 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 868/1871] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 20:54:31 2022] DELETE /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments/7247b0f0-0975-42a1-86c8-08fe3b1f39e4 => generated 0 bytes in 624 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:32.633470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:32.636848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:32.668750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:32.670863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:32.673164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:32.674222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:32.675636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:32.677126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:32.763975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc524ba5-6c99-4cee-8bbc-1945aa3ca8ba tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.137686 Oct 15 20:54:32.765573 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 869/1872] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:54:32 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments => generated 197 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:33.008473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:33.009465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:33.022452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:33.023433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:33.024118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:33.025149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:33.026513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:33.027546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:33.621543 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c7a100e-164a-4b54-a5d1-970fe0449814 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/images/50aa02fa-a46c-4e0d-acdb-ee5346eb1a86" status: 204 len: 0 microversion: 2.1 time: 1.046866 Oct 15 20:54:33.624073 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1004/1873] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:32 2022] DELETE /compute/v2.1/images/50aa02fa-a46c-4e0d-acdb-ee5346eb1a86 => generated 0 bytes in 1050 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 20:54:33.970269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e87c97e7-eca4-4367-a839-b388ea319e0b tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:33.985443 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e87c97e7-eca4-4367-a839-b388ea319e0b tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.358859 Oct 15 20:54:33.986263 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1005/1874] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:33 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:33.991253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-61cf1bdf-ef0c-4d65-949d-12520a83bb07 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f20b280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:34.574933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:34.594748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:34.603580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:34.605102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f13b77d4-2ff5-4c9d-aa74-046da49cd96d tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 1.612919 Oct 15 20:54:34.608075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 870/1875] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:32 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 1617 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:34.616030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3556ef3c-0ab1-4f32-abc7-52a7a29dde10 req-f8339d6e-2a30-4fc1-a159-0b396260a0ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c05e4f55-245d-45f2-8567-e665ccbca061", "name": "network-vif-unplugged", "status": "completed", "tag": "0e64f25a-9a1d-4631-8907-36be7c15d6f3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:34.644151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3556ef3c-0ab1-4f32-abc7-52a7a29dde10 req-f8339d6e-2a30-4fc1-a159-0b396260a0ca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:34.647849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3556ef3c-0ab1-4f32-abc7-52a7a29dde10 req-f8339d6e-2a30-4fc1-a159-0b396260a0ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:34.648305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3556ef3c-0ab1-4f32-abc7-52a7a29dde10 req-f8339d6e-2a30-4fc1-a159-0b396260a0ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:34.672774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3556ef3c-0ab1-4f32-abc7-52a7a29dde10 req-f8339d6e-2a30-4fc1-a159-0b396260a0ca service nova] Creating event network-vif-unplugged:0e64f25a-9a1d-4631-8907-36be7c15d6f3 for instance c05e4f55-245d-45f2-8567-e665ccbca061 on nch1-master-1 Oct 15 20:54:34.681631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3556ef3c-0ab1-4f32-abc7-52a7a29dde10 req-f8339d6e-2a30-4fc1-a159-0b396260a0ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.071447 Oct 15 20:54:34.682838 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 871/1876] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:34 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:34.683470 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-61cf1bdf-ef0c-4d65-949d-12520a83bb07 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/images/1bf895d1-f7b4-488c-a907-15e88a11c1a4" status: 204 len: 0 microversion: 2.1 time: 0.695543 Oct 15 20:54:34.684035 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1006/1877] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:33 2022] DELETE /compute/v2.1/images/1bf895d1-f7b4-488c-a907-15e88a11c1a4 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:34.690651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f2fb07a1-083e-4b2f-b725-3e59bd4f91ac tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:34.693458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:34.696351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:34.708112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:34.709583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:34.710225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:34.711315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:34.711989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:34.712672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:34.744044 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f2fb07a1-083e-4b2f-b725-3e59bd4f91ac tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors" status: 200 len: 2594 microversion: 2.1 time: 0.057734 Oct 15 20:54:34.744622 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1007/1878] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:54:34 2022] GET /compute/v2.1/flavors => generated 2594 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:34.760159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9eae7218-bdc6-45ba-858f-69ee52040dc5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] Calling method '.version_select at 0x7fe74f20b550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:34.787375 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b7511b76-6797-4e62-8c1e-67e103170c06 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.102797 Oct 15 20:54:34.788564 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 872/1879] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:54:34 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f/os-volume_attachments => generated 25 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:34.813568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a26fc86-f1ee-483f-94ea-0320f6bdb776 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:34.831240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a26fc86-f1ee-483f-94ea-0320f6bdb776 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.041256 Oct 15 20:54:34.834529 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 873/1880] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:34 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.116990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-474895a8-e385-4e41-8b6f-95600541c6f5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Action: 'create', calling method: .version_select at 0x7fe7583b94c0>, body: {"keypair": {"name": "tempest-keypair-1065339057", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDZwOMUWv11zOBStOuWH+6lx5HgRE0ojUyau3eekW0oFFL9OSr5JXGz2n65SLit6hg4XpR3c2uGemViRObPZJG8bPFiiLzapT8F2ckRygmk0HU8gRU9pAtliBNtIO/8log=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:35.160428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-474895a8-e385-4e41-8b6f-95600541c6f5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:35.173577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-474895a8-e385-4e41-8b6f-95600541c6f5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for user 0d6fa4a9541641a5a36e909dc19b491d and project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:35.274207 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-474895a8-e385-4e41-8b6f-95600541c6f5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.438366 Oct 15 20:54:35.276112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 874/1881] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.288144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-41223b3c-0c1e-4003-9f44-adbf8374aa1b tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.306699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-41223b3c-0c1e-4003-9f44-adbf8374aa1b tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.029624 Oct 15 20:54:35.320858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 875/1882] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.328084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-494609d9-ada8-4f1d-8c43-7c3bd6e11e72 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe7584d1af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.338667 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-494609d9-ada8-4f1d-8c43-7c3bd6e11e72 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-1065339057" status: 200 len: 498 microversion: 2.1 time: 0.016074 Oct 15 20:54:35.339546 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 876/1883] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1065339057 => generated 498 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.345582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53a53041-f872-4a52-ad98-bbedb6d00ab0 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.386260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53a53041-f872-4a52-ad98-bbedb6d00ab0 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?minDisk=2" status: 200 len: 3209 microversion: 2.1 time: 0.045559 Oct 15 20:54:35.400932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 877/1884] 10.222.0.16 () {58 vars in 1165 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/detail?minDisk=2 => generated 3209 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:35.406527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f7f837b-2e0c-4bf9-9020-34749f1f3cd7 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe7583b9430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.439906 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f7f837b-2e0c-4bf9-9020-34749f1f3cd7 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1065339057" status: 202 len: 0 microversion: 2.1 time: 0.034893 Oct 15 20:54:35.448813 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 878/1885] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:54:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1065339057 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:35.460481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fcc228cd-690c-40b2-9219-ebd32a203f94 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.482508 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fcc228cd-690c-40b2-9219-ebd32a203f94 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.029713 Oct 15 20:54:35.485154 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 879/1886] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.508493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce8611fb-f653-4e9a-bf50-894e6cdaccd6 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Action: 'create', calling method: .version_select at 0x7fe7583d71f0>, body: {"keypair": {"name": "tempest-keypair-1512423789", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI2OKGEIk/g8nGmC3olCpawtBGoYVv75s4iGTmFNCooi7Q5FYr0nLWEpx0BYqz1DNviY3jmE5/AoUxJiDr5CUG2Nx4hmckrU7mhLZ8HQ6jkKcs3H74d0yggtBxuUqQ0MfQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:35.531014 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9eae7218-bdc6-45ba-858f-69ee52040dc5 tempest-ListImageFiltersTestJSON-505006441 tempest-ListImageFiltersTestJSON-505006441-project] 10.222.0.16 "DELETE /compute/v2.1/images/84a484d1-2c91-4a62-b8fd-3470464affd5" status: 204 len: 0 microversion: 2.1 time: 0.774729 Oct 15 20:54:35.531967 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1008/1887] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:54:34 2022] DELETE /compute/v2.1/images/84a484d1-2c91-4a62-b8fd-3470464affd5 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:35.537446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3dc34a10-7dcf-4f46-b995-b06de7d46fc7 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.563881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-ce8611fb-f653-4e9a-bf50-894e6cdaccd6 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:35.575254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-ce8611fb-f653-4e9a-bf50-894e6cdaccd6 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for user 0d6fa4a9541641a5a36e909dc19b491d and project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:35.593580 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3dc34a10-7dcf-4f46-b995-b06de7d46fc7 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?minRam=1025" status: 200 len: 2145 microversion: 2.1 time: 0.060349 Oct 15 20:54:35.594188 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1009/1888] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/detail?minRam=1025 => generated 2145 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:35.611152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7dc6a024-d52f-448f-a098-13d3171cf366 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.636028 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce8611fb-f653-4e9a-bf50-894e6cdaccd6 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.149122 Oct 15 20:54:35.636986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 880/1889] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:35 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.639551 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7dc6a024-d52f-448f-a098-13d3171cf366 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?limit=1" status: 200 len: 476 microversion: 2.1 time: 0.032865 Oct 15 20:54:35.645076 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1010/1890] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/detail?limit=1 => generated 476 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:54:35.662617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.665073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:35.669800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-943c1252-1b61-442c-9951-56eb4ab42508 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe74f20b160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.672875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:35.674264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:35.674950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:35.676241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:35.677017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:35.688644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:35.712402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-943c1252-1b61-442c-9951-56eb4ab42508 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1512423789" status: 202 len: 0 microversion: 2.1 time: 0.066061 Oct 15 20:54:35.714000 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1011/1891] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:54:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1512423789 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:35.726201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-320b613a-1832-4665-9e08-e2f7e3c59c78 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.741963 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-320b613a-1832-4665-9e08-e2f7e3c59c78 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.026144 Oct 15 20:54:35.746590 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1012/1892] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.760189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64384bba-ece5-4726-af8b-65e8020c7232 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Action: 'create', calling method: .version_select at 0x7fe74f28bd30>, body: {"keypair": {"name": "tempest-keypair-1256228236", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsNe3/1ILNCqFyfYWDeTKLD6jEXC2OQHLmietMWW+/vdaZq7KZEwO0jhglaFjU1mpqq4Gz5RX156sCTNM9vRbwKAxfsdF9laBYVsex3m3Wmui3uYrKyumsoJn2g9GNnG1PI1mrVjZ61i0GY3khna+wzlTpCCmy5HNlrmbj3XLqBUpipTOXmsnr4sChzC53KCd8LXuwc1i/CZPvF+3XipvAgFSE53pCtLOeB1kYMOBaiUPLQTWXR3JpckqFIQwhIH0zoHlJvZE8hh90XcPojYN56tI0OlrGqojbediJYD0rUsJu4weZpbn8vilb3JuDY+jwssnSA8wzBx3A/8y9Pp1B nova@ubuntu"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:35.801640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-64384bba-ece5-4726-af8b-65e8020c7232 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:35.807986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-64384bba-ece5-4726-af8b-65e8020c7232 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for user 0d6fa4a9541641a5a36e909dc19b491d and project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:35.846148 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64384bba-ece5-4726-af8b-65e8020c7232 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 574 microversion: 2.1 time: 0.092831 Oct 15 20:54:35.847536 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1013/1893] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:35 2022] POST /compute/v2.1/os-keypairs => generated 574 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:35.858493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-549eeea9-34bd-428f-8f1d-3827b8b8ed84 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.902014 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-549eeea9-34bd-428f-8f1d-3827b8b8ed84 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?marker=452" status: 200 len: 2500 microversion: 2.1 time: 0.047701 Oct 15 20:54:35.903049 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1014/1894] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/detail?marker=452 => generated 2500 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:35.909151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9bf214af-63ea-4b49-9da8-43a2390a9131 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe74f279310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:35.959311 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9bf214af-63ea-4b49-9da8-43a2390a9131 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1256228236" status: 202 len: 0 microversion: 2.1 time: 0.053306 Oct 15 20:54:35.960454 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1015/1895] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:54:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1256228236 => generated 0 bytes in 57 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:35.983093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-253d893b-07ab-462b-9c83-eea870aef49c tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.022160 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-253d893b-07ab-462b-9c83-eea870aef49c tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.053034 Oct 15 20:54:36.026639 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1016/1896] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.040875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d7feef4-0f67-4ea5-b20a-87a664aed3c4 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Action: 'create', calling method: .version_select at 0x7fe74f20b5e0>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-47975867", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF8h3mzRTWQbaAFayVv3a+frRbLGvNu5Wm+vXA9HjmcXemZPTa/vSEBxeCZ3NduAl7JYEJCDrmzTIlMN6gpu+y+W2QlAeBM9ND1uNpLO5A22MQSIO9lQjfyLTnB/SiVnvw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:36.110706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7d7feef4-0f67-4ea5-b20a-87a664aed3c4 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:36.120406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7d7feef4-0f67-4ea5-b20a-87a664aed3c4 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for user 0d6fa4a9541641a5a36e909dc19b491d and project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:36.166232 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d7feef4-0f67-4ea5-b20a-87a664aed3c4 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.138224 Oct 15 20:54:36.176959 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1017/1897] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:36 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.186124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-796411bd-d536-4365-b068-ebd10e37c1da tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.245812 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-796411bd-d536-4365-b068-ebd10e37c1da tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors?minDisk=2" status: 200 len: 1663 microversion: 2.1 time: 0.067179 Oct 15 20:54:36.248178 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1018/1898] 10.222.0.16 () {58 vars in 1144 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors?minDisk=2 => generated 1663 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:36.256080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:36.266273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:36.267727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:36.271628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-26c39945-68be-45c6-9686-2cd7641e96dc tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Action: 'create', calling method: .version_select at 0x7fe74f2390d0>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-603443282", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOhzm4W3vANNRElHukfCtQXxkVWN+cUNxsbfScJR4xgEzrE6y2H4X9xDTWVVrjOGzpQACVH3H01bDbiJNyRoqHy1wQbKzR2iqUtVbTwrzYtk0uQZOQPh1+n3m/z+BoC77w=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:36.279045 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a73f100-c78b-4302-a6cd-ed389ff84c2b tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.634207 Oct 15 20:54:36.279745 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 881/1899] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:35 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:36.286348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4497e3b5-c5e9-4eca-aa5f-ae45155f9861 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.315733 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4497e3b5-c5e9-4eca-aa5f-ae45155f9861 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.034581 Oct 15 20:54:36.317577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-26c39945-68be-45c6-9686-2cd7641e96dc tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:36.323716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-26c39945-68be-45c6-9686-2cd7641e96dc tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for user 0d6fa4a9541641a5a36e909dc19b491d and project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:36.324031 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 882/1900] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.329191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c8b71ad-182c-4470-a7c3-429e6ce6e7f1 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.365297 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-26c39945-68be-45c6-9686-2cd7641e96dc tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 404 microversion: 2.1 time: 0.114021 Oct 15 20:54:36.366625 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1019/1901] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:36 2022] POST /compute/v2.1/os-keypairs => generated 404 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.381352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-91dece27-8ed0-44e7-9e15-4e5c9a3a9084 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Action: 'create', calling method: .version_select at 0x7fe74f2a20d0>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-208892458", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIYkHGFwBf0DICOXFb+qc1W7dZxRy/VhLyG5qrQB/ugjngjhOFzOEQtg+Qfxkic6Am0xdhjF3AYdJ9XwWOElzGXmGtIyqxQqWEAEE+PjgYck+q30hHAV6ZeLmPt9lj34TA=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:36.391520 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c8b71ad-182c-4470-a7c3-429e6ce6e7f1 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors?minRam=1025" status: 200 len: 1111 microversion: 2.1 time: 0.066329 Oct 15 20:54:36.394876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 883/1902] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors?minRam=1025 => generated 1111 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:36.412730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb0a4dea-6667-4257-921c-e2488ef53fd0 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.427363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-91dece27-8ed0-44e7-9e15-4e5c9a3a9084 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:36.431684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb0a4dea-6667-4257-921c-e2488ef53fd0 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors?limit=1" status: 200 len: 299 microversion: 2.1 time: 0.027383 Oct 15 20:54:36.433120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 884/1903] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors?limit=1 => generated 299 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.447776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-91dece27-8ed0-44e7-9e15-4e5c9a3a9084 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Getting quotas for user 0d6fa4a9541641a5a36e909dc19b491d and project c456724943014d179f9aab69cb956cc7. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:36.449269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6efb9dad-d3ef-4e7a-a5ad-75b36fbdacbd tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.459379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6efb9dad-d3ef-4e7a-a5ad-75b36fbdacbd tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.017605 Oct 15 20:54:36.459967 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 885/1904] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.470402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:36.474656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dbaa006-c555-4b0f-b37f-4e3beb9c39fc tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.498060 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-91dece27-8ed0-44e7-9e15-4e5c9a3a9084 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 404 microversion: 2.1 time: 0.120693 Oct 15 20:54:36.498904 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1020/1905] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:54:36 2022] POST /compute/v2.1/os-keypairs => generated 404 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.508555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b40743f-7231-47eb-a110-14a853baead5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe74f2353a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.512383 ncdv-master-1 devstack@n-api.service[138689]: WARNING oslo_db.sqlalchemy.utils [None req-6b40743f-7231-47eb-a110-14a853baead5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Unique keys not in sort_keys. The sorting order may be unstable. Oct 15 20:54:36.519727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:36.525289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5dbaa006-c555-4b0f-b37f-4e3beb9c39fc tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors?marker=452" status: 200 len: 1300 microversion: 2.1 time: 0.055571 Oct 15 20:54:36.533084 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b40743f-7231-47eb-a110-14a853baead5 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 1090 microversion: 2.1 time: 0.028145 Oct 15 20:54:36.533726 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1021/1906] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/os-keypairs => generated 1090 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:36.540154 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 886/1907] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors?marker=452 => generated 1300 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:36.546432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b06854d3-92e2-49b7-a501-7d80a61c8cab tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe74f239160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.555142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fdc1b6df-9742-4429-be57-d25fe8ff9498 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.585381 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fdc1b6df-9742-4429-be57-d25fe8ff9498 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.037865 Oct 15 20:54:36.588899 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b06854d3-92e2-49b7-a501-7d80a61c8cab tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-208892458" status: 202 len: 0 microversion: 2.1 time: 0.048279 Oct 15 20:54:36.593345 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1022/1908] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:54:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-208892458 => generated 0 bytes in 53 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 20:54:36.601924 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 887/1909] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:36.608984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8730c18a-5acb-4337-98dd-827d07575040 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.618183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8730c18a-5acb-4337-98dd-827d07575040 tempest-FlavorsV2TestJSON-1858453864 tempest-FlavorsV2TestJSON-1858453864-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.014145 Oct 15 20:54:36.618843 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 888/1910] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:54:36 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:36.625729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2c3ab79b-d8d3-4dfa-93da-dde777978b05 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe74f152dc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.656782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:36.706744 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2c3ab79b-d8d3-4dfa-93da-dde777978b05 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-603443282" status: 202 len: 0 microversion: 2.1 time: 0.110806 Oct 15 20:54:36.708533 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1023/1911] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:54:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-603443282 => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:36.720538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-383d57a2-ce16-4da4-8eec-54c19496e421 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] Calling method '.version_select at 0x7fe7583efc10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:36.772614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-383d57a2-ce16-4da4-8eec-54c19496e421 tempest-KeyPairsV2TestJSON-662423212 tempest-KeyPairsV2TestJSON-662423212-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-47975867" status: 202 len: 0 microversion: 2.1 time: 0.059152 Oct 15 20:54:36.774220 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 889/1912] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:54:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-47975867 => generated 0 bytes in 61 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:54:36.785411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "062c4651-90be-430a-a69b-372fac258ebb", "tag": "e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8"}, {"name": "network-changed", "server_uuid": "c05e4f55-245d-45f2-8567-e665ccbca061", "tag": "0e64f25a-9a1d-4631-8907-36be7c15d6f3"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:36.805318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:36.806894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:36.807468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:36.851261 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] Creating event network-changed:e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8 for instance 062c4651-90be-430a-a69b-372fac258ebb on nch1-master-1 Oct 15 20:54:36.851784 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] Creating event network-changed:0e64f25a-9a1d-4631-8907-36be7c15d6f3 for instance c05e4f55-245d-45f2-8567-e665ccbca061 on nch1-master-1 Oct 15 20:54:36.871263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-54acb5db-7c3f-40d5-afa2-f4185b19b4f4 req-6aa547f6-f58c-487f-8471-6c59a629ac1e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.088185 Oct 15 20:54:36.871899 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1024/1913] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:54:36 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:37.309189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:37.310497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:37.330367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:37.330978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:37.331538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:37.332370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:37.332932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:37.333491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:37.658781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:37.669892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:37.675063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:37.679329 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b22d7e09-536a-4a11-a6cc-164416889cc5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.376632 Oct 15 20:54:37.681118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 890/1914] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:37 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:38.707560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:38.709332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:38.714613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=2,nova_api:DELETE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:38.737137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:38.744011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:38.744689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:38.745409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:38.745879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:38.746315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:39.082833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:39.098483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d240f95-9f6d-4d32-8311-1b0e34d242e9 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe74f28bb80>, body: {"keypair": {"name": "tempest-keypair-1620204043", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPYUEIcCQIiLpreYUlXlv/jZqRPgNvSmtpgJKzFGOUWYgKZt7VpQOJfwrE1JBRiQLJWPISAlnNWr6y16U4MrujcY9McoeH5paAM6yIb+sTrpW8iZcaAoNPH162cSjEvk7w=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:39.105072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:39.107132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:39.108901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0091b13e-45c1-4dcc-8b2e-bb5f87c35097 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.405537 Oct 15 20:54:39.119977 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 891/1915] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:38 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:39.128295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-2d240f95-9f6d-4d32-8311-1b0e34d242e9 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:39.133810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-2d240f95-9f6d-4d32-8311-1b0e34d242e9 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:39.197390 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2d240f95-9f6d-4d32-8311-1b0e34d242e9 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.2 time: 0.557944 Oct 15 20:54:39.205095 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1025/1916] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:38 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 567 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:54:39.213922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c726c91a-51b1-4010-8c5d-7bd7d6a48216 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe7583b9280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:39.238897 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c726c91a-51b1-4010-8c5d-7bd7d6a48216 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-1620204043" status: 200 len: 513 microversion: 2.2 time: 0.033033 Oct 15 20:54:39.239485 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 892/1917] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Oct 15 20:54:39 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1620204043 => generated 513 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:39.253969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7430462e-4985-4af0-b411-aa036589588c tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f2a2430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:39.293031 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7430462e-4985-4af0-b411-aa036589588c tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1620204043" status: 204 len: 0 microversion: 2.2 time: 0.041215 Oct 15 20:54:39.304732 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1026/1918] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Oct 15 20:54:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1620204043 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:39.310437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06b1c612-2ab5-4bb7-99b5-a3d2e945608c tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe74f1f84c0>, body: {"keypair": {"name": "tempest-keypair-6338144", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB/UDWrzHTqJIGNOWafrnTCFzdeC0I2MUhA2U511L7+XqCTUi2D4W1WQEBqStoRDEy4E4JQ4zqvb8YCVdaoWKfVKOYtIR+cmunecYcmzmeDoBybKJ/PmqbouCeE4P5qKew=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:39.347293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-06b1c612-2ab5-4bb7-99b5-a3d2e945608c tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:39.354519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-06b1c612-2ab5-4bb7-99b5-a3d2e945608c tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:39.398177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-06b1c612-2ab5-4bb7-99b5-a3d2e945608c tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 398 microversion: 2.2 time: 0.092036 Oct 15 20:54:39.399406 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 893/1919] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:39 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 95 msecs (HTTP/1.1 201) 9 headers in 362 bytes (2 switches on core 0) Oct 15 20:54:39.411443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-92bbba8f-f239-46a8-b330-bb170f148140 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f235670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:39.446743 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-92bbba8f-f239-46a8-b330-bb170f148140 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-6338144" status: 204 len: 0 microversion: 2.2 time: 0.039650 Oct 15 20:54:39.448190 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1027/1920] 10.222.0.16 () {60 vars in 1252 bytes} [Sat Oct 15 20:54:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-6338144 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:39.459195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d10463c9-cf4b-4d90-ab17-3c42575f1678 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe74f1b05e0>, body: {"keypair": {"name": "tempest-keypair-307239425", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsNe3/1ILNCqFyfYWDeTKLD6jEXC2OQHLmietMWW+/vdaZq7KZEwO0jhglaFjU1mpqq4Gz5RX156sCTNM9vRbwKAxfsdF9laBYVsex3m3Wmui3uYrKyumsoJn2g9GNnG1PI1mrVjZ61i0GY3khna+wzlTpCCmy5HNlrmbj3XLqBUpipTOXmsnr4sChzC53KCd8LXuwc1i/CZPvF+3XipvAgFSE53pCtLOeB1kYMOBaiUPLQTWXR3JpckqFIQwhIH0zoHlJvZE8hh90XcPojYN56tI0OlrGqojbediJYD0rUsJu4weZpbn8vilb3JuDY+jwssnSA8wzBx3A/8y9Pp1B nova@ubuntu"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:39.486702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d10463c9-cf4b-4d90-ab17-3c42575f1678 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:39.504832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d10463c9-cf4b-4d90-ab17-3c42575f1678 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:39.547846 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d10463c9-cf4b-4d90-ab17-3c42575f1678 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 588 microversion: 2.2 time: 0.093593 Oct 15 20:54:39.548621 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 894/1921] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:39 2022] POST /compute/v2.1/os-keypairs => generated 588 bytes in 95 msecs (HTTP/1.1 201) 9 headers in 362 bytes (2 switches on core 0) Oct 15 20:54:39.559841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dadcb0dd-3bb3-4fb3-83af-6a99e9923726 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f2398b0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:39.581694 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dadcb0dd-3bb3-4fb3-83af-6a99e9923726 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-307239425" status: 204 len: 0 microversion: 2.2 time: 0.025904 Oct 15 20:54:39.593213 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1028/1922] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Oct 15 20:54:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-307239425 => generated 0 bytes in 27 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:39.596365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1728b56-35be-43d8-9e8c-116d5950be3b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe7584d1ca0>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-102368947", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPK2S4Ya5xIWDFBVPhjxWCSpTDGT/x+/HkdKSY/HCOyW7TncidShAqhsOJmHzlcQZuinvhmWiG0yLATSxpqGNBGaZx3jYU7J5FAoTsJbrxH3GYGzEIO/+7XjF3Y+wiVl1Q=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:39.621068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a1728b56-35be-43d8-9e8c-116d5950be3b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:39.629588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a1728b56-35be-43d8-9e8c-116d5950be3b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:39.669125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a1728b56-35be-43d8-9e8c-116d5950be3b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 420 microversion: 2.2 time: 0.077943 Oct 15 20:54:39.670189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 895/1923] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:39 2022] POST /compute/v2.1/os-keypairs => generated 420 bytes in 80 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:54:39.686319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-710aff61-9952-4aff-9510-b75d66232145 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe74f313040>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-1938158801", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCwQymOQLDz1tImuBYf69QodPXVEtaIEQ61Wm++Tojc8hne9DfFCdjRkyfUcGqD+e9/Cc4tqaGPKCEq5IIn7wPlXHXHY7ERC/+o2Q2nPxXn1SrKl0D+FSpPEfjqTlWT5xg=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:39.755408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-710aff61-9952-4aff-9510-b75d66232145 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:39.769153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-710aff61-9952-4aff-9510-b75d66232145 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:39.824752 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-710aff61-9952-4aff-9510-b75d66232145 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 421 microversion: 2.2 time: 0.144471 Oct 15 20:54:39.826588 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1029/1924] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:39 2022] POST /compute/v2.1/os-keypairs => generated 421 bytes in 147 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:54:39.843720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-509355d7-c093-415d-afd3-c993bf749ed3 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe7584d1430>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-76549494", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBETxPQpHGBytQhWDzt0e+o6SWZVpv4kL1HxvQIm9PTWm4LD4WEinup4HFd0Ua/TxrerJ/aWhCx3QGyytpU1Q1xrIyjFuGNArr5xWa9aCZQC9UWd/yqVi6mKoOIOEoQRr3w=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:39.873828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-509355d7-c093-415d-afd3-c993bf749ed3 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:39.879876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-509355d7-c093-415d-afd3-c993bf749ed3 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:39.913784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-509355d7-c093-415d-afd3-c993bf749ed3 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 419 microversion: 2.2 time: 0.072555 Oct 15 20:54:39.914447 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 896/1925] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:39 2022] POST /compute/v2.1/os-keypairs => generated 419 bytes in 76 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:54:39.932373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ef6545e-d30e-41d9-8134-92d8b3c05779 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f3139d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:39.942400 ncdv-master-1 devstack@n-api.service[138689]: WARNING oslo_db.sqlalchemy.utils [None req-8ef6545e-d30e-41d9-8134-92d8b3c05779 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Unique keys not in sort_keys. The sorting order may be unstable. Oct 15 20:54:39.965384 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ef6545e-d30e-41d9-8134-92d8b3c05779 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 1139 microversion: 2.2 time: 0.028016 Oct 15 20:54:39.967720 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1030/1926] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Oct 15 20:54:39 2022] GET /compute/v2.1/os-keypairs => generated 1139 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:39.983527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f180b806-5a63-43fe-9613-1b3218a10e49 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f1b00d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.012127 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f180b806-5a63-43fe-9613-1b3218a10e49 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-76549494" status: 204 len: 0 microversion: 2.2 time: 0.034672 Oct 15 20:54:40.014035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 897/1927] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Oct 15 20:54:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-76549494 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:40.025396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-79adcdb1-3d53-4778-80d4-858d3e93d5f1 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f1f4430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.063216 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-79adcdb1-3d53-4778-80d4-858d3e93d5f1 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1938158801" status: 204 len: 0 microversion: 2.2 time: 0.038135 Oct 15 20:54:40.064173 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1031/1928] 10.222.0.16 () {60 vars in 1321 bytes} [Sat Oct 15 20:54:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1938158801 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:40.073213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a755c36b-922a-4b8e-8ec2-8a04f0fbe09b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f1bc940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.103651 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a755c36b-922a-4b8e-8ec2-8a04f0fbe09b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-102368947" status: 204 len: 0 microversion: 2.2 time: 0.034765 Oct 15 20:54:40.104231 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 898/1929] 10.222.0.16 () {60 vars in 1318 bytes} [Sat Oct 15 20:54:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-102368947 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:40.121476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a46a7a9c-7a7f-4b4a-a311-0c17d229bf37 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe74f2a2790>, body: {"keypair": {"name": "tempest-keypair-271860738", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBdHQHYanNJOeZ0F+PHUFmxqXGL85LO11vkF7RGXGaP/H+NH5jG4UWhq/7CH+16bKORN47ytfcVJ68U7IiJ9DAifENEnGaY/raPJAiPzuLS+kibVBPqZ/z4V5OBRVXB7Dw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:40.150317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a46a7a9c-7a7f-4b4a-a311-0c17d229bf37 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:40.154627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.155461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:40.158536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a46a7a9c-7a7f-4b4a-a311-0c17d229bf37 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:40.175354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:40.175727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:40.176176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:40.177162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:40.182473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:40.183401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:40.232700 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a46a7a9c-7a7f-4b4a-a311-0c17d229bf37 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.2 time: 0.113716 Oct 15 20:54:40.240440 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1032/1930] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Oct 15 20:54:40 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 119 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Oct 15 20:54:40.251675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-71a23f2b-dea0-4220-9ec3-a5ffc2fbee5b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f2a2310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.282463 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-71a23f2b-dea0-4220-9ec3-a5ffc2fbee5b tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-271860738" status: 200 len: 512 microversion: 2.2 time: 0.039809 Oct 15 20:54:40.282993 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1033/1931] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Oct 15 20:54:40 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-271860738 => generated 512 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:40.305465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bedff230-adfb-4e05-93db-ebabba8fbdde tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f239820>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.316456 ncdv-master-1 devstack@n-api.service[138689]: WARNING oslo_db.sqlalchemy.utils [None req-bedff230-adfb-4e05-93db-ebabba8fbdde tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Unique keys not in sort_keys. The sorting order may be unstable. Oct 15 20:54:40.328148 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bedff230-adfb-4e05-93db-ebabba8fbdde tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 369 microversion: 2.2 time: 0.029142 Oct 15 20:54:40.329663 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1034/1932] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Oct 15 20:54:40 2022] GET /compute/v2.1/os-keypairs => generated 369 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:40.345992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-59cfdf8f-12f7-4c33-81b3-0e258c1bf0de tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f1f4670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:40.422944 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-59cfdf8f-12f7-4c33-81b3-0e258c1bf0de tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-271860738" status: 204 len: 0 microversion: 2.2 time: 0.084396 Oct 15 20:54:40.425529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1035/1933] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Oct 15 20:54:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-271860738 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:40.444314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Action: 'create', calling method: .version_select at 0x7fe74f20b940>, body: {"keypair": {"name": "tempest-keypair-1115591538", "type": "x509"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:40.497752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:40.519914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:40.523428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:40.541530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:40.547888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:40.550050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ba53b94-0d1a-4f56-84be-0fcec60e37ee tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.403884 Oct 15 20:54:40.551703 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 899/1934] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:40 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:40.592654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.processutils [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Running cmd (subprocess): openssl req -x509 -nodes -days 3650 -config /tmp/tmpjbat2dpl/temp.conf -newkey rsa:2048 -outform PEM -keyout /tmp/tmpjbat2dpl/temp.key -subj /CN=f049946797ae43b3bd722e6d20cb8cce -extensions v3_req_client {{(pid=138689) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 15 20:54:40.782430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.processutils [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] CMD "openssl req -x509 -nodes -days 3650 -config /tmp/tmpjbat2dpl/temp.conf -newkey rsa:2048 -outform PEM -keyout /tmp/tmpjbat2dpl/temp.key -subj /CN=f049946797ae43b3bd722e6d20cb8cce -extensions v3_req_client" returned: 0 in 0.191s {{(pid=138689) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 15 20:54:40.783968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.processutils [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Running cmd (subprocess): openssl pkcs12 -export -inkey /tmp/tmpjbat2dpl/temp.key -password pass: {{(pid=138689) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 15 20:54:40.850614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.processutils [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] CMD "openssl pkcs12 -export -inkey /tmp/tmpjbat2dpl/temp.key -password pass:" returned: 0 in 0.066s {{(pid=138689) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 15 20:54:40.928712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:40.953073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Getting quotas for user f049946797ae43b3bd722e6d20cb8cce and project 1deec50c0f374014bb6a24d2cb8be67b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:41.014431 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c5e757e4-e9f6-49f0-a574-624bfa728f11 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 4693 microversion: 2.2 time: 0.582763 Oct 15 20:54:41.017086 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1036/1935] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Oct 15 20:54:40 2022] POST /compute/v2.1/os-keypairs => generated 4693 bytes in 586 msecs (HTTP/1.1 201) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:41.028909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e47f0ae3-4cc5-473e-bf5f-0fcb4c2ad8e6 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f1b0310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:41.048757 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e47f0ae3-4cc5-473e-bf5f-0fcb4c2ad8e6 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-1115591538" status: 200 len: 1534 microversion: 2.2 time: 0.024070 Oct 15 20:54:41.056440 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 900/1936] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Oct 15 20:54:41 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1115591538 => generated 1534 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:41.063014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aff12af1-5539-45d7-9cda-60c4cdc6e8a5 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe74f28b1f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:41.067965 ncdv-master-1 devstack@n-api.service[138689]: WARNING oslo_db.sqlalchemy.utils [None req-aff12af1-5539-45d7-9cda-60c4cdc6e8a5 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Unique keys not in sort_keys. The sorting order may be unstable. Oct 15 20:54:41.088075 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aff12af1-5539-45d7-9cda-60c4cdc6e8a5 tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 1391 microversion: 2.2 time: 0.030247 Oct 15 20:54:41.089058 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1037/1937] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Oct 15 20:54:41 2022] GET /compute/v2.1/os-keypairs => generated 1391 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:54:41.101919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45d959bf-1381-40bd-84f6-5baf1de0cc3a tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] Calling method '.version_select at 0x7fe758344820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:41.160110 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45d959bf-1381-40bd-84f6-5baf1de0cc3a tempest-KeyPairsV22TestJSON-1250413600 tempest-KeyPairsV22TestJSON-1250413600-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1115591538" status: 204 len: 0 microversion: 2.2 time: 0.059322 Oct 15 20:54:41.161181 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 901/1938] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Oct 15 20:54:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1115591538 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:41.581607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:41.583175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:41.603684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:41.604791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:41.608242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:41.610241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:41.611126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:41.611657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:42.073214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:42.087659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:42.092663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:42.094640 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f7b590ea-e7f4-4a94-96c3-395d2bd3954a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.518673 Oct 15 20:54:42.095132 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1038/1939] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:41 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:42.390165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8af3acbb-1219-4216-90fc-3704de20f823 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f1f8550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:42.495903 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8af3acbb-1219-4216-90fc-3704de20f823 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85" status: 200 len: 637 microversion: 2.1 time: 0.117874 Oct 15 20:54:42.497300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 902/1940] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:54:42 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85 => generated 637 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:54:42.513493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a7aaaae9-3d7f-436f-8749-bf0f5523f2a7 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe74f1f4280>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:42.595081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=89,nova_api:DELETE=7,nova_api:INSERT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:42.621201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a7aaaae9-3d7f-436f-8749-bf0f5523f2a7 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:42.639770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a7aaaae9-3d7f-436f-8749-bf0f5523f2a7 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:43.127325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:43.128155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:43.146324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:43.147974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:43.149491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:43.150258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:43.150734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:43.154255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:43.409107 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a7aaaae9-3d7f-436f-8749-bf0f5523f2a7 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.900921 Oct 15 20:54:43.410217 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1039/1941] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:42 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:43.419557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4ac4ca48-44e4-4e2d-8c6b-f46bfc9b07cc tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f313820>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:44.096570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:44.114651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:44.127916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:44.135836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eae706b4-fdd1-4781-acdc-5155438c3ca0 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 1.012381 Oct 15 20:54:44.144487 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4ac4ca48-44e4-4e2d-8c6b-f46bfc9b07cc tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "DELETE /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata/os_version" status: 204 len: 0 microversion: 2.1 time: 0.724570 Oct 15 20:54:44.146038 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1040/1942] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Oct 15 20:54:43 2022] DELETE /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata/os_version => generated 0 bytes in 731 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:44.151333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 903/1943] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:43 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 1029 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:44.154777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-62d10ea1-b9d9-49a5-8867-e8e7f7da2397 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f279d30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:44.230639 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-62d10ea1-b9d9-49a5-8867-e8e7f7da2397 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 37 microversion: 2.1 time: 0.080700 Oct 15 20:54:44.232032 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1041/1944] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:54:44 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 37 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:54:44.242682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1c665ee-55c1-422b-a073-db7e8bbfb9de tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe74f1f8c10>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:44.355446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c1c665ee-55c1-422b-a073-db7e8bbfb9de tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:44.375498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c1c665ee-55c1-422b-a073-db7e8bbfb9de tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:45.061576 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c1c665ee-55c1-422b-a073-db7e8bbfb9de tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.822689 Oct 15 20:54:45.063224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 904/1945] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:44 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 825 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:45.073187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d3ffab9-85d8-494a-88e8-6e9a049c37ee tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f1f40d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:45.165883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:45.167325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:45.179395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:45.181651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:45.184149 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2d3ffab9-85d8-494a-88e8-6e9a049c37ee tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata/os_distro" status: 200 len: 33 microversion: 2.1 time: 0.115229 Oct 15 20:54:45.185120 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1042/1946] 10.222.0.16 () {58 vars in 1290 bytes} [Sat Oct 15 20:54:45 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata/os_distro => generated 33 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:54:45.185963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:45.186983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:45.187677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:45.188329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:45.211218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1eac1328-85b3-484d-8fe4-0ef82a9484ae tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe74f2794c0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:45.308937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1eac1328-85b3-484d-8fe4-0ef82a9484ae tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:45.322444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1eac1328-85b3-484d-8fe4-0ef82a9484ae tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:45.680706 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1eac1328-85b3-484d-8fe4-0ef82a9484ae tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.480099 Oct 15 20:54:45.683590 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1043/1947] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:45 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:45.693534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f4731d2-6c1d-482f-9e5f-79a3fd74d987 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f279c10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:45.762371 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f4731d2-6c1d-482f-9e5f-79a3fd74d987 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.072827 Oct 15 20:54:45.764128 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1044/1948] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:54:45 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:45.781247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-371a5e12-8264-4b79-a6ad-8850e29cac8c tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe74f239670>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:45.892899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-371a5e12-8264-4b79-a6ad-8850e29cac8c tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:45.912890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-371a5e12-8264-4b79-a6ad-8850e29cac8c tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:46.255904 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-371a5e12-8264-4b79-a6ad-8850e29cac8c tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.485375 Oct 15 20:54:46.257320 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1045/1949] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:45 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:46.268045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b364c135-029c-4a03-b6b6-b2dd118f1762 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe74f239940>, body: {"metadata": {"os_version": "value2", "architecture": "value3"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:46.359586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-b364c135-029c-4a03-b6b6-b2dd118f1762 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:46.376557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-b364c135-029c-4a03-b6b6-b2dd118f1762 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:46.557004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:46.596189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:46.598253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:46.599484 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c15a9a9-40db-4545-9237-37161b0c1365 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 1.437580 Oct 15 20:54:46.612223 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 905/1950] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:45 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 1451 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:47.011957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b364c135-029c-4a03-b6b6-b2dd118f1762 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 64 microversion: 2.1 time: 0.749065 Oct 15 20:54:47.012571 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1046/1951] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:46 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 64 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:47.023199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-41159c15-9fc3-48c1-ba5e-c9bea01b206c tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f14da60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:47.116257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-41159c15-9fc3-48c1-ba5e-c9bea01b206c tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 64 microversion: 2.1 time: 0.097530 Oct 15 20:54:47.116839 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 906/1952] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:54:47 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 64 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:47.139160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23e3c177-cf47-4f92-ab86-79a762fddb7d tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe74f279820>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:47.252660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-23e3c177-cf47-4f92-ab86-79a762fddb7d tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:47.264592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-23e3c177-cf47-4f92-ab86-79a762fddb7d tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:47.634017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:47.635180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:47.654060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:47.654980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:47.657993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:47.659453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:47.660099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:47.660531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:47.919436 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23e3c177-cf47-4f92-ab86-79a762fddb7d tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.792529 Oct 15 20:54:47.920318 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1047/1953] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:47 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:47.929904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-907b4c5c-7eb8-4584-8d0a-5e295da6a433 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update', calling method: .version_select at 0x7fe74f239430>, body: {"meta": {"os_version": "alt"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:48.063896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-907b4c5c-7eb8-4584-8d0a-5e295da6a433 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:48.076727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-907b4c5c-7eb8-4584-8d0a-5e295da6a433 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:48.672295 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-907b4c5c-7eb8-4584-8d0a-5e295da6a433 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata/os_version" status: 200 len: 31 microversion: 2.1 time: 0.745651 Oct 15 20:54:48.675136 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1048/1954] 10.222.0.16 () {60 vars in 1313 bytes} [Sat Oct 15 20:54:47 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata/os_version => generated 31 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:54:48.689159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d67bfa39-68fb-4468-b29f-368b85b68e4f tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f1525e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:48.772656 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d67bfa39-68fb-4468-b29f-368b85b68e4f tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 58 microversion: 2.1 time: 0.091232 Oct 15 20:54:48.774141 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1049/1955] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:54:48 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 58 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:48.786017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6c5f7e4e-2151-43ee-bbd3-a01f9526bc25 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'update_all', calling method: .version_select at 0x7fe7546204c0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:48.861988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:48.890637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:48.894954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:48.897655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6c5f7e4e-2151-43ee-bbd3-a01f9526bc25 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:48.901976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-169291a6-1fc7-4ded-98ba-f2328206716f tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 1.274485 Oct 15 20:54:48.902756 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 907/1956] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:47 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 1277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:48.931625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6c5f7e4e-2151-43ee-bbd3-a01f9526bc25 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:49.318633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsV221TestJSON-server-1032446937", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "cadfc2ff-c772-4612-8737-4ebb0f654e97"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:49.373714 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6c5f7e4e-2151-43ee-bbd3-a01f9526bc25 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 61 microversion: 2.1 time: 0.591700 Oct 15 20:54:49.374320 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1050/1957] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:54:48 2022] PUT /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 61 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:49.384224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-357b8838-5705-4aa4-862a-e08072842490 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Action: 'create', calling method: .version_select at 0x7fe74f235c10>, body: {"metadata": {"os_version": "alt1", "architecture": "value3"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:54:49.487488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] validate_networks() for [('cadfc2ff-c772-4612-8737-4ebb0f654e97', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:54:49.506497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-357b8838-5705-4aa4-862a-e08072842490 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:49.513081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-357b8838-5705-4aa4-862a-e08072842490 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Getting quotas for user cfcea0f8ebea419eb6f74c1091d8fe41 and project 8e65df04cd7f4c6cb91f322aeadc965f. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:50.249532 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-357b8838-5705-4aa4-862a-e08072842490 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "POST /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 85 microversion: 2.1 time: 0.865208 Oct 15 20:54:50.253426 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1051/1958] 10.222.0.16 () {60 vars in 1281 bytes} [Sat Oct 15 20:54:49 2022] POST /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 85 bytes in 874 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:50.269638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:50.270371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:50.278690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:50.279272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:50.279724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:50.280512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:50.280983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:50.281487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:51.097566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:51.121093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:51.124648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:51.129597 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4993a2ff-6486-4a6d-b5a9-33a762e6f0f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.872194 Oct 15 20:54:51.146488 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1052/1959] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:50 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:51.158523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8193607c-4b81-436b-b01a-3ec84a8a2cf1 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] Calling method '.version_select at 0x7fe74f152a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:51.209233 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8193607c-4b81-436b-b01a-3ec84a8a2cf1 tempest-ImagesMetadataTestJSON-18767953 tempest-ImagesMetadataTestJSON-18767953-project] 10.222.0.16 "GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata" status: 200 len: 85 microversion: 2.1 time: 0.060927 Oct 15 20:54:51.209686 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1053/1960] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:54:51 2022] GET /compute/v2.1/images/938ff769-4e7c-4794-ac9f-2c630beb1c85/metadata => generated 85 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:54:52.159951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:52.161494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:52.183949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:52.184490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:52.189436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:52.190275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:52.190784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:52.191253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:52.548596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:54:52.549151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:54:52.549905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:54:52.550778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:54:52.551207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:54:52.573195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:52.573616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:52.576677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:52.580474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:52.580967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:52.581430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:52.664485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Getting quotas for project 421588a6feed49f28f8ab75d089c01dc. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:54:52.679090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Getting quotas for user 42766820f2a940a3bf3ff646bd358414 and project 421588a6feed49f28f8ab75d089c01dc. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:54:52.721489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:54:52.726573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:54:52.830042 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-11bcd32c-efff-4c06-a2e9-764e79037d00 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.21 time: 3.777669 Oct 15 20:54:52.830596 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 908/1961] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:54:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 3779 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:54:52.848847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-462ed30d-6ddb-4174-b506-b6d529bfa95d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:52.849795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-462ed30d-6ddb-4174-b506-b6d529bfa95d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:52.865950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=7,nova_api:DELETE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:52.940229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:52.952740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:52.956122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:52.957840 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5b130b1-8ffd-4e39-a6f6-b6b7035a9864 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.802178 Oct 15 20:54:52.959430 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1054/1962] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:52 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:53.116674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-462ed30d-6ddb-4174-b506-b6d529bfa95d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:53.135326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-462ed30d-6ddb-4174-b506-b6d529bfa95d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:53.136786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-462ed30d-6ddb-4174-b506-b6d529bfa95d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:53.140850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-462ed30d-6ddb-4174-b506-b6d529bfa95d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1259 microversion: 2.21 time: 0.302643 Oct 15 20:54:53.143788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 909/1963] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:52 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1259 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:53.989011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:53.989933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:53.998817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:53.999320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:53.999766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:54.000493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:54.012428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:54.013142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:54.176627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:54.178225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:54.200441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:54.207764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:54.209708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:54.212072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:54.214137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:54.218024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:54.413188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:54.440991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:54.443480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:54.444941 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c2be85fc-5969-4729-9d9d-a757fa2ca583 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1814 microversion: 2.25 time: 0.459813 Oct 15 20:54:54.446481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1055/1964] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:53 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1814 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:54.631200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:54.650566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:54.658128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:54.659883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60a88c7e-6842-45c2-8f8f-6e0daf415fb0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1263 microversion: 2.21 time: 0.492268 Oct 15 20:54:54.660788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 910/1965] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:54 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1263 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:55.474804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:55.479384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:55.522083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:55.524922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:55.527820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:55.528720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:55.529252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:55.529815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:55.689114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:55.689733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:55.724675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:55.725611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:55.727259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:55.733785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:55.735046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:55.736884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:55.899175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:55.942213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:55.949590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:55.957779 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da20e09e-34b7-432a-a55e-91e483bbaeab tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1804 microversion: 2.25 time: 0.485234 Oct 15 20:54:55.960483 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1056/1966] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:55 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1804 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:55.993107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f3ccedad-425b-4d24-b3e5-36ce19e99771 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '.version_select at 0x7fe74f279670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:55.999482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3ccedad-425b-4d24-b3e5-36ce19e99771 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:56.001117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3ccedad-425b-4d24-b3e5-36ce19e99771 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:56.004116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f3ccedad-425b-4d24-b3e5-36ce19e99771 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:56.041391 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f3ccedad-425b-4d24-b3e5-36ce19e99771 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 2783 microversion: 2.25 time: 0.055683 Oct 15 20:54:56.043564 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1057/1967] 10.222.0.16 () {60 vars in 1184 bytes} [Sat Oct 15 20:54:55 2022] GET /compute/v2.1/os-migrations => generated 2783 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:56.068276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:56.069935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:56.083600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:56.084617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:56.089643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:56.092398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:56.093987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:56.095016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:56.118035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:56.136129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:56.137688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:56.146244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b130bd5-6576-493d-993c-88876db2f9aa tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1317 microversion: 2.21 time: 0.460817 Oct 15 20:54:56.149581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 911/1968] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:55 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1317 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 20:54:56.587717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid c05e4f55-245d-45f2-8567-e665ccbca061 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:56.663408 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3aa754d4-1ca1-4ef6-ad26-e1a423da83d1 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 2423 microversion: 2.25 time: 0.604417 Oct 15 20:54:56.681068 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1058/1969] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:56 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 2423 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:56.715924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:56.717358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:56.736051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:56.736574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:56.736922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:56.737676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:56.738141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:56.738576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:57.182444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:57.183564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:57.219180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:57.224045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:57.226120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:57.231236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:57.232736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:57.234566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:57.295770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=90 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:57.424438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:57.440434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:57.445666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:57.447259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ecd3a875-b70e-49b1-b99f-eddd869f19b0 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1317 microversion: 2.21 time: 0.271926 Oct 15 20:54:57.465315 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1059/1970] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:57 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1317 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:57.770104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] Lazy-loading 'services' on Instance uuid c05e4f55-245d-45f2-8567-e665ccbca061 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:57.798652 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9ea0ad1-beac-498d-af04-8591a2391e26 tempest-LiveAutoBlockMigrationV225Test-488829451 tempest-LiveAutoBlockMigrationV225Test-488829451-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 2423 microversion: 2.25 time: 1.102232 Oct 15 20:54:57.813821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 912/1971] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:56 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 2423 bytes in 1118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:58.009883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:58.011041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:58.019121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:58.020031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:58.020742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:58.021897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:58.022564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:58.023212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:58.323250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lazy-loading 'services' on Instance uuid c05e4f55-245d-45f2-8567-e665ccbca061 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:54:58.480521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:58.481240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:58.492628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:58.494300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:58.495855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:58.500663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:58.501259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:58.501713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:58.653568 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb689721-fb18-40ab-a793-e0ff1a2e3cc4 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments/a02d2cad-c453-42f6-aabc-ca1baa1d0574" status: 202 len: 0 microversion: 2.25 time: 0.645259 Oct 15 20:54:58.655631 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1060/1972] 10.222.0.16 () {60 vars in 1457 bytes} [Sat Oct 15 20:54:58 2022] DELETE /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments/a02d2cad-c453-42f6-aabc-ca1baa1d0574 => generated 0 bytes in 650 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:54:58.672138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:58.680132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:58.697181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:58.698643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:58.699357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:58.701073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:58.701806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:58.703837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:58.760303 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d65cc590-81f5-46f9-bbda-d8ffa7b36733 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments" status: 200 len: 197 microversion: 2.25 time: 0.100557 Oct 15 20:54:58.761564 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1061/1973] 10.222.0.16 () {60 vars in 1343 bytes} [Sat Oct 15 20:54:58 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments => generated 197 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:54:58.814003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:58.831950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:58.833483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:54:58.834631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bde25774-4425-4545-b74e-176c03c026e7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1317 microversion: 2.21 time: 0.362205 Oct 15 20:54:58.857128 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 913/1974] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:58 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1317 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:54:59.774386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:59.784881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:59.795499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=68 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:54:59.800559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:59.802354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:59.803247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:59.804848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:59.806666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:59.808711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:59.867773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:54:59.869049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:54:59.876248 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa937272-f77b-4d78-a396-4eca8db564b5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments" status: 200 len: 25 microversion: 2.25 time: 0.107221 Oct 15 20:54:59.877998 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1062/1975] 10.222.0.16 () {60 vars in 1343 bytes} [Sat Oct 15 20:54:59 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061/os-volume_attachments => generated 25 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:54:59.885790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:59.886273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:59.886711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:54:59.887444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:54:59.887886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:54:59.889232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:00.549225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-2031437699", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:00.676487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:00.693168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:00.723754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:00.731581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:00.736222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2081a757-cecc-41cf-814d-d53e54fde4cd tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 0.872603 Oct 15 20:55:00.738566 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 914/1976] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:54:59 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 876 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:01.779396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:01.781117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:01.794465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:01.795387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:01.796098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:01.796676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:01.798052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:01.805290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:01.806227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:01.806922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:01.807973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:01.808749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:01.809304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:01.824293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:01.827479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:01.827946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:01.835304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:01.835304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:01.835304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:01.924322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:01.930552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:01.939433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:01.970153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:01.989750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:02.145555 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d34a7ad4-0f34-479b-9ea3-e5b695cc648f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.918397 Oct 15 20:55:02.147168 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1063/1977] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 1923 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:55:02.162191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70f85039-6661-4cb6-9efe-0c2e547d574c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:02.163073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70f85039-6661-4cb6-9efe-0c2e547d574c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:02.190769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:02.450737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70f85039-6661-4cb6-9efe-0c2e547d574c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:02.463747 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70f85039-6661-4cb6-9efe-0c2e547d574c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1215 microversion: 2.1 time: 0.306700 Oct 15 20:55:02.475542 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1064/1978] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:02 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1215 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:02.616048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:02.783704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:02.815304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:02.822493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:02.829418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24534123-8ed0-4090-8622-a2ab8a61a4e4 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 1.059255 Oct 15 20:55:02.832821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 915/1979] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:01 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 1066 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:03.499198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:03.499198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:03.512082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:03.512592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:03.513118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:03.517854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:03.518352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:03.518797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:03.733882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:03.749248 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cdd971e8-aa18-44a1-92cb-40fce1ceda9f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1219 microversion: 2.1 time: 0.264506 Oct 15 20:55:03.754816 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1065/1980] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:03 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1219 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:03.819618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=36 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:03.841396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-2e061c93-1666-4bfe-a9bc-3237a5fa5a48 req-5d3e0052-a6af-4835-8e6e-0c0aee9e9509 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a851ba31-b7e6-4f6f-b288-50a8c13c2b81", "tag": "5022ebbe-b66a-4f00-95c9-6a08c60165b8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:03.861560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e061c93-1666-4bfe-a9bc-3237a5fa5a48 req-5d3e0052-a6af-4835-8e6e-0c0aee9e9509 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:03.862125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e061c93-1666-4bfe-a9bc-3237a5fa5a48 req-5d3e0052-a6af-4835-8e6e-0c0aee9e9509 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:03.862566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e061c93-1666-4bfe-a9bc-3237a5fa5a48 req-5d3e0052-a6af-4835-8e6e-0c0aee9e9509 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:03.923259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-2e061c93-1666-4bfe-a9bc-3237a5fa5a48 req-5d3e0052-a6af-4835-8e6e-0c0aee9e9509 service nova] Creating event network-changed:5022ebbe-b66a-4f00-95c9-6a08c60165b8 for instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 on nch1-master-1 Oct 15 20:55:03.945596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-2e061c93-1666-4bfe-a9bc-3237a5fa5a48 req-5d3e0052-a6af-4835-8e6e-0c0aee9e9509 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.112243 Oct 15 20:55:03.953670 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1066/1981] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (3 switches on core 0) Oct 15 20:55:03.982456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67869c66-474f-4896-a1a4-4ac58f253018 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Action: 'create', calling method: .version_select at 0x7fe7584d1af0>, body: {"keypair": {"name": "tempest-keypair-681472305", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH/DldPh/EJ36RvJTNes1WW7ctZwJyJpU+XcGQAzUEDr6IUvB3owHp9mpr79UWInQXwni8qmy+HAPw6VUT7TSEo9OOfKFQw4YChO4mUjwc2N4mfm8EFw8nPr//2BhCqeEQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:03.991184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:03.998016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:04.025901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:04.026694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:04.029648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:04.031697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:04.035246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:04.035717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:04.049188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-67869c66-474f-4896-a1a4-4ac58f253018 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Getting quotas for project 0a0d3787e2a842029da9fb7d0a6715b0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:04.080237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-67869c66-474f-4896-a1a4-4ac58f253018 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Getting quotas for user de4d5d0008324d7a807a8f8361a60a4e and project 0a0d3787e2a842029da9fb7d0a6715b0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:04.143939 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67869c66-474f-4896-a1a4-4ac58f253018 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.328774 Oct 15 20:55:04.144845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 916/1982] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:55:03 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:55:04.150778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f523d78d-6ead-4704-a932-ccf48008a662"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:04.162626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:04.198818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:04.199331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:04.204053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:04.205373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:04.207568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:04.209238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:04.849531 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07a541c7-ac5d-4951-ab4b-a4ea4c3ec26f tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "POST /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.703318 Oct 15 20:55:04.852177 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 917/1983] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 20:55:04 2022] POST /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments => generated 194 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:04.862742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:04.866316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:04.883635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:04.886182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:04.888694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:04.890669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:04.891683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:04.893857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:05.303371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:05.315576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:05.318152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:05.319579 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d50c7a62-9eec-4141-9c43-fbcfb17274ae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 1.360642 Oct 15 20:55:05.320351 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1067/1984] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:03 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 1366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:05.527220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:05.533967 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1623a3e-2924-4a88-b0e2-1703e69b0efc tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1275 microversion: 2.1 time: 0.680416 Oct 15 20:55:05.541357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 918/1985] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:04 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1275 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:06.352107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:06.353380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:06.369977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:06.371197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:06.371998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:06.374074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:06.376068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:06.376685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:06.560945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:06.562265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:06.574768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:06.575302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:06.575809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:06.616525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:06.625277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.040s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:06.625898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:06.958796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:06.976641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:07.005017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:07.006272 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c71e8fe-6759-452b-a538-b2d5c9caf634 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 0.658280 Oct 15 20:55:07.023206 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1068/1986] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:06 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:07.282357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:07.303829 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-42cc7c39-2695-47d5-b1d4-f4e68a74ae61 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1273 microversion: 2.1 time: 0.741914 Oct 15 20:55:07.308580 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 919/1987] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:06 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1273 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:08.060772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:08.061582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:08.077418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:08.080499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:08.081331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:08.083238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:08.084952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:08.085602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:08.347046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:08.348520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:08.375448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:08.377187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:08.378111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:08.379197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:08.379834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:08.382289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:08.564063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:08.577102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:08.580948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:08.597185 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-694da75c-d311-4692-baa7-2fc35f78b64d tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 0.529434 Oct 15 20:55:08.602443 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1069/1988] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:08 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:09.408705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-775938565", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c130cb51-e60d-43b7-af80-10e77fbe9595"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:09.525512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] validate_networks() for [('c130cb51-e60d-43b7-af80-10e77fbe9595', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:09.571088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:09.634908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5abf83df-4ead-4579-bcbd-9f3e6b43c9e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 1.299793 Oct 15 20:55:09.635888 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 920/1989] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:08 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 1309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:09.650818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:09.654150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:09.672670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:09.675988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:09.677964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:09.680533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:09.682809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:09.683281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:10.435964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:10.436540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:10.437162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:10.437688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:10.438229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:10.455404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:10.458661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:10.459475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:10.461920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:10.462676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:10.467384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:10.519412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:10.527312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for user 5d93a680570f4d23b1f6df534bc082b2 and project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:10.560949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:10.567932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:10.662646 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e090abe0-ff19-48a2-a407-c2de8504e68b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.360032 Oct 15 20:55:10.663869 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1070/1990] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 1363 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:10.961365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:10.987456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:10.997651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:11.002241 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c5243c83-9508-49c1-8e3d-fb0a5ae38317 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 1.364128 Oct 15 20:55:11.009816 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 921/1991] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:09 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 1370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:11.036282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "e2b82935-5991-4072-a68a-4084bcb19faa"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:11.036468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:11.038190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:11.039158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:11.070082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:11.071027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:11.071765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:11.073293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:11.073943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:11.074544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:11.136503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:11.137077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:11.137529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:11.139766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:11.140247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:11.140686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:11.405185 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] HTTP exception thrown: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Request method failure captured: Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: request: PUT /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments/f523d78d-6ead-4704-a932-ccf48008a662 HTTP/1.1 Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 74 Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx3uDC4kuG6vKusz52P-ssnduWNqHCC6WIn48pnxd38rHgktfUat4wsu8Bo38cplWf2yTAm5E3swWjnbSR46SDRJALjE-_4n82GKKuXX9iO-4jeEDXZw4qp6UXIgiaP4_XRs74itFn90juf3wZBJLyTxc2dmlMdXeRpz7wac8m_5HEk Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 971ec797f8d94d0989654cfa2d952a80 Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesAdminNegativeTest-477462580 Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 20:55:11.454454 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesAdminNegativeTest-477462580 Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 971ec797f8d94d0989654cfa2d952a80 Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-477462580 Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesAdminNegativeTest-477462580-project-admin Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 9749157277104da9bc3ffeef563adeb4 Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesAdminNegativeTest-477462580-project-admin Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: {"volumeAttachment": {"volumeId": "e2b82935-5991-4072-a68a-4084bcb19faa"}} Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:55:11.456784 ncdv-master-1 devstack@n-api.service[138689]: exception: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'server_id': '062c4651-90be-430a-a69b-372fac258ebb', 'id': 'f523d78d-6ead-4704-a932-ccf48008a662', 'body': {'volumeAttachment': {'volumeId': 'e2b82935-5991-4072-a68a-4084bcb19faa'}}} Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 20:55:11.458195 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. (HTTP 404) (Request-ID: req-6d66e226-38e6-4dd3-8855-a712d5b75718) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 436, in _update_volume_swap Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi new_volume = self.volume_api.get(context, new_volume_id) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 20:55:11.459623 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:11.461209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 445, in _update_volume_swap Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. Oct 15 20:55:11.462759 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:11.470671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Returning 400 to user: Volume e2b82935-5991-4072-a68a-4084bcb19faa could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:11.472396 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f651cf5-19ee-4981-80ee-d7d389b948c4 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] 10.222.0.16 "PUT /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments/f523d78d-6ead-4704-a932-ccf48008a662" status: 400 len: 107 microversion: 2.1 time: 0.807335 Oct 15 20:55:11.480214 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1071/1992] 10.222.0.16 () {60 vars in 1433 bytes} [Sat Oct 15 20:55:10 2022] PUT /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments/f523d78d-6ead-4704-a932-ccf48008a662 => generated 107 bytes in 815 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:55:11.487014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0b4736f-799f-4670-9594-1cc48d5259cc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:11.490704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c0b4736f-799f-4670-9594-1cc48d5259cc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:11.624850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c0b4736f-799f-4670-9594-1cc48d5259cc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:11.645710 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c0b4736f-799f-4670-9594-1cc48d5259cc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1207 microversion: 2.1 time: 0.163640 Oct 15 20:55:11.646837 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1072/1993] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:11 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1207 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:11.661943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestManualDisk-server-1849259967", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "d34cda32-f9fd-441b-9f18-4ca595a015a2"}], "security_groups": [{"name": "tempest-securitygroup--1500588122"}], "key_name": "tempest-keypair-681472305", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "OS-DCF:diskConfig": "}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:11.869459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] validate_networks() for [('d34cda32-f9fd-441b-9f18-4ca595a015a2', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:11.923014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:11.938260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e7630fa0-12b8-4281-a1f2-044988049ca5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.924233 Oct 15 20:55:11.940413 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 922/1994] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:11 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 930 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:11.953100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:11.955128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:11.971758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:11.973667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:11.974402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:11.975416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:11.976393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:11.977643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:12.193000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lazy-loading 'services' on Instance uuid 062c4651-90be-430a-a69b-372fac258ebb {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:55:12.368396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a774e25-7262-4b60-88a1-8e3566db508e tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "DELETE /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments/f523d78d-6ead-4704-a932-ccf48008a662" status: 202 len: 0 microversion: 2.1 time: 0.425884 Oct 15 20:55:12.371791 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 923/1995] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 20:55:11 2022] DELETE /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments/f523d78d-6ead-4704-a932-ccf48008a662 => generated 0 bytes in 430 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:55:12.392939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:12.394597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:12.408664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:12.410291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:12.413205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:12.414182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:12.415475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:12.416022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:12.893684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Getting quotas for project 0a0d3787e2a842029da9fb7d0a6715b0. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:12.909529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Getting quotas for user de4d5d0008324d7a807a8f8361a60a4e and project 0a0d3787e2a842029da9fb7d0a6715b0. Resources: dict_keys(['metadata_items']) {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:12.926659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:12.927088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:12.927856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:12.928276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:12.928997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:12.941019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:12.942002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:12.942831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:12.944684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:12.945518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:12.945845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:12.976366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Getting quotas for project 0a0d3787e2a842029da9fb7d0a6715b0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:12.991082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Getting quotas for user de4d5d0008324d7a807a8f8361a60a4e and project 0a0d3787e2a842029da9fb7d0a6715b0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:13.017450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:13.056084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:13.149475 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-072f9359-c208-4504-af59-aec33223bef4 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.501189 Oct 15 20:55:13.151232 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1073/1996] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:11 2022] POST /compute/v2.1/servers => generated 409 bytes in 1504 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:13.160305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e6238ac1-b03e-4b56-b720-0673e53ac34f req-7c2ba0e7-889f-4ccf-81e9-0b5e74d1c706 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a", "tag": "b8b5e35e-5f7f-41f0-b601-d0d07c25b03a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:13.188450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e6238ac1-b03e-4b56-b720-0673e53ac34f req-7c2ba0e7-889f-4ccf-81e9-0b5e74d1c706 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.203287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e6238ac1-b03e-4b56-b720-0673e53ac34f req-7c2ba0e7-889f-4ccf-81e9-0b5e74d1c706 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.204266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e6238ac1-b03e-4b56-b720-0673e53ac34f req-7c2ba0e7-889f-4ccf-81e9-0b5e74d1c706 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.230754 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e6238ac1-b03e-4b56-b720-0673e53ac34f req-7c2ba0e7-889f-4ccf-81e9-0b5e74d1c706 service nova] Creating event network-changed:b8b5e35e-5f7f-41f0-b601-d0d07c25b03a for instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a on nch2-master-1 Oct 15 20:55:13.275130 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e6238ac1-b03e-4b56-b720-0673e53ac34f req-7c2ba0e7-889f-4ccf-81e9-0b5e74d1c706 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.122501 Oct 15 20:55:13.282430 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1074/1997] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:13.304190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:13.307158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:13.326599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.329704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.330753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.336995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.338619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.344061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.422244 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8941e077-b513-4e41-a43b-657283716fc9 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.138108 Oct 15 20:55:13.426075 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1075/1998] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:55:13 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments => generated 197 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:13.436735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:13.437633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:13.467257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.467257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.467257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.467257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.467624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.467624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.587699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:13.597739 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-77bf591a-26f2-4b91-b12a-fe94aa4ecbbe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1267 microversion: 2.1 time: 0.170332 Oct 15 20:55:13.598340 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1076/1999] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:13 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1267 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:13.614540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:13.615235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:13.632689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.633234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.633657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.635407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:13.636021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:13.636687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:13.870983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:13.885617 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1104f710-6195-40c2-aa70-1716ae0e0d24 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.286178 Oct 15 20:55:13.886794 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1077/2000] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:13 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:13.892392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17df1371-78a2-4e37-9266-6c4732e86303 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:13.898042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17df1371-78a2-4e37-9266-6c4732e86303 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:13.996356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:14.007253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:14.010651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:14.015456 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86b6a145-7e73-43cd-b338-54a3512e9c3e tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 1.641749 Oct 15 20:55:14.016277 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 924/2001] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:12 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 1644 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:14.021721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d3b002fe-8561-4e25-8ca3-f7a6865eb619 req-48d98895-b30f-4558-857c-2589ed9e6139 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a851ba31-b7e6-4f6f-b288-50a8c13c2b81", "name": "network-vif-plugged", "status": "completed", "tag": "5022ebbe-b66a-4f00-95c9-6a08c60165b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:14.026116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17df1371-78a2-4e37-9266-6c4732e86303 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:14.045547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d3b002fe-8561-4e25-8ca3-f7a6865eb619 req-48d98895-b30f-4558-857c-2589ed9e6139 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.046087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d3b002fe-8561-4e25-8ca3-f7a6865eb619 req-48d98895-b30f-4558-857c-2589ed9e6139 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.046595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d3b002fe-8561-4e25-8ca3-f7a6865eb619 req-48d98895-b30f-4558-857c-2589ed9e6139 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.048891 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17df1371-78a2-4e37-9266-6c4732e86303 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1277 microversion: 2.1 time: 0.159789 Oct 15 20:55:14.051419 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1078/2002] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:13 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1277 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:14.075019 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d3b002fe-8561-4e25-8ca3-f7a6865eb619 req-48d98895-b30f-4558-857c-2589ed9e6139 service nova] Creating event network-vif-plugged:5022ebbe-b66a-4f00-95c9-6a08c60165b8 for instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 on nch1-master-1 Oct 15 20:55:14.090675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d3b002fe-8561-4e25-8ca3-f7a6865eb619 req-48d98895-b30f-4558-857c-2589ed9e6139 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072328 Oct 15 20:55:14.092021 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 925/2003] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:14.439941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:14.442331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:14.462286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.469548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.469548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.469548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.469548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.473126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.576697 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d75f301-6963-4f18-a584-088f83321bc0 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.141844 Oct 15 20:55:14.584554 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1079/2004] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:55:14 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb/os-volume_attachments => generated 25 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:55:14.618790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:14.625734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:14.626465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:14.645694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.646575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.647244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.648217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.661970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.668847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.816984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:14.837583 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a0fa533-85b0-4c2c-860b-d6f436119df3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1265 microversion: 2.1 time: 0.219599 Oct 15 20:55:14.838480 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 926/2005] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:14 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1265 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:14.916619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:14.917554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-1297770847", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "f88dd690-f671-4f8e-8831-3ff9fe36c6b1"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:14.919027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:14.932890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.933428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.933858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.936678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:14.937179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:14.937615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:14.991622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] validate_networks() for [('f88dd690-f671-4f8e-8831-3ff9fe36c6b1', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:15.332811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:15.373697 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae5e894e-f529-49dd-af75-f1574c5feab4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.466184 Oct 15 20:55:15.374738 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 927/2006] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:14 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:15.384354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:15.385312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:15.392863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:15.397798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:15.399169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:15.400932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:15.401804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:15.402465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.008331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:16.008998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:16.009642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:16.010124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:16.010640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:16.025709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.026476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.027027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.027976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.029314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.030069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.059326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Getting quotas for project e08b85b4f6cf47a88fe24ead8e6d45e9. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:16.072244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Getting quotas for user ec8dcd7615cc43b3866034d5af34a9a1 and project e08b85b4f6cf47a88fe24ead8e6d45e9. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:16.101242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:16.105540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:16.158252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.174470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.176699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.181774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c70b156-eba8-405d-8a87-3215bd93d028 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1359 microversion: 2.21 time: 0.805659 Oct 15 20:55:16.184791 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 928/2007] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:15 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1359 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:16.213340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:16.219038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:16.224105 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-95ffe3a7-8671-41ef-9109-8adf32fbe170 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.318165 Oct 15 20:55:16.225425 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1080/2008] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 1320 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:16.230568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:16.231265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:16.249280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.249280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.249280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.249280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.249735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.249735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.260039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.262742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.265392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.270211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.270973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.271721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.500469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.515717 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-80c688af-2c4c-4d16-b1c1-fb3b114a6633 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.288975 Oct 15 20:55:16.532263 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1081/2009] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:16 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:16.532263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e076cc70-3f11-43cf-9c38-1f3005a1fbb1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:16.532263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e076cc70-3f11-43cf-9c38-1f3005a1fbb1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:16.565512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.583141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6bad2c1-da66-4e75-890e-90adb609be72 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1329 microversion: 2.1 time: 0.396329 Oct 15 20:55:16.588573 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 929/2010] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:16 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1329 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:16.596896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:16.602240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:16.614359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.615764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.616853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.619316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:16.621946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:16.622652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:16.630565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e076cc70-3f11-43cf-9c38-1f3005a1fbb1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.654387 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e076cc70-3f11-43cf-9c38-1f3005a1fbb1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1218 microversion: 2.1 time: 0.134053 Oct 15 20:55:16.657797 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1082/2011] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:16 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1218 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:16.896674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:16.910907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f75628f5-6513-448a-87d0-5c5ffc5685db tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.316649 Oct 15 20:55:16.922189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 930/2012] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:16 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:17.128385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-72d7a03b-1d39-4534-9366-f3d9d4910023 req-74fa55d1-af44-4a6c-a0d8-c684835cee74 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7dafc5b2-733a-433a-bbcc-5136827624c7", "tag": "cd9552df-0245-4ae6-8b4f-63fd9d25ec69"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:17.145341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72d7a03b-1d39-4534-9366-f3d9d4910023 req-74fa55d1-af44-4a6c-a0d8-c684835cee74 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.146065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72d7a03b-1d39-4534-9366-f3d9d4910023 req-74fa55d1-af44-4a6c-a0d8-c684835cee74 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.157765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72d7a03b-1d39-4534-9366-f3d9d4910023 req-74fa55d1-af44-4a6c-a0d8-c684835cee74 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.170433 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-72d7a03b-1d39-4534-9366-f3d9d4910023 req-74fa55d1-af44-4a6c-a0d8-c684835cee74 service nova] Creating event network-changed:cd9552df-0245-4ae6-8b4f-63fd9d25ec69 for instance 7dafc5b2-733a-433a-bbcc-5136827624c7 on nch1-master-1 Oct 15 20:55:17.182856 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-72d7a03b-1d39-4534-9366-f3d9d4910023 req-74fa55d1-af44-4a6c-a0d8-c684835cee74 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057908 Oct 15 20:55:17.183684 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1083/2013] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:17.220988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:17.227280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:17.234960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.235731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.237227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.242884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.243382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.244104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.541513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:17.544440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:17.545086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:17.545958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:17.551220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:17.552652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:17.553024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.553610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.553837 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-624b1716-0aad-4f2e-b0f4-7a206b126adb tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1552 microversion: 2.21 time: 0.343338 Oct 15 20:55:17.554169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.565165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 931/2014] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:17 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1552 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:17.565404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.565404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.565404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.565404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:17.587109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:17.587806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:17.597866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.600202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.600853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.601791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.602420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.603034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.654023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:55:17.805538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c04d6662-f3d6-45eb-a2a8-54fe57d27279 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 204 len: 0 microversion: 2.21 time: 0.222444 Oct 15 20:55:17.806143 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 932/2015] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:55:17 2022] DELETE /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 15 20:55:17.809331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:17.813611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:17.814337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:17.822286 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-76a16030-7eb7-418f-b27b-206ad8afca6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.281935 Oct 15 20:55:17.822850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.823128 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1084/2016] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:17 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:17.837392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.837392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.837392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.837392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.838418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.838528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:17.838528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:17.839813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:17.844302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.850801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.851377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.852349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.852893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.853610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.938212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:17.947345 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-60088167-4bb9-42b3-9021-974e6af65948 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1278 microversion: 2.1 time: 0.123018 Oct 15 20:55:17.947941 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1085/2017] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:17 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1278 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:17.954947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:17.955843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:17.963696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.964204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.964928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.965877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:17.966174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:17.966731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:17.975473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:17.987219 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb65a13c-3af1-4419-b13d-4b8563853b21 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1337 microversion: 2.1 time: 0.179788 Oct 15 20:55:17.991381 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 933/2018] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:17 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1337 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:18.001882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:18.002638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:18.018826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:18.019636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:18.023267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:18.024824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:18.025993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:18.026696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:18.163522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:18.173203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:18.174656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:18.175828 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-340cc1f1-13ab-4202-a96f-156762fdd9ed tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1558 microversion: 2.21 time: 0.225579 Oct 15 20:55:18.176398 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1086/2019] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:17 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1558 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:18.395166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:18.403560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-23a0769b-2711-4a63-a008-bd343f92c5e6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.410892 Oct 15 20:55:18.404122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 934/2020] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:17 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:18.869284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:18.874024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:18.888635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:18.889366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:18.889866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:18.890611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:18.891268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:18.892063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:18.972285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:18.973089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:18.983046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:18.983863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:18.984507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:18.985542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:18.986239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:18.986846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.131385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:19.143780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce19ed67-8d89-4924-8293-97f267812645 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.293697 Oct 15 20:55:19.145473 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1087/2021] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:18 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:19.150404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:19.151275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:19.158518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:19.159284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:19.159892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.160955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:19.161628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:19.162722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.268771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:19.302604 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d41a97ab-fb1b-4c76-9f75-299cc1764bc5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1288 microversion: 2.1 time: 0.333060 Oct 15 20:55:19.303245 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 935/2022] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:18 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1288 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:19.314664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:19.315324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:19.337356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:19.338155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:19.339812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.343175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:19.343957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:19.344567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.401024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:19.415236 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b18a4559-64ce-4acf-be6b-3b491246bb51 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1335 microversion: 2.1 time: 0.268415 Oct 15 20:55:19.416274 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1088/2023] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:19 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1335 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:19.432364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:19.436532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:19.445819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:19.446644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:19.447086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.447821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:19.448287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:19.448721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:19.715612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:19.727404 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f75155a5-86c2-4d88-ae80-11c983b35d10 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.303986 Oct 15 20:55:19.742149 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1089/2024] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:19 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:20.138724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8db3ee59-f9cc-47af-ad98-aedc9e35e28c req-f024c0de-4895-4fca-9a1d-5bbe00c07149 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a851ba31-b7e6-4f6f-b288-50a8c13c2b81", "name": "network-vif-deleted", "tag": "5022ebbe-b66a-4f00-95c9-6a08c60165b8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:20.173963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8db3ee59-f9cc-47af-ad98-aedc9e35e28c req-f024c0de-4895-4fca-9a1d-5bbe00c07149 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.178342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8db3ee59-f9cc-47af-ad98-aedc9e35e28c req-f024c0de-4895-4fca-9a1d-5bbe00c07149 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.179212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8db3ee59-f9cc-47af-ad98-aedc9e35e28c req-f024c0de-4895-4fca-9a1d-5bbe00c07149 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.217514 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8db3ee59-f9cc-47af-ad98-aedc9e35e28c req-f024c0de-4895-4fca-9a1d-5bbe00c07149 service nova] Creating event network-vif-deleted:5022ebbe-b66a-4f00-95c9-6a08c60165b8 for instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 on nch1-master-1 Oct 15 20:55:20.229788 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8db3ee59-f9cc-47af-ad98-aedc9e35e28c req-f024c0de-4895-4fca-9a1d-5bbe00c07149 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094133 Oct 15 20:55:20.242096 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1090/2025] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 20:55:20.254788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:20.255941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:20.266413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.274710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.275420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.276199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.276682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.277206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.378283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:20.394104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:20.398693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:20.411581 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b7a4df9-52b7-42f3-b42e-461f6c0af014 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1558 microversion: 2.21 time: 1.106266 Oct 15 20:55:20.414176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 936/2026] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:19 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1558 bytes in 1110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:20.429714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:20.430404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:20.466813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.467570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.473319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.479483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.480273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.483601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.626112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:20.637342 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da1cccef-a304-406f-ab7a-0a4d926c6252 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.393005 Oct 15 20:55:20.639162 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1091/2027] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:20 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:20.644600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:20.645354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:20.654453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.663251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.664395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.665531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.666184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.666829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.673299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:20.690774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61e73bd0-1b92-45b6-8e48-c3710ff13292 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1276 microversion: 2.1 time: 0.276063 Oct 15 20:55:20.703691 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 937/2028] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:20 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1276 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:20.753229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:20.754289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:20.766234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.767284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.767999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:20.769274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:20.769966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:20.771775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.085841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:21.097838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4c9e526-bd54-47f2-9c14-68893465c2d5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.457285 Oct 15 20:55:21.098485 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1092/2029] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:20 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:21.189430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:21.205779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c92d9dda-a5e3-4e64-badf-11564c4f9d0c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.458349 Oct 15 20:55:21.206735 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 938/2030] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:20 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:21.458872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:21.459862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:21.471528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.473070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.473863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.474704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.475729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.476538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.529512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.530015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.530459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.548293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lazy-loading 'fault' on Instance uuid a851ba31-b7e6-4f6f-b288-50a8c13c2b81 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:55:21.625792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:21.635923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:21.638142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42766820f2a940a3bf3ff646bd358414', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '421588a6feed49f28f8ab75d089c01dc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:21.642176 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0764c95e-a06d-4dc8-971a-55d9a8a85ff7 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 200 len: 1349 microversion: 2.21 time: 0.187936 Oct 15 20:55:21.653264 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1093/2031] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:21 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 1349 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:55:21.666729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:21.668045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:21.676063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.676849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.677920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.678845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.679456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.680045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.737064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:21.738527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:21.751262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.752152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.753598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:21.756697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:21.758397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:21.760210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.155112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:22.177180 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-989536a7-f044-4605-8b1f-d57c5cdf16b1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.444786 Oct 15 20:55:22.181402 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1094/2032] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:21 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:22.188649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:22.195789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:22.213582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.213582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.213582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.213582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.213976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.213976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.472119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:22.486575 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aadfb67b-bd99-4721-8a29-7b1b02b3226a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.825557 Oct 15 20:55:22.505721 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 939/2033] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:21 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 845 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:22.513214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:22.513894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:22.520201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:22.521419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.522086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.522804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.523827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.525875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.532182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.535156 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-306338cc-303d-4626-a0ef-8aa2657a5cf8 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.351851 Oct 15 20:55:22.536531 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1095/2034] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:22 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:22.542538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-aa139a28-8780-4bf9-b2ce-23551e50ee2b req-d62981c5-724d-40cd-a6ab-86998c15bfed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a", "name": "network-vif-plugged", "status": "completed", "tag": "b8b5e35e-5f7f-41f0-b601-d0d07c25b03a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:22.562423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-aa139a28-8780-4bf9-b2ce-23551e50ee2b req-d62981c5-724d-40cd-a6ab-86998c15bfed service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.562978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-aa139a28-8780-4bf9-b2ce-23551e50ee2b req-d62981c5-724d-40cd-a6ab-86998c15bfed service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.563646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-aa139a28-8780-4bf9-b2ce-23551e50ee2b req-d62981c5-724d-40cd-a6ab-86998c15bfed service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.589928 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-aa139a28-8780-4bf9-b2ce-23551e50ee2b req-d62981c5-724d-40cd-a6ab-86998c15bfed service nova] Creating event network-vif-plugged:b8b5e35e-5f7f-41f0-b601-d0d07c25b03a for instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a on nch2-master-1 Oct 15 20:55:22.607712 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-aa139a28-8780-4bf9-b2ce-23551e50ee2b req-d62981c5-724d-40cd-a6ab-86998c15bfed service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068849 Oct 15 20:55:22.612340 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1096/2035] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:22.688454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:22.688454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:22.700198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.700198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.700600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.701563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.702209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.702782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.721863 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] HTTP exception thrown: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Request method failure captured: Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 HTTP/1.1 Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4Y-yydBKO88xUlT75nh7NYCLwZ-8aMSQr6wGBwwFnjY0CZa82JpMeG_UcpZs38w1ZV2-Z83beTV4G3BE51jc6FbDNfrphcZSPadkGd8GSoZBhQmaHIb8Xs8k2bLuzpx6piYdyAodshS0fwc-wssUGe3cdiXbWqhgrrsyG0zQJrWWU Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Openstack-Nova-Api-Version: 2.21 Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 421588a6feed49f28f8ab75d089c01dc Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:55:22.723977 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 421588a6feed49f28f8ab75d089c01dc Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-InstanceActionsV221TestJSON-1885966084-project Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 42766820f2a940a3bf3ff646bd358414 Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-InstanceActionsV221TestJSON-1885966084-project Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.732468 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'a851ba31-b7e6-4f6f-b288-50a8c13c2b81'} Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:22.733719 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.737444 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.739321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Returning 404 to user: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:22.739321 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-22e77988-f0be-453f-a2fc-5b84d62b7bae tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 404 len: 111 microversion: 2.21 time: 0.054002 Oct 15 20:55:22.742412 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1097/2036] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:22 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:55:22.758096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '.version_select at 0x7fe75847edc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:22.758096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:22.769927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.770477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.771244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.772128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.772604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.773062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.887054 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa7a2c92-989b-4aa1-b537-34f1628b3875 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81/os-instance-actions" status: 200 len: 611 microversion: 2.21 time: 0.137657 Oct 15 20:55:22.898915 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1098/2037] 10.222.0.16 () {60 vars in 1337 bytes} [Sat Oct 15 20:55:22 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81/os-instance-actions => generated 611 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 20:55:22.909138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:22.909865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:22.922214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.923155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.923884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.924769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.925736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.927067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.955566 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] HTTP exception thrown: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Request method failure captured: Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 HTTP/1.1 Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4Y-yydBKO88xUlT75nh7NYCLwZ-8aMSQr6wGBwwFnjY0CZa82JpMeG_UcpZs38w1ZV2-Z83beTV4G3BE51jc6FbDNfrphcZSPadkGd8GSoZBhQmaHIb8Xs8k2bLuzpx6piYdyAodshS0fwc-wssUGe3cdiXbWqhgrrsyG0zQJrWWU Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Openstack-Nova-Api-Version: 2.21 Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 421588a6feed49f28f8ab75d089c01dc Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:55:22.957296 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 421588a6feed49f28f8ab75d089c01dc Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-InstanceActionsV221TestJSON-1885966084-project Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 42766820f2a940a3bf3ff646bd358414 Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-InstanceActionsV221TestJSON-1885966084-project Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.959091 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'a851ba31-b7e6-4f6f-b288-50a8c13c2b81'} Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:22.960067 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:22.962602 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:22.964413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Returning 404 to user: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:22.964413 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7cd6a589-3de5-4e6e-aa62-2b5ac723e306 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 404 len: 111 microversion: 2.21 time: 0.060431 Oct 15 20:55:22.967657 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1099/2038] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:55:22 2022] DELETE /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 15 20:55:22.977407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:22.982768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] [instance: a851ba31-b7e6-4f6f-b288-50a8c13c2b81] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:22.994951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.996129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:22.996129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:22.998343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:22.999419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.000125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.021632 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] HTTP exception thrown: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Request method failure captured: Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 HTTP/1.1 Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSx4Y-yydBKO88xUlT75nh7NYCLwZ-8aMSQr6wGBwwFnjY0CZa82JpMeG_UcpZs38w1ZV2-Z83beTV4G3BE51jc6FbDNfrphcZSPadkGd8GSoZBhQmaHIb8Xs8k2bLuzpx6piYdyAodshS0fwc-wssUGe3cdiXbWqhgrrsyG0zQJrWWU Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Openstack-Nova-Api-Version: 2.21 Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 421588a6feed49f28f8ab75d089c01dc Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 20:55:23.023563 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/421588a6feed49f28f8ab75d089c01dc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 421588a6feed49f28f8ab75d089c01dc Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1885966084 Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-InstanceActionsV221TestJSON-1885966084-project Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 42766820f2a940a3bf3ff646bd358414 Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-InstanceActionsV221TestJSON-1885966084-project Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:23.025272 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'a851ba31-b7e6-4f6f-b288-50a8c13c2b81'} Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:23.026250 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. Oct 15 20:55:23.027692 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 20:55:23.029838 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] Returning 404 to user: Instance a851ba31-b7e6-4f6f-b288-50a8c13c2b81 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:23.029838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6bfb5518-ef8a-480b-a844-3bb196e53b07 tempest-InstanceActionsV221TestJSON-1885966084 tempest-InstanceActionsV221TestJSON-1885966084-project] 10.222.0.16 "GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81" status: 404 len: 111 microversion: 2.21 time: 0.056500 Oct 15 20:55:23.031613 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1100/2039] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:55:22 2022] GET /compute/v2.1/servers/a851ba31-b7e6-4f6f-b288-50a8c13c2b81 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:55:23.058568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:23.102989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c046bf10-40b9-45de-bf59-7680c7de43bf tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.595472 Oct 15 20:55:23.104707 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 940/2040] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:22 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:23.210864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:23.211628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:23.233300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.233300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.233300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.233300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.236311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.237024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.340370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-85824cd5-5f4d-4c92-a289-272a62f6290f req-0fad6a62-a2d0-4db5-b8ce-0a3ed69a2508 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e20e4c9-1939-425a-a7a8-d284327057bc", "tag": "5d6409b1-05df-4a6a-8356-24ea8f253baa"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:23.351127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:23.391442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-85824cd5-5f4d-4c92-a289-272a62f6290f req-0fad6a62-a2d0-4db5-b8ce-0a3ed69a2508 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.393343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-85824cd5-5f4d-4c92-a289-272a62f6290f req-0fad6a62-a2d0-4db5-b8ce-0a3ed69a2508 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.394090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-85824cd5-5f4d-4c92-a289-272a62f6290f req-0fad6a62-a2d0-4db5-b8ce-0a3ed69a2508 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.437403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-85824cd5-5f4d-4c92-a289-272a62f6290f req-0fad6a62-a2d0-4db5-b8ce-0a3ed69a2508 service nova] Creating event network-changed:5d6409b1-05df-4a6a-8356-24ea8f253baa for instance 8e20e4c9-1939-425a-a7a8-d284327057bc on nch2-master-1 Oct 15 20:55:23.456504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-85824cd5-5f4d-4c92-a289-272a62f6290f req-0fad6a62-a2d0-4db5-b8ce-0a3ed69a2508 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.121009 Oct 15 20:55:23.457721 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 941/2041] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:23.519596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:23.520984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:23.546195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.547525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.548155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.549144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.550198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.551050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.877490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:23.888505 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8d4c6026-1503-43d7-a450-32a650d9eb2d tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.683632 Oct 15 20:55:23.902380 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1101/2042] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:23 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:23.913100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:23.913883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:23.921989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.923251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.924148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:23.925952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:23.926713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:23.927468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:24.083038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:24.103889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25d4ab8a-3030-461d-ad49-9680681e1226 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.591641 Oct 15 20:55:24.106394 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 942/2043] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:23 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:24.130583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:24.132915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:24.145781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:24.146198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:24.146830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:24.147386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:24.147836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:24.148266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:24.249163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:24.264259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8f14b3e0-0ccb-41ee-8395-679f00270276 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.355636 Oct 15 20:55:24.265322 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1102/2044] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:23 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:24.442780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:24.451580 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82a09339-0cf0-4b3a-8f9b-8eb97abc7bbd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1315 microversion: 2.1 time: 0.325025 Oct 15 20:55:24.455467 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 943/2045] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:24 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1315 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:24.915766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:24.916317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:24.929584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:24.930515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:24.930652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:24.931707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:24.931872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:24.932308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.133261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:25.134534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:25.142651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.143145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.143584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.150841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.150841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.150841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.213072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:25.216036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:25.226406 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a731ef99-1b23-49cd-bfc3-47a869505673 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.314521 Oct 15 20:55:25.227090 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1103/2046] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:24 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:25.291833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:25.293426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:25.304095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.304686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.305461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.306901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.307869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.308662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.428841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:25.440161 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c264d59-3174-4631-abc7-2f49dd5068fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.312071 Oct 15 20:55:25.440763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 944/2047] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:25 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:25.478003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:25.478612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:25.489298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.490567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.494543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.496074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.496743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.497382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.622324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:25.634079 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9f6695ce-5f95-44a2-bab0-7970d253416f tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.346352 Oct 15 20:55:25.649068 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1104/2048] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:25 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:25.649068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c2347c41-88e8-4ee6-89f5-a2614d7651e2 req-f14b5491-7157-47a5-a1b1-5c639d0e394a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4c39d08f-a040-4454-a0a5-2c699cd98138", "tag": "55dff384-41ed-4361-b4c9-5f0a77f3d5d8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:25.661537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c2347c41-88e8-4ee6-89f5-a2614d7651e2 req-f14b5491-7157-47a5-a1b1-5c639d0e394a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.665931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c2347c41-88e8-4ee6-89f5-a2614d7651e2 req-f14b5491-7157-47a5-a1b1-5c639d0e394a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.665931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c2347c41-88e8-4ee6-89f5-a2614d7651e2 req-f14b5491-7157-47a5-a1b1-5c639d0e394a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.675438 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c2347c41-88e8-4ee6-89f5-a2614d7651e2 req-f14b5491-7157-47a5-a1b1-5c639d0e394a service nova] Creating event network-changed:55dff384-41ed-4361-b4c9-5f0a77f3d5d8 for instance 4c39d08f-a040-4454-a0a5-2c699cd98138 on nch1-master-1 Oct 15 20:55:25.680537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c2347c41-88e8-4ee6-89f5-a2614d7651e2 req-f14b5491-7157-47a5-a1b1-5c639d0e394a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037997 Oct 15 20:55:25.681127 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1105/2049] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:25.698722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:25.709075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c3b8de6-dc91-4a91-af4c-11702979aa9c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1502 microversion: 2.1 time: 0.235404 Oct 15 20:55:25.710312 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 945/2050] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:25 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1502 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:25.736504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:25.736682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:25.743090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.743580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.744024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.744843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.745332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.745762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.777492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:55:25.904409 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fa3ffaa1-e766-4b5b-8327-e248f5ccff2e tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 204 len: 0 microversion: 2.1 time: 0.172545 Oct 15 20:55:25.905817 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1106/2051] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:55:25 2022] DELETE /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:55:25.906507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f4696ea7-f57d-4f14-9abf-ffffe228afd4 req-689d1340-0c3d-4c0e-90c0-e2496dd6be69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dafc5b2-733a-433a-bbcc-5136827624c7", "name": "network-vif-plugged", "status": "completed", "tag": "cd9552df-0245-4ae6-8b4f-63fd9d25ec69"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:25.918242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:25.920918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:25.930274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.931165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4696ea7-f57d-4f14-9abf-ffffe228afd4 req-689d1340-0c3d-4c0e-90c0-e2496dd6be69 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.934103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.934103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.934509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4696ea7-f57d-4f14-9abf-ffffe228afd4 req-689d1340-0c3d-4c0e-90c0-e2496dd6be69 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.934639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:25.934746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4696ea7-f57d-4f14-9abf-ffffe228afd4 req-689d1340-0c3d-4c0e-90c0-e2496dd6be69 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.934848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:25.934848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:25.978387 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f4696ea7-f57d-4f14-9abf-ffffe228afd4 req-689d1340-0c3d-4c0e-90c0-e2496dd6be69 service nova] Creating event network-vif-plugged:cd9552df-0245-4ae6-8b4f-63fd9d25ec69 for instance 7dafc5b2-733a-433a-bbcc-5136827624c7 on nch1-master-1 Oct 15 20:55:25.989509 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f4696ea7-f57d-4f14-9abf-ffffe228afd4 req-689d1340-0c3d-4c0e-90c0-e2496dd6be69 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091184 Oct 15 20:55:25.990582 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 946/2052] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:26.044161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:26.140746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:26.152999 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f2ff5726-8262-4474-8fd1-fbb0b44f3845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1508 microversion: 2.1 time: 0.239761 Oct 15 20:55:26.154695 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1107/2053] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:25 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1508 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:26.253376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:26.255236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:26.268553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:26.270013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:26.272900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:26.273717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:26.274204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:26.274649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:26.469550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:26.470777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:26.479938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:26.480804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:26.481442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:26.482884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:26.484113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:26.484713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:26.541307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:26.552837 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-770fc386-e82e-4d23-8180-a724d0ae35d8 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.306531 Oct 15 20:55:26.553687 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 947/2054] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:26 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:26.643432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=36 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:26.660649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:26.661301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:26.667641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:26.668131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:26.668578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:26.676605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:26.677162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:26.677608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:26.687872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:26.710204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:26.726399 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff8cb8e2-5ac0-4b44-af79-168a3c207127 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1307 microversion: 2.1 time: 0.262319 Oct 15 20:55:26.727939 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1108/2055] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:26 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1307 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:26.742003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-2130616640", "imageRef": "", "flavorRef": "452", "networks": [{"uuid": "6bc761ff-f597-43f8-85e3-57edd2825dc3"}], "block_device_mapping_v2": [{"uuid": "065f0e82-0f1d-48e0-9c8a-7207edadc632", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:26.828547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] validate_networks() for [('6bc761ff-f597-43f8-85e3-57edd2825dc3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:27.090853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:27.100417 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-464abff6-64e8-461b-a26f-6a1e263fb7de tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.444354 Oct 15 20:55:27.108909 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 948/2056] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:26 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:27.180825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:27.181727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:27.188020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.188516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.188962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.189721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.190178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.190602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.313164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:27.314832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:27.315451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:27.315979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:27.316692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:27.331895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.332857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.333253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.337218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.337218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.337218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.353343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:27.363772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Getting quotas for user c613201f55b94a528c6c36e5b5772799 and project fe14ede74b2140269cd7dfc32fe14f9c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:27.438682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:27.440888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='065f0e82-0f1d-48e0-9c8a-7207edadc632',volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:27.470382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:27.490495 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75d1c957-a9c6-459b-b42a-b5809745a117 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1508 microversion: 2.1 time: 0.313287 Oct 15 20:55:27.505481 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 949/2057] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:27 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1508 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:27.583656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:27.584607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:27.591851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.592380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.593065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.593740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.594185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.594607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.600286 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cb8fd2f2-5a8a-4406-8c2e-11ed899c875a tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.868573 Oct 15 20:55:27.601475 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1109/2058] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 871 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:27.613579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de8c80e9-1aed-440e-be13-b55878e35db5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:27.614707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-de8c80e9-1aed-440e-be13-b55878e35db5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:27.728925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-de8c80e9-1aed-440e-be13-b55878e35db5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:27.743332 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de8c80e9-1aed-440e-be13-b55878e35db5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1053 microversion: 2.1 time: 0.132906 Oct 15 20:55:27.744576 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1110/2059] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:27 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1053 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:27.765130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:27.766293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:27.776439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.777313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.778087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.779353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:27.780816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:27.781581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:27.994460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:28.005619 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fda7aaf6-1b41-429f-8d20-fee8ce78525f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1488 microversion: 2.1 time: 0.246673 Oct 15 20:55:28.007921 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1111/2060] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:27 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1488 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:28.050203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:28.050203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:28.057615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.058274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.058714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.059428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.061118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.063421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.085202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:28.097653 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d9648e6-c97b-4c5b-b54e-bf73821c36f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.518437 Oct 15 20:55:28.098256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 950/2061] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:27 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:28.128549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:28.129217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:28.147834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.148289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.148740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.149497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.149961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.150395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.254161 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35aa9cc2-7663-42f1-9acd-a1d9f588a7f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7/action" status: 202 len: 0 microversion: 2.1 time: 0.220720 Oct 15 20:55:28.255140 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1112/2062] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:55:28 2022] POST /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7/action => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:55:28.309007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:28.309784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:28.327275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.327930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.328632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.329828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.330589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.331606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.384504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:28.392946 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac6e7646-6e1e-4f69-9adf-feec902bdf2d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.270236 Oct 15 20:55:28.393653 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 951/2063] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:28 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:28.518443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:28.519431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:28.525608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.526212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.526866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.527921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.528782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.529550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.561950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:28.570864 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-40b3c62c-5461-4afa-a3f8-aa10363d0eca tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1493 microversion: 2.1 time: 0.298062 Oct 15 20:55:28.572158 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1113/2064] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:28 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1493 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:28.770180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:28.770722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:28.779552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.780400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.784190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.785009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:28.785521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:28.786031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:28.799494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:28.812291 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe5c645c-11ee-44b9-aed4-26c00a3b9cc0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1508 microversion: 2.1 time: 0.298000 Oct 15 20:55:28.812867 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 952/2065] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:28 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1508 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:28.853848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:28.863398 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d1b4233b-6d94-4239-aedf-3dd0c939c5fc tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1151 microversion: 2.1 time: 0.096980 Oct 15 20:55:28.864682 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1114/2066] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:28 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1151 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:55:29.124173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:29.124778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:29.131306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:29.131898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:29.132612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:29.133409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:29.133936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:29.134435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:29.230610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-751787613", "imageRef": "", "flavorRef": "452", "networks": [{"uuid": "9c7544ec-ea93-44b4-996f-09fc127b83d3"}], "block_device_mapping_v2": [{"uuid": "8bfcf92f-12a4-4045-bc1e-1ad61a6281de", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:29.339751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] validate_networks() for [('9c7544ec-ea93-44b4-996f-09fc127b83d3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:29.559754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:29.577541 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f0418a17-53af-4714-9fcb-99ecba6983bb tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.457211 Oct 15 20:55:29.579673 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 953/2067] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:29 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:29.587402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:29.589970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:29.597576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:29.598069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:29.598505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:29.599216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:29.599829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:29.600417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.027044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:30.038141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75cfdfb7-d069-47d1-8cad-5967d6714257 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.455895 Oct 15 20:55:30.039339 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 954/2068] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:29 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:30.064328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:30.065004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:30.082127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.084932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.085420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.086185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.088323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.088771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.508928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:30.523212 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ebf0a360-bba3-46e3-8987-70b079ec6443 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 200 len: 1473 microversion: 2.1 time: 0.482440 Oct 15 20:55:30.524534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 955/2069] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:30 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 1473 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:30.529893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:30.530726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:30.537413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.544333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.544333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.544333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.544333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.545302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.675735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:30.703598 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25a2699f-a7a0-4994-9885-83b1850e45f8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 200 len: 1466 microversion: 2.1 time: 0.174510 Oct 15 20:55:30.705478 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 956/2070] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:30 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 1466 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:30.733207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:30.733976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:30.751126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.751607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.752030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.759613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.760734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.762141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.924766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:30.938771 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14918587-3285-428d-bda3-9072ce0369e3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1169 microversion: 2.1 time: 0.229728 Oct 15 20:55:30.939375 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 957/2071] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:30 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1169 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:30.947316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-565980850"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:30.956594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:30.968847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.969376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.969824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:30.976248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:30.976815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:30.977303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.252326 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7baac1e6-4053-44cf-b04b-8b6497ace19f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7/action" status: 202 len: 0 microversion: 2.1 time: 0.311651 Oct 15 20:55:31.253840 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 958/2072] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:55:30 2022] POST /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7/action => generated 0 bytes in 314 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:55:31.259833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:31.261380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:31.271757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.272034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.272542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.273661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.274039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.274480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.499843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:31.516620 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e38fa08-a4d7-4a4e-a92d-b2514269049b tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1318 microversion: 2.1 time: 0.260916 Oct 15 20:55:31.517995 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 959/2073] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:31 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1318 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:31.524682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ed5e0f64-ec71-419c-947c-a74f933341c6 req-158b253d-87ce-4c21-92bb-96ca4c8b10d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a", "name": "network-vif-deleted", "tag": "b8b5e35e-5f7f-41f0-b601-d0d07c25b03a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:31.535961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ed5e0f64-ec71-419c-947c-a74f933341c6 req-158b253d-87ce-4c21-92bb-96ca4c8b10d3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.536454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ed5e0f64-ec71-419c-947c-a74f933341c6 req-158b253d-87ce-4c21-92bb-96ca4c8b10d3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.537018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ed5e0f64-ec71-419c-947c-a74f933341c6 req-158b253d-87ce-4c21-92bb-96ca4c8b10d3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.549729 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ed5e0f64-ec71-419c-947c-a74f933341c6 req-158b253d-87ce-4c21-92bb-96ca4c8b10d3 service nova] Creating event network-vif-deleted:b8b5e35e-5f7f-41f0-b601-d0d07c25b03a for instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a on nch2-master-1 Oct 15 20:55:31.555780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ed5e0f64-ec71-419c-947c-a74f933341c6 req-158b253d-87ce-4c21-92bb-96ca4c8b10d3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035668 Oct 15 20:55:31.556740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 960/2074] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:31.563993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:31.566128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:31.573300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.574925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.575748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.577802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.578628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.579282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.826919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:31.836178 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98d49426-0a83-4316-aadf-886cfe66ec58 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.277877 Oct 15 20:55:31.838979 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 961/2075] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:31 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:31.844946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54e93338-c858-405c-92b0-197a7048ae96 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f1b01f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:31.908180 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54e93338-c858-405c-92b0-197a7048ae96 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.067908 Oct 15 20:55:31.909273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 962/2076] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:31 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:31.916766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8ce4dc9c-85d4-4419-890e-11e439b46326 req-2da20d15-e5cb-4333-bb58-3f6be42fdc72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4c39d08f-a040-4454-a0a5-2c699cd98138", "name": "network-vif-plugged", "status": "completed", "tag": "55dff384-41ed-4361-b4c9-5f0a77f3d5d8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:31.929512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8ce4dc9c-85d4-4419-890e-11e439b46326 req-2da20d15-e5cb-4333-bb58-3f6be42fdc72 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.930064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8ce4dc9c-85d4-4419-890e-11e439b46326 req-2da20d15-e5cb-4333-bb58-3f6be42fdc72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.930879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8ce4dc9c-85d4-4419-890e-11e439b46326 req-2da20d15-e5cb-4333-bb58-3f6be42fdc72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.945962 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8ce4dc9c-85d4-4419-890e-11e439b46326 req-2da20d15-e5cb-4333-bb58-3f6be42fdc72 service nova] Creating event network-vif-plugged:55dff384-41ed-4361-b4c9-5f0a77f3d5d8 for instance 4c39d08f-a040-4454-a0a5-2c699cd98138 on nch1-master-1 Oct 15 20:55:31.951721 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8ce4dc9c-85d4-4419-890e-11e439b46326 req-2da20d15-e5cb-4333-bb58-3f6be42fdc72 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040726 Oct 15 20:55:31.952352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 963/2077] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:31.959509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:31.960233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:31.968501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.969006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.969481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.970404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:31.971049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:31.971653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:31.992525 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a HTTP/1.1 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:55:31.995316 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 20:55:31.997009 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a'} Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:31.998887 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 20:55:32.001002 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:32.004854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:32.005999 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc020a46-a722-480e-a1f8-b882c6625975 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 404 len: 111 microversion: 2.1 time: 0.052291 Oct 15 20:55:32.007373 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 964/2078] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:31 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:55:32.021643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:32.022606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:32.031271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:32.031879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:32.032395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:32.033306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:32.033770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:32.034205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:32.588455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:32.596539 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddd2737f-03ca-4271-a729-9ea535dd080f tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1199 microversion: 2.1 time: 0.587856 Oct 15 20:55:32.597139 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 965/2079] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:32 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1199 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:32.602025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1879020803", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:32.657099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:33.377158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:33.377729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:33.378290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:33.378863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:33.379164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:33.391214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:33.391758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:33.392253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:33.393486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:33.394073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:33.394734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:33.445151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:33.456082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:33.477744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:33.480862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:33.537002 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78463f5e-e617-488d-8a14-db6a19ec7f2d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.938726 Oct 15 20:55:33.537957 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 966/2080] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 940 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:33.545793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:33.546675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:33.554101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:33.554871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:33.555787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:33.556871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:33.557534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:33.558137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:33.563988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:33.822106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:33.833986 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b10bca86-1b46-4bc8-a1b2-318f6e27b66a tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1507 microversion: 2.1 time: 0.292464 Oct 15 20:55:33.836771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 967/2081] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:33 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1507 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:33.850577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:33.854439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:33.870679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:33.871189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:33.871646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:33.872376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:33.872912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:33.877616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:34.474648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:34.486146 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9c901897-1263-4667-a22c-33af0489a22b tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.646400 Oct 15 20:55:34.486832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 968/2082] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:33 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:34.510236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba573559-af3f-4352-b101-d1d7ab9da7f5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f23e8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:34.584594 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba573559-af3f-4352-b101-d1d7ab9da7f5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.096371 Oct 15 20:55:34.595405 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 969/2083] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:34 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:34.602203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:34.603384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:34.611803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:34.612854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:34.613633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:34.614624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:34.615792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:34.617257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:34.735664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:34.745398 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ceb09a05-f8d9-435a-a115-dd25fceeb7c8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1267 microversion: 2.1 time: 0.147913 Oct 15 20:55:34.745963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 970/2084] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:34 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:34.753622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5cf4fe17-a3c4-4ea0-bf38-658dea09f79a req-47a09405-156b-46de-957a-b80a91bc9870 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "23abfd9b-7822-4ed5-82d0-e857368017de", "tag": "166c20c8-2a75-4402-bb95-0d1df37d8744"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:34.767233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5cf4fe17-a3c4-4ea0-bf38-658dea09f79a req-47a09405-156b-46de-957a-b80a91bc9870 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:34.767714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5cf4fe17-a3c4-4ea0-bf38-658dea09f79a req-47a09405-156b-46de-957a-b80a91bc9870 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:34.768145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5cf4fe17-a3c4-4ea0-bf38-658dea09f79a req-47a09405-156b-46de-957a-b80a91bc9870 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:34.784915 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5cf4fe17-a3c4-4ea0-bf38-658dea09f79a req-47a09405-156b-46de-957a-b80a91bc9870 service nova] Creating event network-changed:166c20c8-2a75-4402-bb95-0d1df37d8744 for instance 23abfd9b-7822-4ed5-82d0-e857368017de on nch1-master-1 Oct 15 20:55:34.792041 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5cf4fe17-a3c4-4ea0-bf38-658dea09f79a req-47a09405-156b-46de-957a-b80a91bc9870 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042443 Oct 15 20:55:34.792824 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 971/2085] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:34.799063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:34.799872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:34.809195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:34.810842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:34.811661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:34.812687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:34.813423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:34.813940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:35.064663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:35.079644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bacdea57-8715-48a3-9fc3-05f4b76947c6 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1199 microversion: 2.1 time: 0.285602 Oct 15 20:55:35.096772 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 972/2086] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:34 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1199 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:35.118037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-765d1c96-a2a8-4df7-926d-2882b17e3e68 req-6a0081f0-3bcb-46db-b27d-324ad089075d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e20e4c9-1939-425a-a7a8-d284327057bc", "name": "network-vif-plugged", "status": "completed", "tag": "5d6409b1-05df-4a6a-8356-24ea8f253baa"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:35.141539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-765d1c96-a2a8-4df7-926d-2882b17e3e68 req-6a0081f0-3bcb-46db-b27d-324ad089075d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:35.142037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-765d1c96-a2a8-4df7-926d-2882b17e3e68 req-6a0081f0-3bcb-46db-b27d-324ad089075d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:35.142469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-765d1c96-a2a8-4df7-926d-2882b17e3e68 req-6a0081f0-3bcb-46db-b27d-324ad089075d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:35.181679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-765d1c96-a2a8-4df7-926d-2882b17e3e68 req-6a0081f0-3bcb-46db-b27d-324ad089075d service nova] Creating event network-vif-plugged:5d6409b1-05df-4a6a-8356-24ea8f253baa for instance 8e20e4c9-1939-425a-a7a8-d284327057bc on nch2-master-1 Oct 15 20:55:35.187899 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-765d1c96-a2a8-4df7-926d-2882b17e3e68 req-6a0081f0-3bcb-46db-b27d-324ad089075d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090080 Oct 15 20:55:35.189253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 973/2087] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:35.521356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "9b319902-151f-49b6-bbd5-9bce65a305f1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:35.524906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:35.546650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:35.547634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:35.548635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:35.553195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:35.554044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:35.554723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:35.711911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] HTTP exception thrown: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Request method failure captured: Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138/os-volume_attachments/06fbf0ff-d496-4884-ab14-ab7e96e07464 HTTP/1.1 Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 74 Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx3uDC4kuG6vKusz52P-ssnduWNqHCC6WIn48pnxd38rHgktfUat4wsu8Bo38cplWf2yTAm5E3swWjnbSR46SDRJALjE-_4n82GKKuXX9iO-4jeEDXZw4qp6UXIgiaP4_XRs74itFn90juf3wZBJLyTxc2dmlMdXeRpz7wac8m_5HEk Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 971ec797f8d94d0989654cfa2d952a80 Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesAdminNegativeTest-477462580 Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:55:35.716175 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/971ec797f8d94d0989654cfa2d952a80"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesAdminNegativeTest-477462580 Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 971ec797f8d94d0989654cfa2d952a80 Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-477462580 Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesAdminNegativeTest-477462580-project-admin Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 9749157277104da9bc3ffeef563adeb4 Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesAdminNegativeTest-477462580-project-admin Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: {"volumeAttachment": {"volumeId": "9b319902-151f-49b6-bbd5-9bce65a305f1"}} Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:55:35.718446 ncdv-master-1 devstack@n-api.service[138688]: exception: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '4c39d08f-a040-4454-a0a5-2c699cd98138', 'id': '06fbf0ff-d496-4884-ab14-ab7e96e07464', 'body': {'volumeAttachment': {'volumeId': '9b319902-151f-49b6-bbd5-9bce65a305f1'}}} Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 20:55:35.719530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. (HTTP 404) (Request-ID: req-5af2850a-f94a-4b75-b11b-21ef26f8eb79) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 430, in _update_volume_swap Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi old_volume = self.volume_api.get(context, old_volume_id) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 20:55:35.721205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:35.722778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 432, in _update_volume_swap Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. Oct 15 20:55:35.724374 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:35.725964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] Returning 404 to user: Volume 06fbf0ff-d496-4884-ab14-ab7e96e07464 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:35.725964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-05ecb2aa-922d-486e-9b12-486da54d9954 tempest-VolumesAdminNegativeTest-477462580 tempest-VolumesAdminNegativeTest-477462580-project-admin] 10.222.0.16 "PUT /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138/os-volume_attachments/06fbf0ff-d496-4884-ab14-ab7e96e07464" status: 404 len: 109 microversion: 2.1 time: 0.208817 Oct 15 20:55:35.728344 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 974/2088] 10.222.0.16 () {60 vars in 1433 bytes} [Sat Oct 15 20:55:35 2022] PUT /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138/os-volume_attachments/06fbf0ff-d496-4884-ab14-ab7e96e07464 => generated 109 bytes in 215 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 20:55:35.742555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:35.743306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:35.759568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:35.762786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:35.763957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:35.765656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:35.767640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:35.769543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:36.030749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:36.040325 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-461fed80-7492-4436-8b48-1de29c7ff733 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1403 microversion: 2.1 time: 0.311915 Oct 15 20:55:36.044172 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 975/2089] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:35 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1403 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:36.066007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd7684ce-512e-4ba0-a9ef-474d00f0fa70 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe758344670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:36.264163 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd7684ce-512e-4ba0-a9ef-474d00f0fa70 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.218574 Oct 15 20:55:36.265761 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 976/2090] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:36 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:36.278560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:36.282684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:36.301639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:36.302741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:36.304963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:36.309592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:36.311299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:36.311994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:36.961515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:36.971549 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae220dc8-5569-4ee7-a46a-35103c96550a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1273 microversion: 2.1 time: 0.704489 Oct 15 20:55:36.976894 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 977/2091] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:36 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1273 bytes in 710 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:37.005542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:37.006513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:37.033646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:37.035386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:37.036964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:37.037747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:37.038566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:37.038686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:37.345399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:37.575469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:37.588907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbcf620f-5ae4-4168-b00c-26cb3cf7177c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1199 microversion: 2.1 time: 0.610559 Oct 15 20:55:37.589561 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 978/2092] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:36 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1199 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:37.610538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:37.614476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:37.635020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:37.635833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:37.637084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:37.637997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:37.641690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:37.642354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:37.741761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:37.912110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:37.926389 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9ada3c6-5fff-43b1-9f47-5fdcd740de0a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1590 microversion: 2.1 time: 0.319744 Oct 15 20:55:37.927567 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 979/2093] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:37 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1590 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:37.945965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:37.946844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:37.955197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:37.963809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:37.964469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:37.965010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:37.965575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:37.965927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:38.043649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:55:38.251891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-728e32e2-7171-44ba-af4e-6e3fce52c373 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 204 len: 0 microversion: 2.1 time: 0.322765 Oct 15 20:55:38.261246 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 980/2094] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:55:37 2022] DELETE /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:55:38.267436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82271925-84e5-4274-bea8-82b2eacfee5a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7583bb310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:38.360252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82271925-84e5-4274-bea8-82b2eacfee5a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.096184 Oct 15 20:55:38.368599 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 981/2095] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:38 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:38.380494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:38.388396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:38.415944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:38.419182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:38.421808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:38.428050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:38.428743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:38.429827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:38.774084 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8318caa-b275-40c5-bfec-42d07d21ff47 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.409814 Oct 15 20:55:38.775046 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 982/2096] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:55:38 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc/os-interface => generated 284 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:38.787354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:38.788275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:38.804090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:38.804742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:38.806078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:38.806942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:38.808697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:38.809280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:38.858802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:39.172646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:39.182091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5607563-c578-4806-8e17-c03b0772d730 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1315 microversion: 2.1 time: 0.405979 Oct 15 20:55:39.187741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 983/2097] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:38 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1315 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:39.203116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:39.204214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:39.219722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:39.220217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:39.220656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:39.225738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:39.226259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:39.226706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:39.244949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=80,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:39.933491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:39.956890 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4fe2ca0d-3462-4450-8540-0a7cae8f97d3 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1513 microversion: 2.1 time: 0.760076 Oct 15 20:55:39.957579 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 984/2098] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:39 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1513 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:39.981433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:39.987579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:39.998436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:40.025228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:40.027871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:40.028451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:40.030168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:40.033750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:40.036259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:40.496772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:40.509573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9677c41c-ff3a-4eac-8b69-15bac27541de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1199 microversion: 2.1 time: 0.550640 Oct 15 20:55:40.514621 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 985/2099] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:39 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1199 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:40.539450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2c2e4d7-92fd-4ac7-acab-5325da00e7bb tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe758344280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:40.628222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2c2e4d7-92fd-4ac7-acab-5325da00e7bb tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.112041 Oct 15 20:55:40.629085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 986/2100] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:40 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:40.642295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c18806ab-26ab-485a-8e41-09cbcb763e35 req-3c6dc7d7-8289-4c90-a68a-216be2fdcbb1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dafc5b2-733a-433a-bbcc-5136827624c7", "name": "network-vif-plugged", "status": "completed", "tag": "cd9552df-0245-4ae6-8b4f-63fd9d25ec69"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:40.662859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c18806ab-26ab-485a-8e41-09cbcb763e35 req-3c6dc7d7-8289-4c90-a68a-216be2fdcbb1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:40.663345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c18806ab-26ab-485a-8e41-09cbcb763e35 req-3c6dc7d7-8289-4c90-a68a-216be2fdcbb1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:40.663785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c18806ab-26ab-485a-8e41-09cbcb763e35 req-3c6dc7d7-8289-4c90-a68a-216be2fdcbb1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:40.693392 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c18806ab-26ab-485a-8e41-09cbcb763e35 req-3c6dc7d7-8289-4c90-a68a-216be2fdcbb1 service nova] Creating event network-vif-plugged:cd9552df-0245-4ae6-8b4f-63fd9d25ec69 for instance 7dafc5b2-733a-433a-bbcc-5136827624c7 on nch1-master-1 Oct 15 20:55:40.701805 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c18806ab-26ab-485a-8e41-09cbcb763e35 req-3c6dc7d7-8289-4c90-a68a-216be2fdcbb1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070709 Oct 15 20:55:40.702422 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 987/2101] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:41.033281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-465396647", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:41.136867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] validate_networks() for [('b70a0bf4-f480-42f6-93c5-ff836a91e53c', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:42.867292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:42.867941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:42.868192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:42.868876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:42.869496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:42.886645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:42.887361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:42.888536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:42.892613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:42.894103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:42.894734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:42.939754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Getting quotas for project 678b3cd3dda745fda1201db024f2d746. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:42.945849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Getting quotas for user 5f0724ab706d407fb067d4ed444e10a4 and project 678b3cd3dda745fda1201db024f2d746. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:42.960481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:42.963513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:43.044264 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc71f1aa-f662-4696-add9-69415f8b2f17 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.340576 Oct 15 20:55:43.049327 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 988/2102] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 2346 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:43.055269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:43.055269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:43.062473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.063361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.063481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.064142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.064712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.065252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.089809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:43.408263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:43.422196 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-257e8834-4b1e-44dc-8406-8effcf5e2655 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1315 microversion: 2.1 time: 0.371525 Oct 15 20:55:43.423193 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 989/2103] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:43 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1315 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:43.437699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:43.438775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:43.447273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.448003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.452330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.453421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.454075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.454680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.493590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.494311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.494922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.496235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lazy-loading 'fault' on Instance uuid 4c39d08f-a040-4454-a0a5-2c699cd98138 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:55:43.551190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:43.576693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e050392-89d9-4977-8443-49bdcc073c85 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 200 len: 1308 microversion: 2.1 time: 0.145733 Oct 15 20:55:43.587941 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 990/2104] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:43 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 1308 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:43.620249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c9709a61-ff82-4890-adae-87a56c5a009b req-c4f04c1b-6ca2-455d-8981-6b7413f857c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8077ee04-cf77-4ad7-aef9-1d0244cfa551", "tag": "8ca72979-3681-49ec-9dad-1321aad91ce3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:43.650083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c9709a61-ff82-4890-adae-87a56c5a009b req-c4f04c1b-6ca2-455d-8981-6b7413f857c8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.651112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c9709a61-ff82-4890-adae-87a56c5a009b req-c4f04c1b-6ca2-455d-8981-6b7413f857c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.653015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c9709a61-ff82-4890-adae-87a56c5a009b req-c4f04c1b-6ca2-455d-8981-6b7413f857c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.689404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c9709a61-ff82-4890-adae-87a56c5a009b req-c4f04c1b-6ca2-455d-8981-6b7413f857c8 service nova] Creating event network-changed:8ca72979-3681-49ec-9dad-1321aad91ce3 for instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 on nch2-master-1 Oct 15 20:55:43.705132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c9709a61-ff82-4890-adae-87a56c5a009b req-c4f04c1b-6ca2-455d-8981-6b7413f857c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102435 Oct 15 20:55:43.706719 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 991/2105] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:43.721252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:43.722445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:43.738548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.742192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.744806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:43.747073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:43.750044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:43.756320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.044513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:44.071739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f723db39-2585-4256-8ab9-c70ed12951c5 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1590 microversion: 2.1 time: 0.362590 Oct 15 20:55:44.074244 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 992/2106] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:43 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1590 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:44.087702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0ea9d0c4-ac03-4836-91bb-3cd7f4f8dd1b req-8b90956f-4d32-46f3-a684-51a786ca5a24 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e20e4c9-1939-425a-a7a8-d284327057bc", "tag": "5d6409b1-05df-4a6a-8356-24ea8f253baa"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:44.101720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ea9d0c4-ac03-4836-91bb-3cd7f4f8dd1b req-8b90956f-4d32-46f3-a684-51a786ca5a24 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.102243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ea9d0c4-ac03-4836-91bb-3cd7f4f8dd1b req-8b90956f-4d32-46f3-a684-51a786ca5a24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.102878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ea9d0c4-ac03-4836-91bb-3cd7f4f8dd1b req-8b90956f-4d32-46f3-a684-51a786ca5a24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.123069 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0ea9d0c4-ac03-4836-91bb-3cd7f4f8dd1b req-8b90956f-4d32-46f3-a684-51a786ca5a24 service nova] Creating event network-changed:5d6409b1-05df-4a6a-8356-24ea8f253baa for instance 8e20e4c9-1939-425a-a7a8-d284327057bc on nch2-master-1 Oct 15 20:55:44.131906 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0ea9d0c4-ac03-4836-91bb-3cd7f4f8dd1b req-8b90956f-4d32-46f3-a684-51a786ca5a24 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054579 Oct 15 20:55:44.133103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 993/2107] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:44.142967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:44.144181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:44.150972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.152512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.153333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.154977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.155498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.155939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.502164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:44.515324 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7a08158-a8a9-4ddf-b91b-e919541d7276 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1199 microversion: 2.1 time: 0.376057 Oct 15 20:55:44.516493 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 994/2108] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:44 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1199 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:44.526842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d1799984-2492-4910-b73a-eb7c446382a6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7584d1790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:44.600282 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d1799984-2492-4910-b73a-eb7c446382a6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.080486 Oct 15 20:55:44.601145 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 995/2109] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:44 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:44.608971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1531f4bb-9387-4847-9185-64ae30963edb req-1bef4436-c89a-498a-b7c7-866404a1aea2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23abfd9b-7822-4ed5-82d0-e857368017de", "name": "network-vif-plugged", "status": "completed", "tag": "166c20c8-2a75-4402-bb95-0d1df37d8744"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:44.622272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1531f4bb-9387-4847-9185-64ae30963edb req-1bef4436-c89a-498a-b7c7-866404a1aea2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.624687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1531f4bb-9387-4847-9185-64ae30963edb req-1bef4436-c89a-498a-b7c7-866404a1aea2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.625465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1531f4bb-9387-4847-9185-64ae30963edb req-1bef4436-c89a-498a-b7c7-866404a1aea2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.648802 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1531f4bb-9387-4847-9185-64ae30963edb req-1bef4436-c89a-498a-b7c7-866404a1aea2 service nova] Creating event network-vif-plugged:166c20c8-2a75-4402-bb95-0d1df37d8744 for instance 23abfd9b-7822-4ed5-82d0-e857368017de on nch1-master-1 Oct 15 20:55:44.656740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1531f4bb-9387-4847-9185-64ae30963edb req-1bef4436-c89a-498a-b7c7-866404a1aea2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053757 Oct 15 20:55:44.658142 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 996/2110] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:44.663591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:44.665774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:44.675308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.676029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.676691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.677843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.678507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.679163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.840577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:44.865261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4466e471-8894-4ea9-b631-47eea3a8c7a4 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1275 microversion: 2.1 time: 0.205707 Oct 15 20:55:44.872222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 997/2111] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:44 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1275 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:44.892896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:44.919283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:44.953561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.954125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.955977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:44.958101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:44.958829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:44.959499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:45.456174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:45.467059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c56f062-73be-4bc0-af68-0a09cff233cd tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1315 microversion: 2.1 time: 0.592627 Oct 15 20:55:45.473351 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 998/2112] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:44 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1315 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:45.486132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:45.487166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 4c39d08f-a040-4454-a0a5-2c699cd98138] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:45.505406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:45.507244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:45.508347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:45.510164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:45.512173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:45.524508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:45.566982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] HTTP exception thrown: Instance 4c39d08f-a040-4454-a0a5-2c699cd98138 could not be found. Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Request method failure captured: Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 HTTP/1.1 Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx3ez4nAqHKAGc2dEhZhe7d3oAFApICfjjW8f0Fa0YuHxk7aHAhoxl4kzLr_Y-kpPsUwFGDnto6Zhc9XcffbarIBNuebOrLg2Am7_daRoNFU80KoSBzs6sCLpTlnRTcwOSn5mQoapiLMPOhPhNz_sBZ5VxVAp2tD5SMuPIHohpxasqo Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e08b85b4f6cf47a88fe24ead8e6d45e9 Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesAdminNegativeTest-797696769 Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:55:45.569710 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesAdminNegativeTest-797696769 Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e08b85b4f6cf47a88fe24ead8e6d45e9 Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-797696769 Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesAdminNegativeTest-797696769-project Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: ec8dcd7615cc43b3866034d5af34a9a1 Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesAdminNegativeTest-797696769-project Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 4c39d08f-a040-4454-a0a5-2c699cd98138 could not be found. Oct 15 20:55:45.571591 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4c39d08f-a040-4454-a0a5-2c699cd98138'} Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4c39d08f-a040-4454-a0a5-2c699cd98138 could not be found. Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:45.572630 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4c39d08f-a040-4454-a0a5-2c699cd98138 could not be found. Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4c39d08f-a040-4454-a0a5-2c699cd98138 could not be found. Oct 15 20:55:45.574399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:45.580980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Returning 404 to user: Instance 4c39d08f-a040-4454-a0a5-2c699cd98138 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:45.582098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7aea000c-4c7c-4af1-9bcf-d23a9d1b09bd tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138" status: 404 len: 111 microversion: 2.1 time: 0.107563 Oct 15 20:55:45.584928 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 999/2113] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:45 2022] GET /compute/v2.1/servers/4c39d08f-a040-4454-a0a5-2c699cd98138 => generated 111 bytes in 111 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:55:45.596994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:45.598722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:45.620557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:45.621365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:45.622047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:45.625923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:45.626963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:45.627574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:46.593370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:46.608323 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2817b49b-cffa-4b3f-8351-649464262113 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1199 microversion: 2.1 time: 1.017453 Oct 15 20:55:46.608962 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1000/2114] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:45 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1199 bytes in 1023 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:46.627904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ff9f280-0e57-4a76-a1f1-9691f34ed048 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe758344160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:46.788727 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ff9f280-0e57-4a76-a1f1-9691f34ed048 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.178626 Oct 15 20:55:46.789937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1001/2115] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:46 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:46.816270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b1e75bb0-41c7-4392-9ae8-17e4ab5e47a0 req-249d195b-4240-4f40-b862-3c5d7d93a40c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4c39d08f-a040-4454-a0a5-2c699cd98138", "name": "network-vif-deleted", "tag": "55dff384-41ed-4361-b4c9-5f0a77f3d5d8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:46.850901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b1e75bb0-41c7-4392-9ae8-17e4ab5e47a0 req-249d195b-4240-4f40-b862-3c5d7d93a40c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:46.861674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b1e75bb0-41c7-4392-9ae8-17e4ab5e47a0 req-249d195b-4240-4f40-b862-3c5d7d93a40c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:46.862642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b1e75bb0-41c7-4392-9ae8-17e4ab5e47a0 req-249d195b-4240-4f40-b862-3c5d7d93a40c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:46.929974 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b1e75bb0-41c7-4392-9ae8-17e4ab5e47a0 req-249d195b-4240-4f40-b862-3c5d7d93a40c service nova] Creating event network-vif-deleted:55dff384-41ed-4361-b4c9-5f0a77f3d5d8 for instance 4c39d08f-a040-4454-a0a5-2c699cd98138 on nch1-master-1 Oct 15 20:55:46.942377 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b1e75bb0-41c7-4392-9ae8-17e4ab5e47a0 req-249d195b-4240-4f40-b862-3c5d7d93a40c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.146076 Oct 15 20:55:46.943458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1002/2116] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:46.960078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:46.960806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:47.007648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:47.010417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:47.015591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:47.018967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:47.019532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:47.020997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:47.533943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:47.544260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a436b6b1-7a67-4943-9479-91d75f502a0b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1323 microversion: 2.1 time: 0.599308 Oct 15 20:55:47.545443 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1003/2117] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:46 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1323 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:47.559420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:47.564732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:47.578169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:47.578663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:47.580560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:47.581542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:47.584232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:47.585057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:47.899099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:47.911042 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-503f871f-cc01-4f59-b56f-f064c2b3a209 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1315 microversion: 2.1 time: 0.363208 Oct 15 20:55:47.916081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1004/2118] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:47 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1315 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:47.931718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:47.934703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:47.954580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:47.955398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:47.955959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:47.958181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:47.958770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:47.962521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:48.017254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:55:48.188086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1aef5498-9585-4210-914e-3a2c9506aad5 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "DELETE /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 204 len: 0 microversion: 2.1 time: 0.270364 Oct 15 20:55:48.189686 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1005/2119] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:55:47 2022] DELETE /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 0 bytes in 273 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:55:48.198070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "60dd8b4e-687e-4997-8daf-28ab29fba35c"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:48.218996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:48.235158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:48.240409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:48.240995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:48.241923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:48.242519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:48.243026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:48.615401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53a60d23-5612-4121-a5fa-7e3670989145 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "POST /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.421289 Oct 15 20:55:48.616821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1006/2120] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 20:55:48 2022] POST /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments => generated 194 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:48.646918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:48.651503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:48.660941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:48.662002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:48.662644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:48.663590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:48.664224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:48.665059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:48.884949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:48.894792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-046fda53-372a-412d-a489-75949ba8fbd8 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1383 microversion: 2.1 time: 0.262863 Oct 15 20:55:48.898020 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1007/2121] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:48 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1383 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:48.913775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-107c0881-d9ff-4b4b-8dd5-b54bbd177044 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f1255e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:48.981622 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-107c0881-d9ff-4b4b-8dd5-b54bbd177044 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.082379 Oct 15 20:55:48.985451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1008/2122] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:48 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:48.990511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d2b18d1c-a8f4-42de-a50d-e08aca13c339 req-f3f59bfc-5bd2-42b2-bfed-47426d051f30 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7", "tag": "0b471f75-8cf1-4971-a96a-2b14e208ce42"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:49.001929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2b18d1c-a8f4-42de-a50d-e08aca13c339 req-f3f59bfc-5bd2-42b2-bfed-47426d051f30 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.002828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2b18d1c-a8f4-42de-a50d-e08aca13c339 req-f3f59bfc-5bd2-42b2-bfed-47426d051f30 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.003460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2b18d1c-a8f4-42de-a50d-e08aca13c339 req-f3f59bfc-5bd2-42b2-bfed-47426d051f30 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.007932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:49.016577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d2b18d1c-a8f4-42de-a50d-e08aca13c339 req-f3f59bfc-5bd2-42b2-bfed-47426d051f30 service nova] Creating event network-changed:0b471f75-8cf1-4971-a96a-2b14e208ce42 for instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 on nch1-master-1 Oct 15 20:55:49.022645 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d2b18d1c-a8f4-42de-a50d-e08aca13c339 req-f3f59bfc-5bd2-42b2-bfed-47426d051f30 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035649 Oct 15 20:55:49.023417 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1009/2123] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:49.028757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:49.030036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:49.037537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.038032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.038486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.039181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.039693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.040133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.219424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:49.227516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-881b2e4a-a702-47d8-9fc7-53c66132d795 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1680 microversion: 2.1 time: 0.202977 Oct 15 20:55:49.228494 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1010/2124] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:49 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1680 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:49.235102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:49.235872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:49.242420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.243035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.243595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.244424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.245104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.245650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.425751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:49.439692 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-269566d2-433b-4fae-90f2-869b06f9bfd3 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1323 microversion: 2.1 time: 0.208924 Oct 15 20:55:49.461936 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1011/2125] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:49 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1323 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:49.615839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:49.616851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:49.624198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.625166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.625850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.626822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.627520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.628465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.914232 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-313a497c-4313-43c8-be5d-24b81cbb7858 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1534 microversion: 2.1 time: 0.450068 Oct 15 20:55:49.930801 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1012/2126] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:49 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1534 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:49.940338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:49.941595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:49.947626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.948131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.948628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:49.949472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:49.951329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:49.951820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:50.309318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:50.319723 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f8c1ac4-0ee2-4cae-8617-4df0212f81ba tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1315 microversion: 2.1 time: 0.387567 Oct 15 20:55:50.320559 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1013/2127] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:49 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1315 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:50.326314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:50.327197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:50.333269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:50.333976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:50.343514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:50.345912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:50.350141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:50.350811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:50.631465 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb413fbc-4d87-48ac-8de3-2650e22cfa47 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1534 microversion: 2.1 time: 0.309942 Oct 15 20:55:50.632478 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1014/2128] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:50 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1534 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:50.656837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c92135f8-ed85-48f7-840c-e2d9bf93b53c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe758406c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:50.754357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c92135f8-ed85-48f7-840c-e2d9bf93b53c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.107929 Oct 15 20:55:50.764068 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1015/2129] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:50 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:50.769532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-89fdd472-e52c-4798-b615-779b46f5e2db req-9d583b29-e298-44fc-a44e-f12240286d3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8077ee04-cf77-4ad7-aef9-1d0244cfa551", "name": "network-vif-plugged", "status": "completed", "tag": "8ca72979-3681-49ec-9dad-1321aad91ce3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:50.790957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-89fdd472-e52c-4798-b615-779b46f5e2db req-9d583b29-e298-44fc-a44e-f12240286d3e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:50.792001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-89fdd472-e52c-4798-b615-779b46f5e2db req-9d583b29-e298-44fc-a44e-f12240286d3e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:50.792650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-89fdd472-e52c-4798-b615-779b46f5e2db req-9d583b29-e298-44fc-a44e-f12240286d3e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:50.811522 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-89fdd472-e52c-4798-b615-779b46f5e2db req-9d583b29-e298-44fc-a44e-f12240286d3e service nova] Creating event network-vif-plugged:8ca72979-3681-49ec-9dad-1321aad91ce3 for instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 on nch2-master-1 Oct 15 20:55:50.818629 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-89fdd472-e52c-4798-b615-779b46f5e2db req-9d583b29-e298-44fc-a44e-f12240286d3e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052878 Oct 15 20:55:50.819764 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1016/2130] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:50.825141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:50.826478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:50.833943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:50.834663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:50.835296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:50.836006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:50.836468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:50.836896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:50.936134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec8dcd7615cc43b3866034d5af34a9a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e08b85b4f6cf47a88fe24ead8e6d45e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:50.969703 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16d072d3-5f34-4690-9f5b-868fec55b4f1 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 200 len: 1613 microversion: 2.1 time: 0.148440 Oct 15 20:55:50.976694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1017/2131] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:50 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 1613 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:50.988194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:51.001089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:51.008283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:51.009085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:51.009427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:51.010161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:51.014854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:51.015579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:51.708715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:51.718528 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-89be67a3-a53f-4e0f-984a-0a1045d441ad tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1323 microversion: 2.1 time: 0.740997 Oct 15 20:55:51.725465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1018/2132] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:50 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1323 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:51.754300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:51.774728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:51.775698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:51.777258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:51.780259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:51.780942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:51.781661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:51.900447 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd11751c-c15b-4441-97b7-a187faa98699 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.169325 Oct 15 20:55:51.913291 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1019/2133] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:55:51 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:51.933665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:51.934748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:51.941760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:51.942391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:51.944866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:51.945754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:51.948216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:51.948767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:52.302472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:52.330911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1407a3d0-89fd-49a5-99e8-e704b2e6cf39 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1315 microversion: 2.1 time: 0.416086 Oct 15 20:55:52.342887 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1020/2134] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:51 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1315 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:52.351441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-116f7327-1e40-4e5f-ac04-17891a4be721 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe75845e3a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:52.447396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-116f7327-1e40-4e5f-ac04-17891a4be721 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1413 microversion: 2.1 time: 0.101576 Oct 15 20:55:52.467430 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1021/2135] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:52 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1413 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:52.473445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": false, "disk_over_commit": false}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:52.478986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:52.492689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:52.493093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:52.493515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:52.494230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:52.494685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:52.495111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:52.522826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lazy-loading 'flavor' on Instance uuid 23abfd9b-7822-4ed5-82d0-e857368017de {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:55:52.535974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Going to try to live migrate instance to nch2-master-1 {{(pid=138688) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:55:52.954776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:54.488771 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdddefec-882c-4fe4-ab07-3576df731d51 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de/action" status: 202 len: 0 microversion: 2.1 time: 2.019042 Oct 15 20:55:54.489799 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1022/2136] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:55:52 2022] POST /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de/action => generated 0 bytes in 2022 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:55:54.494495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bb0a7bd3-39b3-4ea3-a2bd-422a4df503af req-38a3eb45-f62f-44c1-be9d-e5b5a74303c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "062c4651-90be-430a-a69b-372fac258ebb", "name": "network-vif-deleted", "tag": "e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:54.508681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bb0a7bd3-39b3-4ea3-a2bd-422a4df503af req-38a3eb45-f62f-44c1-be9d-e5b5a74303c8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.509466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bb0a7bd3-39b3-4ea3-a2bd-422a4df503af req-38a3eb45-f62f-44c1-be9d-e5b5a74303c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.510088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bb0a7bd3-39b3-4ea3-a2bd-422a4df503af req-38a3eb45-f62f-44c1-be9d-e5b5a74303c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.523262 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-bb0a7bd3-39b3-4ea3-a2bd-422a4df503af req-38a3eb45-f62f-44c1-be9d-e5b5a74303c8 service nova] Creating event network-vif-deleted:e7c2516c-e8fe-46bd-92fb-5c9e0270fcd8 for instance 062c4651-90be-430a-a69b-372fac258ebb on nch1-master-1 Oct 15 20:55:54.528857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bb0a7bd3-39b3-4ea3-a2bd-422a4df503af req-38a3eb45-f62f-44c1-be9d-e5b5a74303c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038032 Oct 15 20:55:54.529964 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1023/2137] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:54.541417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:54.541767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] [instance: 062c4651-90be-430a-a69b-372fac258ebb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:54.549922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.550262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.550849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.551509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.551862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.552292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.570051 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] HTTP exception thrown: Instance 062c4651-90be-430a-a69b-372fac258ebb could not be found. Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Request method failure captured: Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb HTTP/1.1 Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx3ez4nAqHKAGc2dEhZhe7d3oAFApICfjjW8f0Fa0YuHxk7aHAhoxl4kzLr_Y-kpPsUwFGDnto6Zhc9XcffbarIBNuebOrLg2Am7_daRoNFU80KoSBzs6sCLpTlnRTcwOSn5mQoapiLMPOhPhNz_sBZ5VxVAp2tD5SMuPIHohpxasqo Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e08b85b4f6cf47a88fe24ead8e6d45e9 Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesAdminNegativeTest-797696769 Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:55:54.571669 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e08b85b4f6cf47a88fe24ead8e6d45e9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesAdminNegativeTest-797696769 Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e08b85b4f6cf47a88fe24ead8e6d45e9 Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-797696769 Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesAdminNegativeTest-797696769-project Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: ec8dcd7615cc43b3866034d5af34a9a1 Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesAdminNegativeTest-797696769-project Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 062c4651-90be-430a-a69b-372fac258ebb could not be found. Oct 15 20:55:54.573539 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '062c4651-90be-430a-a69b-372fac258ebb'} Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 062c4651-90be-430a-a69b-372fac258ebb could not be found. Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:54.574427 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 062c4651-90be-430a-a69b-372fac258ebb could not be found. Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 062c4651-90be-430a-a69b-372fac258ebb could not be found. Oct 15 20:55:54.581280 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:54.584765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Returning 404 to user: Instance 062c4651-90be-430a-a69b-372fac258ebb could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:54.584765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce218ec3-82e8-482d-9b52-3f3520300a78 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb" status: 404 len: 111 microversion: 2.1 time: 0.041825 Oct 15 20:55:54.584765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1024/2138] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:54 2022] GET /compute/v2.1/servers/062c4651-90be-430a-a69b-372fac258ebb => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:55:54.584765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:54.584765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:54.589613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.590108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.590553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.592086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.592682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.593135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.636725 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bbfc29c4-dd16-491d-9252-9b7ce22cc008 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061755 Oct 15 20:55:54.644086 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1025/2139] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:55:54 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:54.650854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:54.651473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:54.658841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.659581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.660201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.660903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.661416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.662033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.915122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:54.927767 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bf29fe5a-021f-4f4d-9ab8-0bfc6052ce01 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1323 microversion: 2.1 time: 0.282144 Oct 15 20:55:54.928354 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1026/2140] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:54 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1323 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:54.947796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:54.948480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:54.955839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.956375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.956731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:54.957666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:54.958132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:54.958686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:55.203955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:55.211819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a902d8f-b0e4-456b-88d5-bf955e12f7ff tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1502 microversion: 2.1 time: 0.282364 Oct 15 20:55:55.212393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1027/2141] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:54 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1502 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:55.217395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e072658-2fde-4c81-9041-1535b2807566 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7584dbee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:55.267419 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e072658-2fde-4c81-9041-1535b2807566 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1422 microversion: 2.1 time: 0.053972 Oct 15 20:55:55.268075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1028/2142] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:55 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1422 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:55.272954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:55.273607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:55.280273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:55.282092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:55.282843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:55.283482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:55.284502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:55.285344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:55.286141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:55.473844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:55.483846 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e2772bc-1048-4699-891e-773bf8f31d5e tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.214666 Oct 15 20:55:55.485197 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1029/2143] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:55 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:55.490291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1421c5f4-e453-4c8a-a554-dfefccda8053 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] Calling method '.version_select at 0x7fe7584dbc10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:55.518850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1421c5f4-e453-4c8a-a554-dfefccda8053 tempest-VolumesAdminNegativeTest-797696769 tempest-VolumesAdminNegativeTest-797696769-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1573082579" status: 202 len: 0 microversion: 2.1 time: 0.032012 Oct 15 20:55:55.520151 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1030/2144] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:55:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1573082579 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:55:55.536902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:55.537596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:55.552196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:55.552695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:55.553176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:55.553908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:55.554379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:55.554833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:55.594978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:55.835249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lazy-loading 'services' on Instance uuid 82e74d13-2be4-47d0-93fa-fb24eb81c212 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:55:55.930471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2f3e950-7092-44f4-9a2e-8c3a50dbd2e3 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments/60dd8b4e-687e-4997-8daf-28ab29fba35c" status: 202 len: 0 microversion: 2.1 time: 0.408919 Oct 15 20:55:55.931623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1031/2145] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 20:55:55 2022] DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments/60dd8b4e-687e-4997-8daf-28ab29fba35c => generated 0 bytes in 411 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:55:55.948434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8aa6024-0358-4939-9975-b14c63fb5ed2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7584db280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:56.032538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8aa6024-0358-4939-9975-b14c63fb5ed2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 200 len: 1422 microversion: 2.1 time: 0.096223 Oct 15 20:55:56.044496 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1032/2146] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:55:55 2022] GET /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 1422 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:56.051356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c3264f0c-1466-4b54-af00-651e9d384222 req-310cac86-d094-4e76-b7ea-dc0116902b8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7", "name": "network-vif-plugged", "status": "completed", "tag": "0b471f75-8cf1-4971-a96a-2b14e208ce42"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:56.068336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c3264f0c-1466-4b54-af00-651e9d384222 req-310cac86-d094-4e76-b7ea-dc0116902b8a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:56.068841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c3264f0c-1466-4b54-af00-651e9d384222 req-310cac86-d094-4e76-b7ea-dc0116902b8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:56.069347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c3264f0c-1466-4b54-af00-651e9d384222 req-310cac86-d094-4e76-b7ea-dc0116902b8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:56.094734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c3264f0c-1466-4b54-af00-651e9d384222 req-310cac86-d094-4e76-b7ea-dc0116902b8a service nova] Creating event network-vif-plugged:0b471f75-8cf1-4971-a96a-2b14e208ce42 for instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 on nch1-master-1 Oct 15 20:55:56.102334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c3264f0c-1466-4b54-af00-651e9d384222 req-310cac86-d094-4e76-b7ea-dc0116902b8a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056333 Oct 15 20:55:56.103227 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1033/2147] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:55:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:56.109227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:56.110027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:56.117974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:56.118566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:56.119096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:56.119879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:56.120402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:56.120899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:56.270957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:56.279924 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e022d2c9-2108-412e-b990-e2766f4a109f tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1323 microversion: 2.1 time: 0.174983 Oct 15 20:55:56.293921 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1034/2148] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:56 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1323 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:56.302785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-971be6c4-9cde-47e6-927f-11abb8ef4fc0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7584db1f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:56.829221 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-971be6c4-9cde-47e6-927f-11abb8ef4fc0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 204 len: 0 microversion: 2.1 time: 0.532626 Oct 15 20:55:56.831331 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1035/2149] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:56 2022] DELETE /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:55:56.838728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:56.839891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:56.850344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:56.851178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:56.852223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:56.853153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:56.853778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:56.854753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:56.999844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:57.007998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e4258ca-e65a-43df-8900-411e1cec6b89 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.174696 Oct 15 20:55:57.008671 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1036/2150] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:56 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:57.016542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5e1d94af-ddca-4df5-84e4-10b9288757ed"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:57.019244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:57.025387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.025981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.034130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.034130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.034130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.034130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.320400 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-108e986e-6378-4a51-b71f-3fe0f6b7a160 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.310377 Oct 15 20:55:57.321293 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1037/2151] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 20:55:57 2022] POST /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551/os-volume_attachments => generated 194 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:55:57.344273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:57.345611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:57.361784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.362845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.363233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.365595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.366601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.367453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.415450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:55:57.514928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d0d3e77-c74d-4f5a-8dba-61ecde11b99e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 204 len: 0 microversion: 2.1 time: 0.191868 Oct 15 20:55:57.515832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1038/2152] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:55:57 2022] DELETE /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:55:57.522404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:57.525073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:57.553457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.555275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.561594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.562644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.563728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.564346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.664871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4485a9c7-faff-43bd-aad5-48f3d7754651 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.147680 Oct 15 20:55:57.669993 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1039/2153] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:55:57 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments => generated 25 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:55:57.691481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:57.693152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:57.713823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.714265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.714729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.717840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:57.721018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:57.721782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:57.919545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:57.928165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c749813a-52c5-4b6b-9a5e-e7c3d5d21296 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1518 microversion: 2.1 time: 0.255829 Oct 15 20:55:57.929315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1040/2154] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:57 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1518 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:57.934686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-559847865", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c130cb51-e60d-43b7-af80-10e77fbe9595"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:58.040867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] validate_networks() for [('c130cb51-e60d-43b7-af80-10e77fbe9595', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:55:58.554358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:55:58.554954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:55:58.555556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:55:58.556086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:55:58.556618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:55:58.567894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:58.569111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:58.570028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:58.571151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:58.571852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:58.572523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:58.598532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:55:58.605024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for user 5d93a680570f4d23b1f6df534bc082b2 and project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:55:58.634794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:55:58.637711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:55:58.715909 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab47b808-2c51-4292-9aa0-b3c9947323a7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.780580 Oct 15 20:55:58.730060 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1041/2155] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 788 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:55:58.737762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:58.741351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:58.752243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:58.753147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:58.753806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:58.754760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:58.755528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:58.756334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:58.819030 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7725e333-2338-4fbc-9d55-9c84b0ea8099 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.087252 Oct 15 20:55:58.843816 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1042/2156] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:55:58 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212/os-volume_attachments => generated 25 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:55:58.865840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:58.869356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:58.897277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:58.899179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:58.900298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:58.902455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:58.905522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:58.907580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:59.309534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:59.324249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fdd00233-34cd-4dd9-95eb-70987689681c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.477414 Oct 15 20:55:59.337068 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1043/2157] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:58 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:59.337068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-207397a5-c3eb-41c8-9333-93e4ff1cb578 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1081463452", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-207397a5-c3eb-41c8-9333-93e4ff1cb578 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 243 Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:55:59.371162 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:55:59.373258 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1081463452", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1081463452', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'': 'data1'}, 'networks': [{'uuid': 'b70a0bf4-f480-42f6-93c5-ff836a91e53c'}]}}} Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 20:55:59.374810 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'': 'data1'} Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:55:59.382408 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-207397a5-c3eb-41c8-9333-93e4ff1cb578 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:59.384509 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-207397a5-c3eb-41c8-9333-93e4ff1cb578 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 175 microversion: 2.1 time: 0.055056 Oct 15 20:55:59.397757 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1044/2158] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:55:59 2022] POST /compute/v2.1/servers => generated 175 bytes in 71 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:55:59.403207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:59.408714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:59.422862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:59.423375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:59.423830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:59.424563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:59.425028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:59.425493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:59.589433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:55:59.614270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-29da4ac6-144a-4c29-8eb5-fcc9b3c88835 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1211 microversion: 2.1 time: 0.215033 Oct 15 20:55:59.617116 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1045/2159] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:59 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1211 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:55:59.624087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:59.625141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:59.632262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:59.632753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:59.633235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:59.634058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:59.634795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:59.635642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:59.676525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:55:59.802626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=4,nova_cell1:UPDATE=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:55:59.877233 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-169a868e-937f-4924-aff7-9b76b21c670a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 204 len: 0 microversion: 2.1 time: 0.256592 Oct 15 20:55:59.878993 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1046/2160] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:55:59 2022] DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:55:59.889824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b41af75-a03a-4647-8868-51d21920dab2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:59.890430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b41af75-a03a-4647-8868-51d21920dab2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: 323fb94e-38f8-4842-bb3c-abccde022c6a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:59.898134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8b41af75-a03a-4647-8868-51d21920dab2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] HTTP exception thrown: Instance 323fb94e-38f8-4842-bb3c-abccde022c6a could not be found. Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b41af75-a03a-4647-8868-51d21920dab2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/323fb94e-38f8-4842-bb3c-abccde022c6a/metadata HTTP/1.1 Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:55:59.907632 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:55:59.909538 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 323fb94e-38f8-4842-bb3c-abccde022c6a could not be found. Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '323fb94e-38f8-4842-bb3c-abccde022c6a'} Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 323fb94e-38f8-4842-bb3c-abccde022c6a could not be found. Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:55:59.910344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_metadata.py", line 54, in index Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, server_id) Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 323fb94e-38f8-4842-bb3c-abccde022c6a could not be found. Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 323fb94e-38f8-4842-bb3c-abccde022c6a could not be found. Oct 15 20:55:59.911793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:55:59.914627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b41af75-a03a-4647-8868-51d21920dab2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 404 to user: Instance 323fb94e-38f8-4842-bb3c-abccde022c6a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:55:59.918572 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b41af75-a03a-4647-8868-51d21920dab2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/323fb94e-38f8-4842-bb3c-abccde022c6a/metadata" status: 404 len: 111 microversion: 2.1 time: 0.037856 Oct 15 20:55:59.919980 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1047/2161] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:55:59 2022] GET /compute/v2.1/servers/323fb94e-38f8-4842-bb3c-abccde022c6a/metadata => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:55:59.937245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:55:59.937876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:55:59.950179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:59.951320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:59.952003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:55:59.954406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:55:59.954886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:55:59.956119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.267580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:00.277649 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38b41d9b-5a3d-437a-9571-c84937423efa tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1680 microversion: 2.1 time: 0.347091 Oct 15 20:56:00.293395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1048/2162] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:55:59 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1680 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:00.300742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1277d25f-0f09-4f5f-b7d1-628be0bd7b0b req-eb3963ab-3123-4f1c-90b6-9d67fb34ddde service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dafc5b2-733a-433a-bbcc-5136827624c7", "name": "network-vif-unplugged", "status": "completed", "tag": "cd9552df-0245-4ae6-8b4f-63fd9d25ec69"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:00.315255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1277d25f-0f09-4f5f-b7d1-628be0bd7b0b req-eb3963ab-3123-4f1c-90b6-9d67fb34ddde service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.316746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1277d25f-0f09-4f5f-b7d1-628be0bd7b0b req-eb3963ab-3123-4f1c-90b6-9d67fb34ddde service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.317447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1277d25f-0f09-4f5f-b7d1-628be0bd7b0b req-eb3963ab-3123-4f1c-90b6-9d67fb34ddde service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.358956 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1277d25f-0f09-4f5f-b7d1-628be0bd7b0b req-eb3963ab-3123-4f1c-90b6-9d67fb34ddde service nova] Creating event network-vif-unplugged:cd9552df-0245-4ae6-8b4f-63fd9d25ec69 for instance 7dafc5b2-733a-433a-bbcc-5136827624c7 on nch1-master-1 Oct 15 20:56:00.364517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1277d25f-0f09-4f5f-b7d1-628be0bd7b0b req-eb3963ab-3123-4f1c-90b6-9d67fb34ddde service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.068930 Oct 15 20:56:00.365273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1049/2163] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:00 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:00.379640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4211d79-ab59-4ecc-8cba-3334aed1475d tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1490121360", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4211d79-ab59-4ecc-8cba-3334aed1475d tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 499 Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:00.385391 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:00.387060 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:56:00.387788 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1490121360", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} Oct 15 20:56:00.387788 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:00.387788 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.387788 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1490121360', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': 'b70a0bf4-f480-42f6-93c5-ff836a91e53c'}]}}} Oct 15 20:56:00.387788 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:00.387788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:00.388314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:56:00.390770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:56:00.393446 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:56:00.395681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.395681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.395681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4211d79-ab59-4ecc-8cba-3334aed1475d tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:00.396416 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4211d79-ab59-4ecc-8cba-3334aed1475d tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 687 microversion: 2.1 time: 0.017979 Oct 15 20:56:00.396416 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1050/2164] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:00 2022] POST /compute/v2.1/servers => generated 687 bytes in 21 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:56:00.407357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-40a1f76b-3295-4412-8791-059e091417ed req-fb7a4f34-9570-4ac8-9139-1bf9fa527447 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dafc5b2-733a-433a-bbcc-5136827624c7", "name": "network-vif-deleted", "tag": "cd9552df-0245-4ae6-8b4f-63fd9d25ec69"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:00.429872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-40a1f76b-3295-4412-8791-059e091417ed req-fb7a4f34-9570-4ac8-9139-1bf9fa527447 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.430562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-40a1f76b-3295-4412-8791-059e091417ed req-fb7a4f34-9570-4ac8-9139-1bf9fa527447 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.431194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-40a1f76b-3295-4412-8791-059e091417ed req-fb7a4f34-9570-4ac8-9139-1bf9fa527447 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.462856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-40a1f76b-3295-4412-8791-059e091417ed req-fb7a4f34-9570-4ac8-9139-1bf9fa527447 service nova] Creating event network-vif-deleted:cd9552df-0245-4ae6-8b4f-63fd9d25ec69 for instance 7dafc5b2-733a-433a-bbcc-5136827624c7 on nch1-master-1 Oct 15 20:56:00.469231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-40a1f76b-3295-4412-8791-059e091417ed req-fb7a4f34-9570-4ac8-9139-1bf9fa527447 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080014 Oct 15 20:56:00.470135 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1051/2165] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:00.477526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:00.478778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:00.488244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.488966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.489597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.490478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.491098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.491696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.771280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:00.799662 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-051dd3b1-7523-40c4-bf16-8cc2adce29bf tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.328116 Oct 15 20:56:00.817649 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1052/2166] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:00 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:00.825095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f4bfae3b-49d4-4d56-a286-832771a9b350 req-f9e5dbcc-1834-4032-aae7-251da447bcf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23abfd9b-7822-4ed5-82d0-e857368017de", "name": "network-vif-unplugged", "status": "completed", "tag": "166c20c8-2a75-4402-bb95-0d1df37d8744"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:00.842340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4bfae3b-49d4-4d56-a286-832771a9b350 req-f9e5dbcc-1834-4032-aae7-251da447bcf9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.844370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4bfae3b-49d4-4d56-a286-832771a9b350 req-f9e5dbcc-1834-4032-aae7-251da447bcf9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.845638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4bfae3b-49d4-4d56-a286-832771a9b350 req-f9e5dbcc-1834-4032-aae7-251da447bcf9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.862077 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f4bfae3b-49d4-4d56-a286-832771a9b350 req-f9e5dbcc-1834-4032-aae7-251da447bcf9 service nova] Creating event network-vif-unplugged:166c20c8-2a75-4402-bb95-0d1df37d8744 for instance 23abfd9b-7822-4ed5-82d0-e857368017de on nch1-master-1 Oct 15 20:56:00.869332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f4bfae3b-49d4-4d56-a286-832771a9b350 req-f9e5dbcc-1834-4032-aae7-251da447bcf9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.050080 Oct 15 20:56:00.870822 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1053/2167] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:00 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:00.877576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04562772-c1b9-4bfa-8b42-fcca9d8bbf75 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-139291489", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04562772-c1b9-4bfa-8b42-fcca9d8bbf75 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 499 Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:00.882276 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:00.884103 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:56:00.885072 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-139291489", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} Oct 15 20:56:00.885072 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:00.885072 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.885072 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-139291489', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': 'b70a0bf4-f480-42f6-93c5-ff836a91e53c'}]}}} Oct 15 20:56:00.885072 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:00.885072 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:00.885679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:56:00.887715 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:56:00.889509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:56:00.890284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:00.890284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:00.891954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04562772-c1b9-4bfa-8b42-fcca9d8bbf75 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:00.893134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04562772-c1b9-4bfa-8b42-fcca9d8bbf75 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 689 microversion: 2.1 time: 0.020905 Oct 15 20:56:00.894227 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1054/2168] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:00 2022] POST /compute/v2.1/servers => generated 689 bytes in 23 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:56:00.902570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:00.904596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:00.912529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.913389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.914507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:00.915570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:00.916264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:00.916875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.032176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:01.040558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0452d1c-773a-4bb2-b7e9-f31a8cf77fad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1259 microversion: 2.1 time: 0.144778 Oct 15 20:56:01.041253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1055/2169] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:00 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1259 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:01.063019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1de16ed4-2c86-4755-8f74-1d4e1196bfd2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1622031422", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1de16ed4-2c86-4755-8f74-1d4e1196bfd2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 754 Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:01.067737 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:01.069815 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:56:01.070774 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1622031422", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} Oct 15 20:56:01.070774 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:01.070774 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1622031422', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': 'b70a0bf4-f480-42f6-93c5-ff836a91e53c'}]}}} Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:56:01.071074 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:01.071747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:56:01.073523 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:56:01.074816 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:56:01.075721 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.075721 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.075922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1de16ed4-2c86-4755-8f74-1d4e1196bfd2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:01.086797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1de16ed4-2c86-4755-8f74-1d4e1196bfd2 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 1197 microversion: 2.1 time: 0.026962 Oct 15 20:56:01.090818 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1056/2170] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:01 2022] POST /compute/v2.1/servers => generated 1197 bytes in 49 msecs (HTTP/1.1 400) 9 headers in 382 bytes (1 switches on core 0) Oct 15 20:56:01.146727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd9ea45b-e53e-4e14-a0fe-7036fe06927a tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-735523602", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd9ea45b-e53e-4e14-a0fe-7036fe06927a tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 1265 Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:01.153029 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:01.155535 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:56:01.156356 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-735523602", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "b70a0bf4-f480-42f6-93c5-ff836a91e53c"}]}} Oct 15 20:56:01.156356 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-735523602', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': 'b70a0bf4-f480-42f6-93c5-ff836a91e53c'}]}}} Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.156559 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:56:01.158200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:56:01.165336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 20:56:01.165336 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.165336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd9ea45b-e53e-4e14-a0fe-7036fe06927a tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:01.165336 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd9ea45b-e53e-4e14-a0fe-7036fe06927a tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 2221 microversion: 2.1 time: 0.014804 Oct 15 20:56:01.165336 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1057/2171] 10.222.0.16 () {60 vars in 1148 bytes} [Sat Oct 15 20:56:01 2022] POST /compute/v2.1/servers => generated 2221 bytes in 18 msecs (HTTP/1.1 400) 9 headers in 382 bytes (1 switches on core 0) Oct 15 20:56:01.177529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15fd0403-d053-4707-903a-1f714c27d8ab tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:01.178943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-15fd0403-d053-4707-903a-1f714c27d8ab tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: 2c7b08e5-e5ac-496c-bc07-3f690cc906e7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:01.197921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-15fd0403-d053-4707-903a-1f714c27d8ab tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] HTTP exception thrown: Instance 2c7b08e5-e5ac-496c-bc07-3f690cc906e7 could not be found. Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15fd0403-d053-4707-903a-1f714c27d8ab tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/2c7b08e5-e5ac-496c-bc07-3f690cc906e7/metadata/test2 HTTP/1.1 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 2c7b08e5-e5ac-496c-bc07-3f690cc906e7 could not be found. Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '2c7b08e5-e5ac-496c-bc07-3f690cc906e7', 'id': 'test2'} Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2c7b08e5-e5ac-496c-bc07-3f690cc906e7 could not be found. Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_metadata.py", line 129, in show Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, server_id) Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2c7b08e5-e5ac-496c-bc07-3f690cc906e7 could not be found. Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:01.200850 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:01.207181 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:01.207181 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:01.207181 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2c7b08e5-e5ac-496c-bc07-3f690cc906e7 could not be found. Oct 15 20:56:01.207181 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:01.207181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15fd0403-d053-4707-903a-1f714c27d8ab tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 404 to user: Instance 2c7b08e5-e5ac-496c-bc07-3f690cc906e7 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:01.217717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15fd0403-d053-4707-903a-1f714c27d8ab tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/2c7b08e5-e5ac-496c-bc07-3f690cc906e7/metadata/test2" status: 404 len: 111 microversion: 2.1 time: 0.041126 Oct 15 20:56:01.217717 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1058/2172] 10.222.0.16 () {58 vars in 1281 bytes} [Sat Oct 15 20:56:01 2022] GET /compute/v2.1/servers/2c7b08e5-e5ac-496c-bc07-3f690cc906e7/metadata/test2 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:01.222686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:01.223206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:01.229431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.230196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.230647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.231618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.232927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.233678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.260774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:01.352187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03ff11a7-97e6-4fc4-a5fc-1eec801dda80 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 204 len: 0 microversion: 2.1 time: 0.133086 Oct 15 20:56:01.357424 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1059/2173] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:01 2022] DELETE /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:01.371067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:01.373258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:01.381936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.382537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.383049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.383842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.384387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.385077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.543227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:01.553168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-397ec590-d3d9-451d-b31d-610e076f754c tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1680 microversion: 2.1 time: 0.194357 Oct 15 20:56:01.553872 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1060/2174] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:01 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1680 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:01.567501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:01.568542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:01.575309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.576907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.577386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.578265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.578976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.579415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.820735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:01.829347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-611b3527-38dd-4a7f-be51-ca24426109ca tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1524 microversion: 2.1 time: 0.274084 Oct 15 20:56:01.830017 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1061/2175] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:01 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1524 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:01.834574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:01.835256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:01.842764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.843607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.844997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:01.846049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:01.846970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:01.847584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:02.222011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:02.230015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9758008e-a943-4c14-b153-a04edf474749 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.398940 Oct 15 20:56:02.241268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1062/2176] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:01 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:02.241268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:02.241268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:02.256338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:02.256845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:02.257330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:02.258053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:02.258526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:02.258947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:02.393330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:02.406246 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20033e6d-7779-469e-ab9f-c5c73e1061f1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1259 microversion: 2.1 time: 0.174307 Oct 15 20:56:02.421552 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1063/2177] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:02 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1259 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:02.577766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:02.578563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:02.586384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:02.586886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:02.587332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:02.588100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:02.588577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:02.588995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.211505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:03.223612 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd4ac6e2-f25b-4b5e-9348-6ef0fa0f16b6 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1680 microversion: 2.1 time: 0.650579 Oct 15 20:56:03.224459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1064/2178] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:02 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1680 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:56:03.237478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:03.238278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:03.245751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.246270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.246773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.250552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.250552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.250921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.280515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:03.371457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d895904-7fb2-45ad-9205-7acad296831b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 204 len: 0 microversion: 2.1 time: 0.145597 Oct 15 20:56:03.374503 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1065/2179] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:03 2022] DELETE /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:03.382350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:03.385165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:03.387203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=1,nova_api:UPDATE=4,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:03.395920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.396401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.396726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.397526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.398191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.398795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.525344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f0724ab706d407fb067d4ed444e10a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '678b3cd3dda745fda1201db024f2d746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:03.537547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4c27a19-ae67-4a26-8774-379417b53a7b tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 200 len: 1482 microversion: 2.1 time: 0.161215 Oct 15 20:56:03.538868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1066/2180] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:03 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 1482 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:03.546099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0e8108d7-525e-4101-a856-f4069d04b66c req-3d5cdf1d-86d8-4a42-a775-6cb3bde6d214 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "23abfd9b-7822-4ed5-82d0-e857368017de", "tag": "166c20c8-2a75-4402-bb95-0d1df37d8744"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:03.559123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e8108d7-525e-4101-a856-f4069d04b66c req-3d5cdf1d-86d8-4a42-a775-6cb3bde6d214 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.560046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e8108d7-525e-4101-a856-f4069d04b66c req-3d5cdf1d-86d8-4a42-a775-6cb3bde6d214 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.561100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e8108d7-525e-4101-a856-f4069d04b66c req-3d5cdf1d-86d8-4a42-a775-6cb3bde6d214 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.574500 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0e8108d7-525e-4101-a856-f4069d04b66c req-3d5cdf1d-86d8-4a42-a775-6cb3bde6d214 service nova] Creating event network-changed:166c20c8-2a75-4402-bb95-0d1df37d8744 for instance 23abfd9b-7822-4ed5-82d0-e857368017de on nch1-master-1 Oct 15 20:56:03.589360 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0e8108d7-525e-4101-a856-f4069d04b66c req-3d5cdf1d-86d8-4a42-a775-6cb3bde6d214 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048675 Oct 15 20:56:03.589984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1067/2181] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:03.596901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:03.597920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:03.614278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.615735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.616470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:03.619560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:03.620697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:03.621459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.032970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:04.036726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:04.042315 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d66f4b86-7dfb-4d37-9d6b-b38cbef325cb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.449703 Oct 15 20:56:04.043154 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1068/2182] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:03 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:04.047913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:04.048529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:04.058595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.059098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.059717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.061153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.061709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.062124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.525697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:04.535677 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39a54642-4536-479e-814b-d0026cee5aa4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1554 microversion: 2.1 time: 0.491660 Oct 15 20:56:04.536406 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1069/2183] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:04 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1554 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:04.541028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:04.541668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:04.549073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.549878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.550574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.551664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.552402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.553231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.663085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:04.672420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73e51284-e06b-4f9d-b39c-d8b8865d0f2a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1265 microversion: 2.1 time: 0.134927 Oct 15 20:56:04.689020 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1070/2184] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:04 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1265 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:04.689020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f9b91cdc-a5e2-48d2-b3eb-e6178b45702c req-55661ef3-64bc-4859-a62b-54ed758299b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7", "name": "network-vif-deleted", "tag": "0b471f75-8cf1-4971-a96a-2b14e208ce42"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:04.692451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f9b91cdc-a5e2-48d2-b3eb-e6178b45702c req-55661ef3-64bc-4859-a62b-54ed758299b9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.692938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f9b91cdc-a5e2-48d2-b3eb-e6178b45702c req-55661ef3-64bc-4859-a62b-54ed758299b9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.693401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f9b91cdc-a5e2-48d2-b3eb-e6178b45702c req-55661ef3-64bc-4859-a62b-54ed758299b9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.711968 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f9b91cdc-a5e2-48d2-b3eb-e6178b45702c req-55661ef3-64bc-4859-a62b-54ed758299b9 service nova] Creating event network-vif-deleted:0b471f75-8cf1-4971-a96a-2b14e208ce42 for instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 on nch1-master-1 Oct 15 20:56:04.718115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f9b91cdc-a5e2-48d2-b3eb-e6178b45702c req-55661ef3-64bc-4859-a62b-54ed758299b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043801 Oct 15 20:56:04.722026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1071/2185] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:04.728249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:04.729215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:04.744945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.745760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.746424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.746972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.748181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.751839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.886758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '999bc03f52824573bb3c8bdc6a1179cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e67382792a7d4ccf8100febea9052e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:04.911921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccc2e8b8-4d6b-48c0-9037-845392f88a2a tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 200 len: 1613 microversion: 2.1 time: 0.188453 Oct 15 20:56:04.934122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1072/2186] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:04 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 1613 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:04.939533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:04.945794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] [instance: a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:04.958893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.969909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.970377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.971156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:04.971630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:04.972068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:04.990165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] HTTP exception thrown: Instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 could not be found. Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Request method failure captured: Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 HTTP/1.1 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx5Li82BIaGtOg080daJtGzz43-0hltW5WQIUU1I9GzxzXj8OtmpC3XqRvQLQEEJWjMAinweXZ_xBuU7e0O0uWqNTdMoacwc4lqMojLfOFxUSA9t7q55qoiwG7OliFssfNBID5U9Hsnamnu5IzG6j9vuUBGJXyOEaY1N78iDhC3hH0E Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_678b3cd3dda745fda1201db024f2d746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/678b3cd3dda745fda1201db024f2d746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 678b3cd3dda745fda1201db024f2d746 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-2122335498 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5f0724ab706d407fb067d4ed444e10a4 Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-2122335498-project Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 could not be found. Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7'} Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 could not be found. Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 could not be found. Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:04.991812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 could not be found. Oct 15 20:56:04.999029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:05.001369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] Returning 404 to user: Instance a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:05.002641 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-670a9ad8-a9f1-487d-9773-6e2212df14c9 tempest-ServerMetadataNegativeTestJSON-2122335498 tempest-ServerMetadataNegativeTestJSON-2122335498-project] 10.222.0.16 "GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7" status: 404 len: 111 microversion: 2.1 time: 0.066887 Oct 15 20:56:05.003227 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1073/2187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:04 2022] GET /compute/v2.1/servers/a96ddf1f-cfd4-4be1-ba53-6d6d18497ce7 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:05.068800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:05.069744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:05.080415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:05.083246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:05.085693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:05.086456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:05.086939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:05.087383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:05.815495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:05.830037 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78a518f4-871a-419b-a568-bebd312929de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.766122 Oct 15 20:56:05.832172 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1074/2188] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:05 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:05.840530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:05.841758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:05.851087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:05.851716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:05.852230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:05.853020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:05.853618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:05.854145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:06.433912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:06.452557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df9c3065-cc21-4cb3-8371-4429742e0645 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1554 microversion: 2.1 time: 0.618086 Oct 15 20:56:06.465072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1075/2189] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:05 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1554 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:06.476576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:06.480445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:06.496238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:06.496467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:06.497167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:06.498504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:06.503533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:06.504967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:06.983958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:06.998240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d663a52-cdac-4d32-9eee-17b4d35a7db1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1307 microversion: 2.1 time: 0.536441 Oct 15 20:56:07.001315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1076/2190] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:06 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1307 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:07.013255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a710d112-5c34-469a-94bb-e493881ada39 req-31e65afb-349c-4612-b594-42b3d6d4d23e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82e74d13-2be4-47d0-93fa-fb24eb81c212", "name": "network-vif-deleted", "tag": "328d28c6-9ece-4b0b-b5f5-e870e5285550"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:07.032924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a710d112-5c34-469a-94bb-e493881ada39 req-31e65afb-349c-4612-b594-42b3d6d4d23e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.037679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a710d112-5c34-469a-94bb-e493881ada39 req-31e65afb-349c-4612-b594-42b3d6d4d23e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.038323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a710d112-5c34-469a-94bb-e493881ada39 req-31e65afb-349c-4612-b594-42b3d6d4d23e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.073854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a710d112-5c34-469a-94bb-e493881ada39 req-31e65afb-349c-4612-b594-42b3d6d4d23e service nova] Creating event network-vif-deleted:328d28c6-9ece-4b0b-b5f5-e870e5285550 for instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 on nch2-master-1 Oct 15 20:56:07.084628 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a710d112-5c34-469a-94bb-e493881ada39 req-31e65afb-349c-4612-b594-42b3d6d4d23e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080274 Oct 15 20:56:07.085390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1077/2191] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:07.097476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:07.098646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:07.118138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.118688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.119147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.119898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.120359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.120789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.151695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] HTTP exception thrown: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Request method failure captured: Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 HTTP/1.1 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx2mEK7T3X4qdl4ZGjfMPADYXQ41A4NarhFBzUiFmgd7Vi-imtdGViwGtITjxls95QWFWSucBJMDWthkk33F6uFHB4lNB2Se4Ykgb29tA8f0WCKX5WK0-aCcioy_Z3bsLB8cJ6UMThVeTvYGlchIdnlMdme_Hkkil4sSaCYBIor_Kvc Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e67382792a7d4ccf8100febea9052e1b Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e67382792a7d4ccf8100febea9052e1b Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachSCSIVolumeTestJSON-866681318-project Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 999bc03f52824573bb3c8bdc6a1179cf Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-866681318-project Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '82e74d13-2be4-47d0-93fa-fb24eb81c212'} Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:07.154438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:07.159438 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:07.162706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Returning 404 to user: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:07.163823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87cf495a-2457-468a-9b7e-788b2182961b tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 404 len: 111 microversion: 2.1 time: 0.077123 Oct 15 20:56:07.167400 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1078/2192] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:07 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:07.178449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e0c104bc-0ec4-4447-9466-c2dc12d8100c req-e9590e75-0d1e-4211-870a-de99c91f8526 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8077ee04-cf77-4ad7-aef9-1d0244cfa551", "name": "network-vif-unplugged", "status": "completed", "tag": "8ca72979-3681-49ec-9dad-1321aad91ce3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:07.194260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e0c104bc-0ec4-4447-9466-c2dc12d8100c req-e9590e75-0d1e-4211-870a-de99c91f8526 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.195132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e0c104bc-0ec4-4447-9466-c2dc12d8100c req-e9590e75-0d1e-4211-870a-de99c91f8526 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.195790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e0c104bc-0ec4-4447-9466-c2dc12d8100c req-e9590e75-0d1e-4211-870a-de99c91f8526 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.213363 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e0c104bc-0ec4-4447-9466-c2dc12d8100c req-e9590e75-0d1e-4211-870a-de99c91f8526 service nova] Creating event network-vif-unplugged:8ca72979-3681-49ec-9dad-1321aad91ce3 for instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 on nch2-master-1 Oct 15 20:56:07.219422 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e0c104bc-0ec4-4447-9466-c2dc12d8100c req-e9590e75-0d1e-4211-870a-de99c91f8526 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.050566 Oct 15 20:56:07.220643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1079/2193] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:07 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:07.227279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:07.228716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:07.238361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.239678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.244932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.245735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.246209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.246765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.635625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:07.662936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-515a0c65-1229-424a-8d97-73124ed122d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.439495 Oct 15 20:56:07.664304 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1080/2194] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:07 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:07.672164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69920ea7-9684-4f9a-ad04-e24e8b8b83e5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '.version_select at 0x7fe758535ca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:07.742803 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69920ea7-9684-4f9a-ad04-e24e8b8b83e5 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1640437525" status: 202 len: 0 microversion: 2.1 time: 0.077095 Oct 15 20:56:07.755514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1081/2195] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:56:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1640437525 => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:07.788450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:07.789737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:07.806482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.809770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.813736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:07.824399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:07.826362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:07.827146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:08.020638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:08.034438 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c62d1d64-3c7f-4158-9c05-9cb46cc5a0ab tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 200 len: 1512 microversion: 2.1 time: 0.277085 Oct 15 20:56:08.038117 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1082/2196] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:07 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 1512 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:08.049590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:08.057426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:08.077316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:08.078436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:08.081422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:08.096022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:08.099519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:08.100592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:08.475270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:08.489608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1a91986-e430-4f91-b775-b55fe3f9ae11 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1307 microversion: 2.1 time: 0.449903 Oct 15 20:56:08.499028 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1083/2197] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:08 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1307 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:08.690333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:08.691653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:08.695670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:08.696786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:08.712003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:08.712534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:08.713271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:08.715010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:08.717178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:08.717902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:09.559245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:09.580062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98913469-6a03-467c-9b2c-915686f401dd tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.894886 Oct 15 20:56:09.580838 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1084/2198] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:08 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:09.592962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:09.596954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:09.612271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:09.612774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:09.613243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:09.613988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:09.614645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:09.615247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:09.637580 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 HTTP/1.1 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8077ee04-cf77-4ad7-aef9-1d0244cfa551'} Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:09.640660 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:09.646867 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c90bd8b2-27ce-436d-b45a-76e9434a7ea9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 404 len: 111 microversion: 2.1 time: 0.060439 Oct 15 20:56:09.649886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1085/2199] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:09 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:09.653268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4da129cd-0c8c-4e46-bcc6-beb0550690da", "tag": "ab0aa7bc-7367-426b-a481-53223b9795b8"}, {"server_uuid": "8077ee04-cf77-4ad7-aef9-1d0244cfa551", "name": "network-vif-deleted", "tag": "8ca72979-3681-49ec-9dad-1321aad91ce3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:09.670776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:09.671250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:09.671682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:09.689561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] Creating event network-changed:ab0aa7bc-7367-426b-a481-53223b9795b8 for instance 4da129cd-0c8c-4e46-bcc6-beb0550690da on nch2-master-1 Oct 15 20:56:09.690164 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] Creating event network-vif-deleted:8ca72979-3681-49ec-9dad-1321aad91ce3 for instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 on nch2-master-1 Oct 15 20:56:09.699998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a29f8ef3-13c7-422a-b1cb-22f1a6e9385d req-e7d80030-b599-4e0f-8e2c-b8215394ccb9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.051038 Oct 15 20:56:09.702218 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1086/2200] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:09 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:09.709887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:09.710651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:09.720065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:09.720573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:09.721119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:09.722037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:09.722995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:09.723443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:10.440830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:10.458096 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1649fcaa-cd53-4875-978b-1fee8744d0b4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1307 microversion: 2.1 time: 0.754468 Oct 15 20:56:10.459233 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1087/2201] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:09 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1307 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:10.464306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:10.483470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:10.489781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:10.490279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:10.490928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:10.491803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:10.492427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:10.493233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:10.522751 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551/os-volume_attachments HTTP/1.1 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '8077ee04-cf77-4ad7-aef9-1d0244cfa551'} Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:10.524825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 20:56:10.530507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:10.531817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:10.533560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6db706fd-0630-4dfc-9991-a831272d97a5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.072423 Oct 15 20:56:10.534978 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1088/2202] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 20:56:10 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551/os-volume_attachments => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:10.605459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:10.606208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:10.626449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:10.628793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:10.631617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:10.635887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:10.636716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:10.637357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:10.961329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:10.973428 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fef633bc-2cbb-467a-b5ce-0041a1f37216 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.372616 Oct 15 20:56:10.974532 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1089/2203] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:10 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:10.981062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-617782041", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:11.059557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:56:11.910607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:56:11.911181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:56:11.911509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:56:11.912444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:56:11.912444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:56:11.939522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:11.943695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:11.950296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:11.954358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:11.956410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:11.958299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:12.012066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:56:12.026697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:56:12.048326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:56:12.052112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:56:12.164179 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35bd5328-2c05-4281-a46a-261e925d0a54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.186973 Oct 15 20:56:12.170091 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1090/2204] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 1195 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:56:12.175564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:12.182177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:12.195122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:12.196046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:12.196673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:12.197629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:12.201366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:12.202156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:12.750945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:12.798260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f8aacb6-5ac0-4d09-8e65-228f25f2348e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1307 microversion: 2.1 time: 0.626374 Oct 15 20:56:12.813079 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1091/2205] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:12 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1307 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:12.813079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:12.813079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:12.815213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:12.815989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:12.816659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:12.817707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:12.818394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:12.819333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:12.852240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:13.237824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:13.255301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-28e8d3aa-fbaa-401d-87ab-7b0542c17795 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1393 microversion: 2.1 time: 0.452796 Oct 15 20:56:13.261722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1092/2206] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:12 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1393 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:13.269000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:13.270647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:13.281983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.282976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.283795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.284839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.285599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.286349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.400982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:13.414284 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea7fd9a5-bf17-415c-87f5-046992b78c96 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 200 len: 1266 microversion: 2.1 time: 0.150979 Oct 15 20:56:13.415682 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1093/2207] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:13 2022] GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:13.423876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:13.424640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:13.438191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.438774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.439233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.447308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.448353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.450378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.519257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] HTTP exception thrown: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Request method failure captured: Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 HTTP/1.1 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx2mEK7T3X4qdl4ZGjfMPADYXQ41A4NarhFBzUiFmgd7Vi-imtdGViwGtITjxls95QWFWSucBJMDWthkk33F6uFHB4lNB2Se4Ykgb29tA8f0WCKX5WK0-aCcioy_Z3bsLB8cJ6UMThVeTvYGlchIdnlMdme_Hkkil4sSaCYBIor_Kvc Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e67382792a7d4ccf8100febea9052e1b Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e67382792a7d4ccf8100febea9052e1b Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachSCSIVolumeTestJSON-866681318-project Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 999bc03f52824573bb3c8bdc6a1179cf Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-866681318-project Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '82e74d13-2be4-47d0-93fa-fb24eb81c212'} Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:13.520938 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Returning 404 to user: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6683c397-e809-4924-ba9b-fb806a22c349 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 404 len: 111 microversion: 2.1 time: 0.105105 Oct 15 20:56:13.527768 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1094/2208] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:13 2022] DELETE /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 111 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:13.536900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:13.538272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:13.552809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.554875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.557140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.557928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.558405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.559990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.594784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:13.730565 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5421a5a-103f-4b12-94ff-145e45e87d50 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 204 len: 0 microversion: 2.1 time: 0.205644 Oct 15 20:56:13.731891 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1095/2209] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:13 2022] DELETE /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:13.739427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:13.740517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] [instance: 82e74d13-2be4-47d0-93fa-fb24eb81c212] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:13.747446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.748036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.748512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.749653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.750341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.750781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.767015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] HTTP exception thrown: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Request method failure captured: Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 HTTP/1.1 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx2mEK7T3X4qdl4ZGjfMPADYXQ41A4NarhFBzUiFmgd7Vi-imtdGViwGtITjxls95QWFWSucBJMDWthkk33F6uFHB4lNB2Se4Ykgb29tA8f0WCKX5WK0-aCcioy_Z3bsLB8cJ6UMThVeTvYGlchIdnlMdme_Hkkil4sSaCYBIor_Kvc Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e67382792a7d4ccf8100febea9052e1b Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e67382792a7d4ccf8100febea9052e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e67382792a7d4ccf8100febea9052e1b Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-866681318 Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachSCSIVolumeTestJSON-866681318-project Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 999bc03f52824573bb3c8bdc6a1179cf Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-866681318-project Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '82e74d13-2be4-47d0-93fa-fb24eb81c212'} Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:13.769394 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] Returning 404 to user: Instance 82e74d13-2be4-47d0-93fa-fb24eb81c212 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efccd15f-cec8-4fdb-9565-2e665a63bc18 tempest-AttachSCSIVolumeTestJSON-866681318 tempest-AttachSCSIVolumeTestJSON-866681318-project] 10.222.0.16 "GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212" status: 404 len: 111 microversion: 2.1 time: 0.036253 Oct 15 20:56:13.774117 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1096/2210] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:13 2022] GET /compute/v2.1/servers/82e74d13-2be4-47d0-93fa-fb24eb81c212 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:13.783417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:13.784427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:13.795487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.796184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.796867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.799336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:13.800600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:13.801325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:13.965271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:13.985615 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b86b072-d7c2-4b03-8848-02acee846557 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 200 len: 1272 microversion: 2.1 time: 0.211962 Oct 15 20:56:13.994174 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1097/2211] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:13 2022] GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 1272 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:14.013446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:14.014827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:14.031701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:14.032215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:14.032667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:14.033461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:14.033936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:14.034375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:14.409873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:14.424057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8b77eec-7a8d-4114-9bb9-693f73d00af8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1307 microversion: 2.1 time: 0.419382 Oct 15 20:56:14.424724 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1098/2212] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:13 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1307 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:14.429530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:14.430157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:14.437628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:14.438169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:14.438564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:14.439293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:14.439764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:14.440331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:14.848017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:14.895165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-754a7eb9-aa98-4aac-a1fb-c8ef5e1bf1b5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1383 microversion: 2.1 time: 0.454201 Oct 15 20:56:14.897458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1099/2213] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:14 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1383 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:14.944072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee9c81eb-56ed-4771-a303-7d66305c4ae9 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '.version_select at 0x7fe75845b790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:14.950909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9c81eb-56ed-4771-a303-7d66305c4ae9 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:14.951906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9c81eb-56ed-4771-a303-7d66305c4ae9 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:14.953593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9c81eb-56ed-4771-a303-7d66305c4ae9 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:14.979104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee9c81eb-56ed-4771-a303-7d66305c4ae9 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 2704 microversion: 2.1 time: 0.050436 Oct 15 20:56:14.980426 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1100/2214] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 20:56:14 2022] GET /compute/v2.1/os-migrations => generated 2704 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:15.005988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:15.019214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:15.033518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:15.039243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:15.040098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:15.042192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:15.043421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:15.044071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:15.680732 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f62082fd-e60a-4311-ad7c-705bbd546aa2 tempest-LiveMigrationTest-704853501 tempest-LiveMigrationTest-704853501-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1534 microversion: 2.1 time: 0.686626 Oct 15 20:56:15.686537 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1101/2215] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:14 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1534 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:15.691976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:15.697918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:15.713356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:15.714356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:15.715070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:15.715819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:15.716314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:15.717433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:15.823405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:15.835873 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-49ed6ed0-5480-4a75-a036-7c96e03f7314 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 200 len: 1218 microversion: 2.1 time: 0.147071 Oct 15 20:56:15.836632 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1102/2216] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:15 2022] GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 1218 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:15.847960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:15.849393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:15.859096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:15.859593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:15.860040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:15.860758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:15.861323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:15.861768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:16.292553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:16.303543 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a18bd1ea-8ff1-4c8c-bcd6-9b71a09c4f1f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1307 microversion: 2.1 time: 0.464817 Oct 15 20:56:16.304149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1103/2217] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:15 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1307 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:16.330586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:16.331376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:16.353924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:16.354421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:16.354867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:16.355639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:16.359490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:16.360274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:16.406545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:16.528395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:16.539350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-804ba925-30d6-4dd4-bfa7-2d2c2b20539c tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "DELETE /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 204 len: 0 microversion: 2.1 time: 0.231716 Oct 15 20:56:16.541363 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1104/2218] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:16 2022] DELETE /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 0 bytes in 237 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:16.559022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0fffa310-bf01-44b3-ac22-1e2e5c6c2228 req-a88d67c7-95f2-40d6-85b3-5dfeacbb1e7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4da129cd-0c8c-4e46-bcc6-beb0550690da", "name": "network-vif-plugged", "status": "completed", "tag": "ab0aa7bc-7367-426b-a481-53223b9795b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:16.574547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0fffa310-bf01-44b3-ac22-1e2e5c6c2228 req-a88d67c7-95f2-40d6-85b3-5dfeacbb1e7c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:16.575570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0fffa310-bf01-44b3-ac22-1e2e5c6c2228 req-a88d67c7-95f2-40d6-85b3-5dfeacbb1e7c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:16.577835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0fffa310-bf01-44b3-ac22-1e2e5c6c2228 req-a88d67c7-95f2-40d6-85b3-5dfeacbb1e7c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:16.610731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0fffa310-bf01-44b3-ac22-1e2e5c6c2228 req-a88d67c7-95f2-40d6-85b3-5dfeacbb1e7c service nova] Creating event network-vif-plugged:ab0aa7bc-7367-426b-a481-53223b9795b8 for instance 4da129cd-0c8c-4e46-bcc6-beb0550690da on nch2-master-1 Oct 15 20:56:16.622548 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0fffa310-bf01-44b3-ac22-1e2e5c6c2228 req-a88d67c7-95f2-40d6-85b3-5dfeacbb1e7c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075082 Oct 15 20:56:16.626149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1105/2219] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:16.640078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:16.640743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:16.647909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:16.648467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:16.648985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:16.649974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:16.650617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:16.651223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:17.034536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:17.045695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95b07084-7e90-48c2-956b-4c0242b8f869 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1389 microversion: 2.1 time: 0.416917 Oct 15 20:56:17.046302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1106/2220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:16 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1389 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:17.060810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:17.061870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:17.079015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:17.079533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:17.079982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:17.080716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:17.081209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:17.088248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:17.113914 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 HTTP/1.1 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '04857e01-76b5-45fa-a293-becd94cfd638'} Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:17.115617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:17.121567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e2d8981-31fe-4b89-b3ec-7b870a8c5b71 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 404 len: 111 microversion: 2.1 time: 0.069795 Oct 15 20:56:17.128378 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1107/2221] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:17 2022] GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:17.252557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-483978064", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:17.349904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:56:18.601742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:56:18.602307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:56:18.602634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:56:18.603275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:56:18.603938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:56:18.626738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:18.627808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:18.630313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:18.633009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:18.633844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:18.635201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:18.688396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:56:18.699501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:56:18.719398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:56:18.722855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:56:18.827619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d1994e7-3426-4f81-ae63-130811bf9c5f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.578806 Oct 15 20:56:18.837170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1108/2222] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:17 2022] POST /compute/v2.1/servers => generated 380 bytes in 1588 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 20:56:18.843973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:18.847211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:18.859456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:18.862383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:18.863084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:18.864331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:18.865019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:18.865993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:19.491417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:19.496982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:19.524102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c6c9dd1-8ae8-4432-ad32-b8e0af089ca2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1488 microversion: 2.1 time: 0.685272 Oct 15 20:56:19.528372 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1109/2223] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:18 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1488 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:19.539464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:19.540090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:19.565779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:19.566291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:19.566822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:19.567601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:19.568067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:19.569305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:19.930734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:19.945152 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99da8d01-281e-4bd0-8958-78ad9e9890f5 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 200 len: 1389 microversion: 2.1 time: 0.415531 Oct 15 20:56:19.946005 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1110/2224] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:19 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 1389 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:19.957948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:19.958816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:19.973381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:19.973890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:19.974335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:19.976058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:19.976725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:19.978015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:20.127210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:20.143728 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a037e17-bb9a-4efa-a033-4903b42fc58c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1218 microversion: 2.1 time: 0.196390 Oct 15 20:56:20.144364 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1111/2225] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:19 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1218 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:20.149243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc5b00c7-35da-41d0-8c83-dcfe5736005e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:20.170068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc5b00c7-35da-41d0-8c83-dcfe5736005e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:20.175128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc5b00c7-35da-41d0-8c83-dcfe5736005e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:20.176129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc5b00c7-35da-41d0-8c83-dcfe5736005e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:20.406750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc5b00c7-35da-41d0-8c83-dcfe5736005e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 20:56:20.568934 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc5b00c7-35da-41d0-8c83-dcfe5736005e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da/action" status: 202 len: 0 microversion: 2.1 time: 0.423242 Oct 15 20:56:20.570164 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1112/2226] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:56:20 2022] POST /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da/action => generated 0 bytes in 425 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:20.598929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-30c288b9-7aa5-4c47-8e16-cc63e659c498 req-3a967662-6bda-46ec-92ec-ccf982317dca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23abfd9b-7822-4ed5-82d0-e857368017de", "name": "network-vif-deleted", "tag": "166c20c8-2a75-4402-bb95-0d1df37d8744"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:20.637326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30c288b9-7aa5-4c47-8e16-cc63e659c498 req-3a967662-6bda-46ec-92ec-ccf982317dca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:20.637984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30c288b9-7aa5-4c47-8e16-cc63e659c498 req-3a967662-6bda-46ec-92ec-ccf982317dca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:20.638917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30c288b9-7aa5-4c47-8e16-cc63e659c498 req-3a967662-6bda-46ec-92ec-ccf982317dca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:20.675955 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-30c288b9-7aa5-4c47-8e16-cc63e659c498 req-3a967662-6bda-46ec-92ec-ccf982317dca service nova] Creating event network-vif-deleted:166c20c8-2a75-4402-bb95-0d1df37d8744 for instance 23abfd9b-7822-4ed5-82d0-e857368017de on nch2-master-1 Oct 15 20:56:20.694729 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-30c288b9-7aa5-4c47-8e16-cc63e659c498 req-3a967662-6bda-46ec-92ec-ccf982317dca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122950 Oct 15 20:56:20.695348 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1113/2227] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:20.717975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:20.719802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:20.738292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:20.739794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:20.740749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:20.741891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:20.745994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:20.748743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:21.467539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:21.484798 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa3b7fa4-fe33-4ccd-93fb-02e6fbeca42e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.783336 Oct 15 20:56:21.494501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1114/2228] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:20 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:21.511409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:21.513567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 23abfd9b-7822-4ed5-82d0-e857368017de] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:21.535983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:21.536806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:21.537806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:21.538716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:21.539296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:21.540916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:21.568789 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] HTTP exception thrown: Instance 23abfd9b-7822-4ed5-82d0-e857368017de could not be found. Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Request method failure captured: Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de HTTP/1.1 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzJVMNFSJ0p4PpFJ0U8dQawXH4cP1aPmp0QTTEGgjPUYVyqzTXGaFxiL9a2Au2uMiL8WJFXbsbXM7TakZbSXCacQ53u0dGyroIIoJ9NyVZeGgzPn9JEOEjdrOP1ZlEsoAysHiDKc4bwFsgV1m2jNjrQTCydGjEVr-d_5fjtsldzTuQ Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationTest-392689012 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c613201f55b94a528c6c36e5b5772799 Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 23abfd9b-7822-4ed5-82d0-e857368017de could not be found. Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '23abfd9b-7822-4ed5-82d0-e857368017de'} Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 23abfd9b-7822-4ed5-82d0-e857368017de could not be found. Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 23abfd9b-7822-4ed5-82d0-e857368017de could not be found. Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:21.572563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 23abfd9b-7822-4ed5-82d0-e857368017de could not be found. Oct 15 20:56:21.579472 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:21.582443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Returning 404 to user: Instance 23abfd9b-7822-4ed5-82d0-e857368017de could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:21.583716 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a1ca3447-6802-4166-82c5-2d23eb3ec7de tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de" status: 404 len: 111 microversion: 2.1 time: 0.085443 Oct 15 20:56:21.585138 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1115/2229] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:21 2022] GET /compute/v2.1/servers/23abfd9b-7822-4ed5-82d0-e857368017de => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:21.613104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:21.614175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:21.631426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:21.632598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:21.634111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:21.635105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:21.637449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:21.640540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:21.827019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:21.837252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-caa26dc4-b607-44e8-8ac5-bfc4884c060c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1272 microversion: 2.1 time: 0.250744 Oct 15 20:56:21.838576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1116/2230] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:21 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1272 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:22.511204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:22.512015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:22.523928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:22.526684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:22.527264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:22.530021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:22.531108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:22.532001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:23.176675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:23.206504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7eda288-9ca6-4361-8c01-4ca63a18bbb2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.700756 Oct 15 20:56:23.207434 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1117/2231] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:22 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:23.212247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:23.217618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:23.228006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:23.229644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:23.230287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:23.231313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:23.233306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:23.233936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:23.708717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:23.776300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f21267f3-7367-4439-8019-bf722ecb7ba6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1272 microversion: 2.1 time: 0.565192 Oct 15 20:56:23.778541 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1118/2232] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:23 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1272 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:23.802567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:23.804106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:23.835326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:23.845265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:23.846410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:23.847978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:23.848703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:23.849483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:23.909741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:24.095921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cda82561-dd09-4096-a3a3-b59758828ff3 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "DELETE /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 204 len: 0 microversion: 2.1 time: 0.310553 Oct 15 20:56:24.097478 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1119/2233] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:23 2022] DELETE /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:24.129874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:24.130827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:24.143740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:24.144474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:24.145157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:24.146084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:24.148547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:24.149185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:25.349473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:25.368197 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a6bb7b1-1f84-4675-843e-2bd0bdb4be65 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1667 microversion: 2.1 time: 1.246836 Oct 15 20:56:25.368801 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1120/2234] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:24 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1667 bytes in 1268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 20:56:25.375228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:25.376020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:25.387659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:25.388407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:25.388918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:25.389660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:25.391206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:25.391649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:25.796735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:25.820169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6228fcf-5741-46b3-a2b1-8e9e20f29042 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.449816 Oct 15 20:56:25.829333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1121/2235] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:25 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:25.845624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:25.847497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:25.867062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:25.867568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:25.868013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:25.868738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:25.869617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:25.871375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:26.759967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:26.769714 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-137be59a-628d-4659-940f-3b29dfc0e92b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1314 microversion: 2.1 time: 0.938681 Oct 15 20:56:26.771175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1122/2236] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:25 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1314 bytes in 941 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:26.776963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-05f8238f-3326-47b9-b14e-41b13d7301da req-c549ae63-d9bd-49c1-9e4e-b65a7955a9ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "170b74a2-0481-4eab-acfd-00157c8cb2c5", "tag": "a6059d08-ec01-401a-9918-7c062c85a913"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:26.795011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-05f8238f-3326-47b9-b14e-41b13d7301da req-c549ae63-d9bd-49c1-9e4e-b65a7955a9ad service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:26.795729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-05f8238f-3326-47b9-b14e-41b13d7301da req-c549ae63-d9bd-49c1-9e4e-b65a7955a9ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:26.796312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-05f8238f-3326-47b9-b14e-41b13d7301da req-c549ae63-d9bd-49c1-9e4e-b65a7955a9ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:26.815764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-05f8238f-3326-47b9-b14e-41b13d7301da req-c549ae63-d9bd-49c1-9e4e-b65a7955a9ad service nova] Creating event network-changed:a6059d08-ec01-401a-9918-7c062c85a913 for instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 on nch1-master-1 Oct 15 20:56:26.821660 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-05f8238f-3326-47b9-b14e-41b13d7301da req-c549ae63-d9bd-49c1-9e4e-b65a7955a9ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048995 Oct 15 20:56:26.822836 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1123/2237] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:27.037245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-\u20a1-1004989459", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:27.105354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:56:28.663987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:28.713689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:28.892189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:56:28.893498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:56:28.894138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:56:28.894698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:56:28.895285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:56:28.909671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:28.910489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:28.915634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:28.917490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:28.918130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:28.918762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:28.992920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:56:29.011175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:56:29.044046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:56:29.047013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:56:29.153331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35695b34-4eac-4504-ba01-825f08ca227e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.328903 Oct 15 20:56:29.156691 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1124/2238] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 2333 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:56:29.171033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:29.171672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:29.181308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.187264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.187875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:29.188653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.189158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.196193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:29.279868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.280361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.280805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:29.281289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lazy-loading 'fault' on Instance uuid bec91401-323d-4a36-88b7-88fe24aa813f {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:56:29.393552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:29.403757 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3957feb-beee-45af-a51a-4f1b53a429d0 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 200 len: 1324 microversion: 2.1 time: 0.245860 Oct 15 20:56:29.408747 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1125/2239] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:29 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 1324 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:29.450971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:29.451821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:29.473419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.473928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.477449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:29.481258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.483604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.484068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:29.510135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:29.928646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:29.940878 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-240e5776-fe0d-4d57-a8ad-666772812160 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.518225 Oct 15 20:56:29.942320 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1126/2240] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:29 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:29.958759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:29.963160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:29.972333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.973341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.974575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:29.976039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:29.976708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:29.977470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:30.367726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:30.379338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b18eb53-ba65-4e8b-a97f-e76308f26ce6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1314 microversion: 2.1 time: 0.432346 Oct 15 20:56:30.380014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1127/2241] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:29 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1314 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:30.396916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5772cb18-7157-46dc-a44e-77f2e481bdf0 req-53135077-3cbf-4cbf-a5cd-f82eaf0f31df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bec91401-323d-4a36-88b7-88fe24aa813f", "name": "network-vif-deleted", "tag": "e75505e8-a2ad-4084-9f6f-4d72bd372f6f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:30.410069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5772cb18-7157-46dc-a44e-77f2e481bdf0 req-53135077-3cbf-4cbf-a5cd-f82eaf0f31df service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:30.411273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5772cb18-7157-46dc-a44e-77f2e481bdf0 req-53135077-3cbf-4cbf-a5cd-f82eaf0f31df service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:30.412211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5772cb18-7157-46dc-a44e-77f2e481bdf0 req-53135077-3cbf-4cbf-a5cd-f82eaf0f31df service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:30.427852 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5772cb18-7157-46dc-a44e-77f2e481bdf0 req-53135077-3cbf-4cbf-a5cd-f82eaf0f31df service nova] Creating event network-vif-deleted:e75505e8-a2ad-4084-9f6f-4d72bd372f6f for instance bec91401-323d-4a36-88b7-88fe24aa813f on nch1-master-1 Oct 15 20:56:30.433928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5772cb18-7157-46dc-a44e-77f2e481bdf0 req-53135077-3cbf-4cbf-a5cd-f82eaf0f31df service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052426 Oct 15 20:56:30.434915 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1128/2242] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:30.458627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:30.461496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:30.478933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:30.479822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:30.482239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:30.484777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:30.487655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:30.488209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:30.617846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:30.631230 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ceb76405-a1fa-46d9-a20a-02060116df16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1189 microversion: 2.1 time: 0.192339 Oct 15 20:56:30.632549 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1129/2243] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:30 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1189 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:30.744660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8646a26-fe67-4e39-85b2-e154da3e5cf6 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Action: 'create', calling method: .version_select at 0x7fe754418c10>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:34.129139 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8646a26-fe67-4e39-85b2-e154da3e5cf6 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 3.494324 Oct 15 20:56:34.139884 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1130/2244] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:56:30 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 3506 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:34.146313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:34.147827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: bec91401-323d-4a36-88b7-88fe24aa813f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:34.155538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.156302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.156933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.157954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.158756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.159446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.178240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] HTTP exception thrown: Instance bec91401-323d-4a36-88b7-88fe24aa813f could not be found. Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Request method failure captured: Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f HTTP/1.1 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzJVMNFSJ0p4PpFJ0U8dQawXH4cP1aPmp0QTTEGgjPUYVyqzTXGaFxiL9a2Au2uMiL8WJFXbsbXM7TakZbSXCacQ53u0dGyroIIoJ9NyVZeGgzPn9JEOEjdrOP1ZlEsoAysHiDKc4bwFsgV1m2jNjrQTCydGjEVr-d_5fjtsldzTuQ Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationTest-392689012 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c613201f55b94a528c6c36e5b5772799 Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance bec91401-323d-4a36-88b7-88fe24aa813f could not be found. Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'bec91401-323d-4a36-88b7-88fe24aa813f'} Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bec91401-323d-4a36-88b7-88fe24aa813f could not be found. Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bec91401-323d-4a36-88b7-88fe24aa813f could not be found. Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:34.181432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bec91401-323d-4a36-88b7-88fe24aa813f could not be found. Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Returning 404 to user: Instance bec91401-323d-4a36-88b7-88fe24aa813f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:34.184306 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e79a9772-ed59-4783-bff0-6993e2473b4d tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f" status: 404 len: 111 microversion: 2.1 time: 0.039893 Oct 15 20:56:34.187818 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1131/2245] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:34 2022] GET /compute/v2.1/servers/bec91401-323d-4a36-88b7-88fe24aa813f => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:34.196057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:34.196812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:34.208756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.209419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.209983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.210810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.211423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.212096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.245434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:34.408231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df5d20aa-f725-4515-b67a-4776eb8fd9ed tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 204 len: 0 microversion: 2.25 time: 0.217578 Oct 15 20:56:34.411968 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1132/2246] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:56:34 2022] DELETE /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:56:34.423310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:34.429458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:34.439267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=3,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:34.444540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.446695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.448548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.449628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.450348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.451554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.745298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:34.757648 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd233581-c330-4650-9417-da6e05ddfb39 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.342036 Oct 15 20:56:34.758940 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1133/2247] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:34 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:34.788318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:34.788318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:34.792047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.792564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.793006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:34.793763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:34.794260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:34.795775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:35.010490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:35.019089 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbc9044e-b7a4-4700-840f-d7c82c76f3b6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1314 microversion: 2.1 time: 0.243051 Oct 15 20:56:35.020536 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1134/2248] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:34 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:35.026183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:35.027009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:35.034340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:35.035014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:35.035887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:35.037020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:35.037933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:35.038468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:35.325499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:35.336818 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9cb9151-a2b6-44d9-be63-76a0ad48bb9b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1293 microversion: 2.1 time: 0.314892 Oct 15 20:56:35.349541 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1135/2249] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:35 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1293 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:35.362727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b6c4b4db-634f-4c57-bf7b-b01630b67bbe req-eeb8bb56-ff23-4fb8-a54a-70aa3875378a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "170b74a2-0481-4eab-acfd-00157c8cb2c5", "name": "network-vif-plugged", "status": "completed", "tag": "a6059d08-ec01-401a-9918-7c062c85a913"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:35.374588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6c4b4db-634f-4c57-bf7b-b01630b67bbe req-eeb8bb56-ff23-4fb8-a54a-70aa3875378a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:35.375099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6c4b4db-634f-4c57-bf7b-b01630b67bbe req-eeb8bb56-ff23-4fb8-a54a-70aa3875378a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:35.375693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6c4b4db-634f-4c57-bf7b-b01630b67bbe req-eeb8bb56-ff23-4fb8-a54a-70aa3875378a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:35.392347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b6c4b4db-634f-4c57-bf7b-b01630b67bbe req-eeb8bb56-ff23-4fb8-a54a-70aa3875378a service nova] Creating event network-vif-plugged:a6059d08-ec01-401a-9918-7c062c85a913 for instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 on nch1-master-1 Oct 15 20:56:35.397797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b6c4b4db-634f-4c57-bf7b-b01630b67bbe req-eeb8bb56-ff23-4fb8-a54a-70aa3875378a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047169 Oct 15 20:56:35.398397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1136/2250] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:35.421534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-768a92f3-afdf-4d33-8e39-19bc35f1afb1 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Action: 'create', calling method: .version_select at 0x7fe75426f5e0>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:37.714396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-768a92f3-afdf-4d33-8e39-19bc35f1afb1 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.309973 Oct 15 20:56:37.715134 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1137/2251] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:56:35 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2316 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:37.720798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa0d0070-ebc0-4e12-bf17-cd6b3cbae9fb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '.version_select at 0x7fe7585789d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:37.743859 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa0d0070-ebc0-4e12-bf17-cd6b3cbae9fb tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-603506732" status: 202 len: 0 microversion: 2.1 time: 0.027418 Oct 15 20:56:37.744398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1138/2252] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 20:56:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-603506732 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:37.749863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:37.750793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:37.763782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:37.764626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:37.765255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:37.766275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:37.766903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:37.767478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:37.849171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:37.871534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:37.875525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:37.877552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c215bac-bb49-47bc-a568-03bd63b2ff23 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 200 len: 1664 microversion: 2.25 time: 0.131545 Oct 15 20:56:37.879026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1139/2253] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:56:37 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 1664 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:56:37.886048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1ac5bb63-58ed-4eee-a8a5-9fe4577f1809 req-f7b207c2-01ee-4241-915c-3efde5e01d6d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "836a0d11-3560-4b69-bc0c-a3cec35c04fe", "tag": "a0e4b881-c9f1-4d0d-8f2f-355cc9bbdd08"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:37.904051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1ac5bb63-58ed-4eee-a8a5-9fe4577f1809 req-f7b207c2-01ee-4241-915c-3efde5e01d6d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:37.905199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1ac5bb63-58ed-4eee-a8a5-9fe4577f1809 req-f7b207c2-01ee-4241-915c-3efde5e01d6d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:37.906133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1ac5bb63-58ed-4eee-a8a5-9fe4577f1809 req-f7b207c2-01ee-4241-915c-3efde5e01d6d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:37.929945 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1ac5bb63-58ed-4eee-a8a5-9fe4577f1809 req-f7b207c2-01ee-4241-915c-3efde5e01d6d service nova] Creating event network-changed:a0e4b881-c9f1-4d0d-8f2f-355cc9bbdd08 for instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe on nch2-master-1 Oct 15 20:56:37.961585 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1ac5bb63-58ed-4eee-a8a5-9fe4577f1809 req-f7b207c2-01ee-4241-915c-3efde5e01d6d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073489 Oct 15 20:56:37.962649 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1140/2254] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:37.970885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:37.972864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:37.980675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:37.981971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:37.984041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:37.986816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:37.987705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:37.988279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:38.360628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:38.369415 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c5798a9-d1a8-4fcc-97fe-ae62ff6dab1b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.404998 Oct 15 20:56:38.379132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1141/2255] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:37 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:38.384365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:38.397352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:38.406085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:38.406681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:38.407124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:38.408201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:38.408665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:38.417124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:38.644383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:38.652292 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c33f02e2-7b49-4e5d-9327-ed32df816c82 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1501 microversion: 2.1 time: 0.271902 Oct 15 20:56:38.652859 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1142/2256] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:38 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1501 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:38.659562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:38.660724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:38.670844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:38.671652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:38.672341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:38.673569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:38.674275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:38.674630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:38.938752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:38.954737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:38.964964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40a59e13-e1b3-4d2b-87f8-ddf01a137619 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1293 microversion: 2.1 time: 0.310772 Oct 15 20:56:38.965581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1143/2257] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:38 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1293 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:38.977339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a369af1-c44f-49ad-9426-76307ab14aa3 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Action: 'create', calling method: .version_select at 0x7fe74fa98700>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:39.036642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:40.623822 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a369af1-c44f-49ad-9426-76307ab14aa3 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 1.650228 Oct 15 20:56:40.625710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1144/2258] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:56:38 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 1653 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:40.631214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:40.631944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:40.640094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:40.640823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:40.641534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:40.642469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:40.643108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:40.643725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:40.741140 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6933eee9-bbb4-40b3-b3b2-a7bc8eaba5b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5/action" status: 202 len: 0 microversion: 2.1 time: 0.113898 Oct 15 20:56:40.742322 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1145/2259] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:56:40 2022] POST /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5/action => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:40.751838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:40.753015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: c05e4f55-245d-45f2-8567-e665ccbca061] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:40.765469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:40.767009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:40.767917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:40.768960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:40.769615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:40.770285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:40.824255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] HTTP exception thrown: Instance c05e4f55-245d-45f2-8567-e665ccbca061 could not be found. Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Request method failure captured: Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 HTTP/1.1 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0Kqx1cRlReI4Q6UB0TefoyxU7QaxtINUi3VGPF44_vM_iAolJVYnlTCKaKCO5EDsPJrDn0_NXtibruZ4WghiUIobr4OQ0ZwvuwCfAa6p7mv6OqL8GVlqMttN_zTuIcNuN-gKrFE213p_6mVxqMY_YR-Vpg4wS5DsxLkn6FHJOEnnA Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.25 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f400a79cb647438f968a895142c5410d Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f400a79cb647438f968a895142c5410d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f400a79cb647438f968a895142c5410d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f400a79cb647438f968a895142c5410d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f400a79cb647438f968a895142c5410d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f400a79cb647438f968a895142c5410d Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveAutoBlockMigrationV225Test-778882528-project Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 180d9b7141ff4792ab7ab19e21a3d7ac Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-778882528-project Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c05e4f55-245d-45f2-8567-e665ccbca061 could not be found. Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c05e4f55-245d-45f2-8567-e665ccbca061'} Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c05e4f55-245d-45f2-8567-e665ccbca061 could not be found. Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c05e4f55-245d-45f2-8567-e665ccbca061 could not be found. Oct 15 20:56:40.830591 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c05e4f55-245d-45f2-8567-e665ccbca061 could not be found. Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Returning 404 to user: Instance c05e4f55-245d-45f2-8567-e665ccbca061 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:40.833794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4f763ea-7277-41f4-9116-10fd729431ac tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061" status: 404 len: 111 microversion: 2.25 time: 0.089070 Oct 15 20:56:40.836509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1146/2260] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:56:40 2022] GET /compute/v2.1/servers/c05e4f55-245d-45f2-8567-e665ccbca061 => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:56:40.843462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:40.844164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:40.855289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:40.856228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:40.856943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:40.859188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:40.860259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:40.865456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:40.902798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:40.921139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:41.052785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c16b941-f441-45bd-80bc-45cdd154d756 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 204 len: 0 microversion: 2.1 time: 0.214718 Oct 15 20:56:41.054954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1147/2261] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:40 2022] DELETE /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:41.069490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:41.071168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:41.079754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:41.080502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:41.081280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:41.082638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:41.083841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:41.084467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:41.292028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:41.302368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2dcd7db3-c9fd-49c4-b8a9-661e2e0e3be5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.244904 Oct 15 20:56:41.302967 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1148/2262] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:41 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:41.325851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:41.326745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:41.333586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:41.334488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:41.334941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:41.335660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:41.336304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:41.336737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:41.513981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:41.527635 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01498cfd-7f9c-43db-954d-e0df64445844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1293 microversion: 2.1 time: 0.213020 Oct 15 20:56:41.528481 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1149/2263] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:41 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1293 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:41.533583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8e93f401-5d2a-4f27-bcc5-bb8244f1aeb0 req-c1e5e9d6-54da-49cc-aa70-cdeadd8f95bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c05e4f55-245d-45f2-8567-e665ccbca061", "name": "network-vif-deleted", "tag": "0e64f25a-9a1d-4631-8907-36be7c15d6f3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:41.550757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8e93f401-5d2a-4f27-bcc5-bb8244f1aeb0 req-c1e5e9d6-54da-49cc-aa70-cdeadd8f95bb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:41.551238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8e93f401-5d2a-4f27-bcc5-bb8244f1aeb0 req-c1e5e9d6-54da-49cc-aa70-cdeadd8f95bb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:41.551671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8e93f401-5d2a-4f27-bcc5-bb8244f1aeb0 req-c1e5e9d6-54da-49cc-aa70-cdeadd8f95bb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:41.569381 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8e93f401-5d2a-4f27-bcc5-bb8244f1aeb0 req-c1e5e9d6-54da-49cc-aa70-cdeadd8f95bb service nova] Creating event network-vif-deleted:0e64f25a-9a1d-4631-8907-36be7c15d6f3 for instance c05e4f55-245d-45f2-8567-e665ccbca061 on nch2-master-1 Oct 15 20:56:41.575368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8e93f401-5d2a-4f27-bcc5-bb8244f1aeb0 req-c1e5e9d6-54da-49cc-aa70-cdeadd8f95bb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045541 Oct 15 20:56:41.579211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1150/2264] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:41.585856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f48651c-9d1b-477e-ac06-d1d256e617d4 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Action: 'create', calling method: .version_select at 0x7fe754109d30>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:43.548412 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f48651c-9d1b-477e-ac06-d1d256e617d4 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 1.967749 Oct 15 20:56:43.549523 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1151/2265] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:56:41 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 1970 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:43.555059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:43.556126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:43.568515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:43.569611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:43.570124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:43.570920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:43.571464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:43.572110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:43.729315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:43.743810 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-29b97851-5be7-4b5b-aecd-498cf152caec tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1486 microversion: 2.1 time: 0.193163 Oct 15 20:56:43.745285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1152/2266] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:43 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1486 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:43.749780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6e6d1df-f5b4-415d-a199-4f1b100457f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '.version_select at 0x7fe74f865310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:43.772672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6e6d1df-f5b4-415d-a199-4f1b100457f5 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1078380761" status: 204 len: 0 microversion: 2.25 time: 0.026495 Oct 15 20:56:43.773244 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1153/2267] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 20:56:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1078380761 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:56:43.777808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:43.778553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: ee567153-18c3-4c9b-a3eb-a9c13b250fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:43.785874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:43.787064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:43.787516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:43.788246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:43.788714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:43.789177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:43.805317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] HTTP exception thrown: Instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 could not be found. Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Request method failure captured: Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 HTTP/1.1 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzJVMNFSJ0p4PpFJ0U8dQawXH4cP1aPmp0QTTEGgjPUYVyqzTXGaFxiL9a2Au2uMiL8WJFXbsbXM7TakZbSXCacQ53u0dGyroIIoJ9NyVZeGgzPn9JEOEjdrOP1ZlEsoAysHiDKc4bwFsgV1m2jNjrQTCydGjEVr-d_5fjtsldzTuQ Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationTest-392689012 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c613201f55b94a528c6c36e5b5772799 Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 could not be found. Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ee567153-18c3-4c9b-a3eb-a9c13b250fe3'} Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 could not be found. Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 could not be found. Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:43.807123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 could not be found. Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Returning 404 to user: Instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eae7854d-1726-4bee-bbe1-5904b2c02923 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3" status: 404 len: 111 microversion: 2.1 time: 0.034395 Oct 15 20:56:43.813470 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1154/2268] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:43 2022] GET /compute/v2.1/servers/ee567153-18c3-4c9b-a3eb-a9c13b250fe3 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:43.948764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-212916a9-ae09-4357-a970-13808ad47d6e tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Action: 'create', calling method: .version_select at 0x7fe7584dba60>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:45.525406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-212916a9-ae09-4357-a970-13808ad47d6e tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 1.714314 Oct 15 20:56:45.526144 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1155/2269] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:56:43 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 1716 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:45.531471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:45.532086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:45.539962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:45.540455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:45.540897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:45.541674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:45.542115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:45.542547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:45.671623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:45.679403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c123ddf3-77c6-4342-a176-debd36639407 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1498 microversion: 2.1 time: 0.152229 Oct 15 20:56:45.679972 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1156/2270] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:45 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1498 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:45.684374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e9311a0c-1cab-468d-bd6f-aaff0182ae07 req-83cc10e5-3b2f-4b79-a9af-fb36ce135cc8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "836a0d11-3560-4b69-bc0c-a3cec35c04fe", "name": "network-vif-plugged", "status": "completed", "tag": "a0e4b881-c9f1-4d0d-8f2f-355cc9bbdd08"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:45.693611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e9311a0c-1cab-468d-bd6f-aaff0182ae07 req-83cc10e5-3b2f-4b79-a9af-fb36ce135cc8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:45.694091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e9311a0c-1cab-468d-bd6f-aaff0182ae07 req-83cc10e5-3b2f-4b79-a9af-fb36ce135cc8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:45.694556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e9311a0c-1cab-468d-bd6f-aaff0182ae07 req-83cc10e5-3b2f-4b79-a9af-fb36ce135cc8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:45.706242 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e9311a0c-1cab-468d-bd6f-aaff0182ae07 req-83cc10e5-3b2f-4b79-a9af-fb36ce135cc8 service nova] Creating event network-vif-plugged:a0e4b881-c9f1-4d0d-8f2f-355cc9bbdd08 for instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe on nch2-master-1 Oct 15 20:56:45.711034 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e9311a0c-1cab-468d-bd6f-aaff0182ae07 req-83cc10e5-3b2f-4b79-a9af-fb36ce135cc8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029913 Oct 15 20:56:45.711628 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1157/2271] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:45.717268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:45.717968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:45.724087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:45.724572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:45.725011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:45.725743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:45.726209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:45.726637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:45.862807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:45.870840 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c6da192-dcd3-483a-a2d1-b63cdb055760 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1293 microversion: 2.1 time: 0.157981 Oct 15 20:56:45.871348 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1158/2272] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:45 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1293 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:45.876120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9a2b666-126c-4d0d-9d99-21e495ef5e6e tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe74fc17d30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:46.212730 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9a2b666-126c-4d0d-9d99-21e495ef5e6e tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/e1d2bdb2-fd53-4a4f-8337-6ed95a784b06" status: 200 len: 141 microversion: 2.1 time: 0.340308 Oct 15 20:56:46.213473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1159/2273] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:56:45 2022] GET /compute/v2.1/os-floating-ips/e1d2bdb2-fd53-4a4f-8337-6ed95a784b06 => generated 141 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:46.218657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4751813a-162f-4d4f-aaef-c965f6e64b44 req-2928fd42-db61-44b5-b832-32be7e0ba8e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ee567153-18c3-4c9b-a3eb-a9c13b250fe3", "name": "network-vif-deleted", "tag": "0ac06a96-2831-4954-a5e4-8d904f1d0eb6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:46.230010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4751813a-162f-4d4f-aaef-c965f6e64b44 req-2928fd42-db61-44b5-b832-32be7e0ba8e5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.230665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4751813a-162f-4d4f-aaef-c965f6e64b44 req-2928fd42-db61-44b5-b832-32be7e0ba8e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.231192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4751813a-162f-4d4f-aaef-c965f6e64b44 req-2928fd42-db61-44b5-b832-32be7e0ba8e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.242884 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4751813a-162f-4d4f-aaef-c965f6e64b44 req-2928fd42-db61-44b5-b832-32be7e0ba8e5 service nova] Creating event network-vif-deleted:0ac06a96-2831-4954-a5e4-8d904f1d0eb6 for instance ee567153-18c3-4c9b-a3eb-a9c13b250fe3 on nch2-master-1 Oct 15 20:56:46.248967 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4751813a-162f-4d4f-aaef-c965f6e64b44 req-2928fd42-db61-44b5-b832-32be7e0ba8e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033071 Oct 15 20:56:46.250082 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1160/2274] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:46.255982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:46.256623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:46.262988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.264243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.264851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.265645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.266167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.266686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.296855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:46.388766 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32726756-7bbe-4860-95f6-03cba96c0e90 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 204 len: 0 microversion: 2.1 time: 0.136670 Oct 15 20:56:46.389573 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1161/2275] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:46 2022] DELETE /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:46.395102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:46.395695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:46.401689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.402172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.402610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.403310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.404330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.404764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.432953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:46.508036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b41a91a6-8cc7-493e-83a4-5716ec7ce8e9 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 204 len: 0 microversion: 2.1 time: 0.116858 Oct 15 20:56:46.508790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1162/2276] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:46 2022] DELETE /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:46.514900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:46.515488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:46.521347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.521830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.522267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.522958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:46.523413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:46.523837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:46.549931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:46.623925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0ac27f36-00c5-4f8c-9374-50fffb887969 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 204 len: 0 microversion: 2.25 time: 0.113757 Oct 15 20:56:46.624495 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1163/2277] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:56:46 2022] DELETE /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:56:46.629661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d79c8fb-bd9d-49b5-a20b-8aaf5ef1c36b tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Calling method '.version_select at 0x7fe75418be50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:46.793249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d79c8fb-bd9d-49b5-a20b-8aaf5ef1c36b tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/29740c57-07cf-4a40-b25b-4116c43135c7" status: 200 len: 141 microversion: 2.1 time: 0.167375 Oct 15 20:56:46.793859 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1164/2278] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:56:46 2022] GET /compute/v2.1/os-floating-ips/29740c57-07cf-4a40-b25b-4116c43135c7 => generated 141 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:46.798566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3faf5216-d0ed-4b22-9abd-ed5ec9963e6d tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe75850f670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:47.567368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3faf5216-d0ed-4b22-9abd-ed5ec9963e6d tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/e1d2bdb2-fd53-4a4f-8337-6ed95a784b06" status: 202 len: 0 microversion: 2.1 time: 0.772311 Oct 15 20:56:47.570285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1165/2279] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:56:46 2022] DELETE /compute/v2.1/os-floating-ips/e1d2bdb2-fd53-4a4f-8337-6ed95a784b06 => generated 0 bytes in 776 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:47.575194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:47.575985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:47.582485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:47.582984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:47.583432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:47.584146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:47.584624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:47.585074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:47.600352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:47.807979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:47.821072 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d69c522d-b4dc-4b60-a81f-828d3642f486 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 200 len: 1492 microversion: 2.1 time: 0.249221 Oct 15 20:56:47.843542 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1166/2280] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:47 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 1492 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:47.850708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:47.851776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:47.863653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:47.863653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:47.863653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:47.864119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:47.864869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:47.865451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:48.262824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c613201f55b94a528c6c36e5b5772799', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe14ede74b2140269cd7dfc32fe14f9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:48.273337 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de99f118-8f50-4542-8ca8-b718b6a9c4ae tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 200 len: 1500 microversion: 2.1 time: 0.428543 Oct 15 20:56:48.274150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1167/2281] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:47 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 1500 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:48.278608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:48.279257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:48.285153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:48.285645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:48.286071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:48.286774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:48.287230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:48.287657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:48.930698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:48.940500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:48.942066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '180d9b7141ff4792ab7ab19e21a3d7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f400a79cb647438f968a895142c5410d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:48.943231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-65ce59bd-7cba-444d-b163-df64f554b390 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 200 len: 1548 microversion: 2.25 time: 0.668369 Oct 15 20:56:48.943809 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1168/2282] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:56:48 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 1548 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:56:48.948292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:48.948904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:48.956306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:48.956804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:48.957277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:48.958057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:48.958523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:48.958955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:49.228908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:49.238932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-988f42d7-21a2-427f-b23e-8c2b6d711b86 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 200 len: 1475 microversion: 2.1 time: 0.293977 Oct 15 20:56:49.257433 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1169/2283] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:48 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 1475 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:49.264630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05434535-ed48-4b3e-b080-7c888c747ede tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Calling method '.version_select at 0x7fe74fc17280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:49.592186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-05434535-ed48-4b3e-b080-7c888c747ede tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips" status: 200 len: 144 microversion: 2.1 time: 0.333512 Oct 15 20:56:49.593016 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1170/2284] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:56:49 2022] GET /compute/v2.1/os-floating-ips => generated 144 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:49.598758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:49.599540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:49.608857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:49.609428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:49.609947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:49.610671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:49.611133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:49.611566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:49.758883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:49.767070 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c4903d1-1299-4a82-9dfb-604f17d786d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1474 microversion: 2.1 time: 0.172698 Oct 15 20:56:49.767632 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1171/2285] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:49 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1474 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:49.771918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-902cb1a9-29d2-4087-a767-1c4e260421b8 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe7543a8280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:49.983948 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-902cb1a9-29d2-4087-a767-1c4e260421b8 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ip-pools" status: 200 len: 43 microversion: 2.1 time: 0.215074 Oct 15 20:56:49.985737 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1172/2286] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Oct 15 20:56:49 2022] GET /compute/v2.1/os-floating-ip-pools => generated 43 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:56:49.991746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d7506fdb-1112-40d5-84d1-fe537f5447a6 req-b59bfefa-c624-4b64-ba24-be70a4e31329 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "170b74a2-0481-4eab-acfd-00157c8cb2c5", "name": "network-vif-deleted", "tag": "a6059d08-ec01-401a-9918-7c062c85a913"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:50.005907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d7506fdb-1112-40d5-84d1-fe537f5447a6 req-b59bfefa-c624-4b64-ba24-be70a4e31329 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.006719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d7506fdb-1112-40d5-84d1-fe537f5447a6 req-b59bfefa-c624-4b64-ba24-be70a4e31329 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.007251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d7506fdb-1112-40d5-84d1-fe537f5447a6 req-b59bfefa-c624-4b64-ba24-be70a4e31329 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.020940 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d7506fdb-1112-40d5-84d1-fe537f5447a6 req-b59bfefa-c624-4b64-ba24-be70a4e31329 service nova] Creating event network-vif-deleted:a6059d08-ec01-401a-9918-7c062c85a913 for instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 on nch1-master-1 Oct 15 20:56:50.027122 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d7506fdb-1112-40d5-84d1-fe537f5447a6 req-b59bfefa-c624-4b64-ba24-be70a4e31329 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039553 Oct 15 20:56:50.027935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1173/2287] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:50.034767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:50.035408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:50.041912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.042501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.043102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.043879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.044984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.045468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.061131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 HTTP/1.1 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '170b74a2-0481-4eab-acfd-00157c8cb2c5'} Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:50.063579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57250157-34cb-43ea-9387-ea3b6b651874 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 404 len: 111 microversion: 2.1 time: 0.035131 Oct 15 20:56:50.067107 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1174/2288] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:50 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:50.070333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-500754032"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:50.072647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:50.080200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.080955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.081929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.083288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.083994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.084757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.304151 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-96123c45-38f1-4e92-aabe-f66fe811762d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da/action" status: 202 len: 0 microversion: 2.1 time: 0.237244 Oct 15 20:56:50.304795 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1175/2289] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:56:50 2022] POST /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da/action => generated 0 bytes in 239 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:56:50.309995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:50.311327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] [instance: 4d275e72-929a-469e-9cdd-c5b4c4968506] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:50.318037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.318532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.319079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.319711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.320319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.320596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.335666 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] HTTP exception thrown: Instance 4d275e72-929a-469e-9cdd-c5b4c4968506 could not be found. Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Request method failure captured: Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 HTTP/1.1 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSxzJVMNFSJ0p4PpFJ0U8dQawXH4cP1aPmp0QTTEGgjPUYVyqzTXGaFxiL9a2Au2uMiL8WJFXbsbXM7TakZbSXCacQ53u0dGyroIIoJ9NyVZeGgzPn9JEOEjdrOP1ZlEsoAysHiDKc4bwFsgV1m2jNjrQTCydGjEVr-d_5fjtsldzTuQ Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fe14ede74b2140269cd7dfc32fe14f9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationTest-392689012 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: fe14ede74b2140269cd7dfc32fe14f9c Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationTest-392689012 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c613201f55b94a528c6c36e5b5772799 Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationTest-392689012-project Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 4d275e72-929a-469e-9cdd-c5b4c4968506 could not be found. Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4d275e72-929a-469e-9cdd-c5b4c4968506'} Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d275e72-929a-469e-9cdd-c5b4c4968506 could not be found. Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d275e72-929a-469e-9cdd-c5b4c4968506 could not be found. Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:50.337495 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d275e72-929a-469e-9cdd-c5b4c4968506 could not be found. Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] Returning 404 to user: Instance 4d275e72-929a-469e-9cdd-c5b4c4968506 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d6fef7d-1588-4f63-85ee-0634fd448759 tempest-LiveMigrationTest-392689012 tempest-LiveMigrationTest-392689012-project] 10.222.0.16 "GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506" status: 404 len: 111 microversion: 2.1 time: 0.032865 Oct 15 20:56:50.340648 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1176/2290] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:50 2022] GET /compute/v2.1/servers/4d275e72-929a-469e-9cdd-c5b4c4968506 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:56:50.344451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-456a4748-25dc-4712-9d84-baa65e7527a3 req-c8e4be30-ff10-4744-901d-336bdea27aa3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d275e72-929a-469e-9cdd-c5b4c4968506", "name": "network-vif-deleted", "tag": "374f7499-815c-480d-9b88-199bfe2d2340"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:50.353995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-456a4748-25dc-4712-9d84-baa65e7527a3 req-c8e4be30-ff10-4744-901d-336bdea27aa3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:50.354486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-456a4748-25dc-4712-9d84-baa65e7527a3 req-c8e4be30-ff10-4744-901d-336bdea27aa3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:50.354972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-456a4748-25dc-4712-9d84-baa65e7527a3 req-c8e4be30-ff10-4744-901d-336bdea27aa3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:50.366991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-456a4748-25dc-4712-9d84-baa65e7527a3 req-c8e4be30-ff10-4744-901d-336bdea27aa3 service nova] Creating event network-vif-deleted:374f7499-815c-480d-9b88-199bfe2d2340 for instance 4d275e72-929a-469e-9cdd-c5b4c4968506 on nch2-master-1 Oct 15 20:56:50.371736 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-456a4748-25dc-4712-9d84-baa65e7527a3 req-c8e4be30-ff10-4744-901d-336bdea27aa3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031141 Oct 15 20:56:50.373297 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1177/2291] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:50.378499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b3ae682-fb54-4848-8ccf-019129501423 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Calling method '.version_select at 0x7fe75850f8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:51.613066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b3ae682-fb54-4848-8ccf-019129501423 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/29740c57-07cf-4a40-b25b-4116c43135c7" status: 202 len: 0 microversion: 2.1 time: 1.238544 Oct 15 20:56:51.617332 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1178/2292] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:56:50 2022] DELETE /compute/v2.1/os-floating-ips/29740c57-07cf-4a40-b25b-4116c43135c7 => generated 0 bytes in 1240 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:51.619135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1229041125", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:51.628018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:UPDATE=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:51.680792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:56:52.687675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:56:52.688215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:56:52.688566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:56:52.688974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:56:52.689415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:56:52.699722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:52.700212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:52.701285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:52.702082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:52.702545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:52.703001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:52.725580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:56:52.731232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:56:52.745254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:56:52.748693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:56:52.799003 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-edc48736-7599-45e2-a0b2-dcf53dbb8fef tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.183995 Oct 15 20:56:52.799650 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1179/2293] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 1185 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:56:52.804186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:52.804797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: d16a441b-8807-4eb9-932e-3c2db14bc720] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:52.812400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:52.813030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:52.813358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:52.814067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:52.815065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:52.815065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:52.829586 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] HTTP exception thrown: Instance d16a441b-8807-4eb9-932e-3c2db14bc720 could not be found. Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Request method failure captured: Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 HTTP/1.1 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0Kqx1cRlReI4Q6UB0TefoyxU7QaxtINUi3VGPF44_vM_iAolJVYnlTCKaKCO5EDsPJrDn0_NXtibruZ4WghiUIobr4OQ0ZwvuwCfAa6p7mv6OqL8GVlqMttN_zTuIcNuN-gKrFE213p_6mVxqMY_YR-Vpg4wS5DsxLkn6FHJOEnnA Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.25 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f400a79cb647438f968a895142c5410d Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f400a79cb647438f968a895142c5410d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f400a79cb647438f968a895142c5410d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f400a79cb647438f968a895142c5410d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f400a79cb647438f968a895142c5410d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f400a79cb647438f968a895142c5410d Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveAutoBlockMigrationV225Test-778882528-project Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 180d9b7141ff4792ab7ab19e21a3d7ac Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-778882528-project Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d16a441b-8807-4eb9-932e-3c2db14bc720 could not be found. Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd16a441b-8807-4eb9-932e-3c2db14bc720'} Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d16a441b-8807-4eb9-932e-3c2db14bc720 could not be found. Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d16a441b-8807-4eb9-932e-3c2db14bc720 could not be found. Oct 15 20:56:52.831564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d16a441b-8807-4eb9-932e-3c2db14bc720 could not be found. Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Returning 404 to user: Instance d16a441b-8807-4eb9-932e-3c2db14bc720 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:56:52.835381 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3281be5e-84e7-436d-987d-1a50018edf4a tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720" status: 404 len: 111 microversion: 2.25 time: 0.032859 Oct 15 20:56:52.837647 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1180/2294] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:56:52 2022] GET /compute/v2.1/servers/d16a441b-8807-4eb9-932e-3c2db14bc720 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:56:52.842989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef710377-2b33-471e-829d-2132bc94829b tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe74fc13c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:53.160905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef710377-2b33-471e-829d-2132bc94829b tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips" status: 200 len: 399 microversion: 2.1 time: 0.322126 Oct 15 20:56:53.166287 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1181/2295] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:56:52 2022] GET /compute/v2.1/os-floating-ips => generated 399 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:53.167337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1435720807", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:53.252408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:56:53.928944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:56:53.930004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:56:53.930485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:56:53.930893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:56:53.931458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:56:53.941661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:53.942161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:53.942631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:53.943376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:53.943874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:53.944318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:53.963046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:56:53.968988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:56:53.982458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:56:53.985275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:56:54.035284 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1906260-0418-422a-b86b-626a7fe77137 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.871999 Oct 15 20:56:54.036379 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1182/2296] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:56:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 874 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:56:54.041326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d98ca337-f845-4815-bf02-f34dea7ca2f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe754778dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:54.097831 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d98ca337-f845-4815-bf02-f34dea7ca2f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.060124 Oct 15 20:56:54.099073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1183/2297] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:54 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:54.104742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c26c460b-552e-46b7-a65c-e334b3562e71 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Action: 'create', calling method: .version_select at 0x7fe74f9ad3a0>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:57.359496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c26c460b-552e-46b7-a65c-e334b3562e71 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 3.257749 Oct 15 20:56:57.360706 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1184/2298] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:56:54 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 3261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:57.368748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6b0eff01-adc2-4f51-82b1-c4d1db2fed93 req-cba176e9-10ca-456f-a165-b2d9cd009e96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d16a441b-8807-4eb9-932e-3c2db14bc720", "name": "network-vif-deleted", "tag": "c3cd8e3f-84a3-431a-99bb-a4efc4f21579"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:57.382448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6b0eff01-adc2-4f51-82b1-c4d1db2fed93 req-cba176e9-10ca-456f-a165-b2d9cd009e96 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:57.383331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6b0eff01-adc2-4f51-82b1-c4d1db2fed93 req-cba176e9-10ca-456f-a165-b2d9cd009e96 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:57.383788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6b0eff01-adc2-4f51-82b1-c4d1db2fed93 req-cba176e9-10ca-456f-a165-b2d9cd009e96 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:57.397402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6b0eff01-adc2-4f51-82b1-c4d1db2fed93 req-cba176e9-10ca-456f-a165-b2d9cd009e96 service nova] Creating event network-vif-deleted:c3cd8e3f-84a3-431a-99bb-a4efc4f21579 for instance d16a441b-8807-4eb9-932e-3c2db14bc720 on nch2-master-1 Oct 15 20:56:57.403018 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6b0eff01-adc2-4f51-82b1-c4d1db2fed93 req-cba176e9-10ca-456f-a165-b2d9cd009e96 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040156 Oct 15 20:56:57.403986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1185/2299] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:57.409880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:57.410481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:57.417350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:57.417850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:57.418292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:57.419005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:57.419677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:57.420111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:57.448611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:57.543478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d028320b-1c26-4c26-92ae-490109fcebc1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 204 len: 0 microversion: 2.1 time: 0.137556 Oct 15 20:56:57.545018 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1186/2300] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:56:57 2022] DELETE /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:56:57.555947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:57.556926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:57.569675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:57.570330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:57.570892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:57.572040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:57.572770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:57.573500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:57.605253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:56:57.708991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7cd0ba9a-adac-4049-a7f3-bd51fa51d645 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "DELETE /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 204 len: 0 microversion: 2.25 time: 0.162227 Oct 15 20:56:57.711103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1187/2301] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:56:57 2022] DELETE /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:56:57.719382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bdddf9a-4212-4c12-aae1-281824ec08cd tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe74fb3b8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:59.213430 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9bdddf9a-4212-4c12-aae1-281824ec08cd tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/381dfdb9-c7d5-48da-a902-cc91dbb98389" status: 202 len: 0 microversion: 2.1 time: 1.500029 Oct 15 20:56:59.217198 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1188/2302] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:56:57 2022] DELETE /compute/v2.1/os-floating-ips/381dfdb9-c7d5-48da-a902-cc91dbb98389 => generated 0 bytes in 1506 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:56:59.225546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:59.227034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:56:59.256847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:59.258844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:59.260108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:59.262021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:59.265917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:59.266589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:59.542611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:56:59.555481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a6e644e-3e29-4074-8c46-24938eb97d89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1315 microversion: 2.1 time: 0.336770 Oct 15 20:56:59.573711 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1189/2303] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:56:59 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1315 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:59.590229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-303141b7-831c-4e36-8b70-68e6ec7dbfbf tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe754418c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:56:59.671229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-303141b7-831c-4e36-8b70-68e6ec7dbfbf tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.096228 Oct 15 20:56:59.680762 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1190/2304] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:56:59 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:56:59.686783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3d0c9f72-90e5-4ed6-9d22-4711cd1f2fac req-40e4b432-48b0-43e8-bba1-44bf4c1a4813 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "352695e9-705f-447b-b213-141c9512ce65", "tag": "ead09e8f-0550-4641-b47e-949943f91f0a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:59.698590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3d0c9f72-90e5-4ed6-9d22-4711cd1f2fac req-40e4b432-48b0-43e8-bba1-44bf4c1a4813 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:59.699406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3d0c9f72-90e5-4ed6-9d22-4711cd1f2fac req-40e4b432-48b0-43e8-bba1-44bf4c1a4813 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:59.705800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3d0c9f72-90e5-4ed6-9d22-4711cd1f2fac req-40e4b432-48b0-43e8-bba1-44bf4c1a4813 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:59.752978 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3d0c9f72-90e5-4ed6-9d22-4711cd1f2fac req-40e4b432-48b0-43e8-bba1-44bf4c1a4813 service nova] Creating event network-changed:ead09e8f-0550-4641-b47e-949943f91f0a for instance 352695e9-705f-447b-b213-141c9512ce65 on nch1-master-1 Oct 15 20:56:59.766017 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3d0c9f72-90e5-4ed6-9d22-4711cd1f2fac req-40e4b432-48b0-43e8-bba1-44bf4c1a4813 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083426 Oct 15 20:56:59.771666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1191/2305] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:59.786451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8623ebed-5122-47f8-bbd9-eed767c9c572 req-eaf0f187-2c39-4862-af59-6336542a313a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4da129cd-0c8c-4e46-bcc6-beb0550690da", "name": "network-vif-plugged", "status": "completed", "tag": "ab0aa7bc-7367-426b-a481-53223b9795b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:56:59.813416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8623ebed-5122-47f8-bbd9-eed767c9c572 req-eaf0f187-2c39-4862-af59-6336542a313a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:56:59.814453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8623ebed-5122-47f8-bbd9-eed767c9c572 req-eaf0f187-2c39-4862-af59-6336542a313a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:56:59.815167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8623ebed-5122-47f8-bbd9-eed767c9c572 req-eaf0f187-2c39-4862-af59-6336542a313a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:56:59.824009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:56:59.846543 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8623ebed-5122-47f8-bbd9-eed767c9c572 req-eaf0f187-2c39-4862-af59-6336542a313a service nova] Creating event network-vif-plugged:ab0aa7bc-7367-426b-a481-53223b9795b8 for instance 4da129cd-0c8c-4e46-bcc6-beb0550690da on nch2-master-1 Oct 15 20:56:59.862445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8623ebed-5122-47f8-bbd9-eed767c9c572 req-eaf0f187-2c39-4862-af59-6336542a313a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081311 Oct 15 20:56:59.863932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1192/2306] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:56:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:56:59.870358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ae504a1-b22f-4239-98ea-242d66f70175 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Calling method '.version_select at 0x7fe754418b80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:01.228314 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ae504a1-b22f-4239-98ea-242d66f70175 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878" status: 202 len: 0 microversion: 2.1 time: 1.362904 Oct 15 20:57:01.228930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1193/2307] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:56:59 2022] DELETE /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878 => generated 0 bytes in 1364 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:01.235282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:01.236402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:01.251484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:01.252260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:01.252869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:01.253861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:01.254474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:01.255063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:01.456031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:01.474849 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-816d87e5-3996-407d-85b0-7f4d5da671f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 200 len: 1309 microversion: 2.1 time: 0.244826 Oct 15 20:57:01.475443 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1194/2308] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:01 2022] GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 1309 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:01.488420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:01.489659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] [instance: 73402931-6f69-461a-84ae-e0e91c34b2e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:01.498455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:01.499306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:01.499926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:01.500833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:01.501499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:01.503762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:01.524355 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] HTTP exception thrown: Instance 73402931-6f69-461a-84ae-e0e91c34b2e5 could not be found. Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Request method failure captured: Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 HTTP/1.1 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx0Kqx1cRlReI4Q6UB0TefoyxU7QaxtINUi3VGPF44_vM_iAolJVYnlTCKaKCO5EDsPJrDn0_NXtibruZ4WghiUIobr4OQ0ZwvuwCfAa6p7mv6OqL8GVlqMttN_zTuIcNuN-gKrFE213p_6mVxqMY_YR-Vpg4wS5DsxLkn6FHJOEnnA Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.25 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f400a79cb647438f968a895142c5410d Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f400a79cb647438f968a895142c5410d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f400a79cb647438f968a895142c5410d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f400a79cb647438f968a895142c5410d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f400a79cb647438f968a895142c5410d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f400a79cb647438f968a895142c5410d Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-778882528 Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveAutoBlockMigrationV225Test-778882528-project Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 180d9b7141ff4792ab7ab19e21a3d7ac Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-778882528-project Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 73402931-6f69-461a-84ae-e0e91c34b2e5 could not be found. Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '73402931-6f69-461a-84ae-e0e91c34b2e5'} Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 73402931-6f69-461a-84ae-e0e91c34b2e5 could not be found. Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 73402931-6f69-461a-84ae-e0e91c34b2e5 could not be found. Oct 15 20:57:01.532201 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 73402931-6f69-461a-84ae-e0e91c34b2e5 could not be found. Oct 15 20:57:01.535617 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:01.538124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Returning 404 to user: Instance 73402931-6f69-461a-84ae-e0e91c34b2e5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:01.541289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-944c38d4-05d0-4a21-99e0-67d913495286 tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5" status: 404 len: 111 microversion: 2.25 time: 0.063561 Oct 15 20:57:01.547052 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1195/2309] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:57:01 2022] GET /compute/v2.1/servers/73402931-6f69-461a-84ae-e0e91c34b2e5 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:57:01.552087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:01.576937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:01.578314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Searching by: {'deleted': False, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:57:01.587012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:01.587012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:01.588979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:01.589815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:01.590297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:01.590740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:01.634301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:57:01.670968 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44c9c6a2-4972-45fe-8155-dd21b94c3383 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 340 microversion: 2.1 time: 0.122993 Oct 15 20:57:01.672416 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1196/2310] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:57:01 2022] GET /compute/v2.1/servers => generated 340 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:01.677551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efdc1e39-66ad-4a49-9e1b-66f7d3c98cd9 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe75416c040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:03.699715 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efdc1e39-66ad-4a49-9e1b-66f7d3c98cd9 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/6a03f829-d141-4202-899e-cdada7462b6e" status: 202 len: 0 microversion: 2.1 time: 2.025454 Oct 15 20:57:03.701214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1197/2311] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:57:01 2022] DELETE /compute/v2.1/os-floating-ips/6a03f829-d141-4202-899e-cdada7462b6e => generated 0 bytes in 2027 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:03.709812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-545fc8f1-03b8-48c0-bf9c-b42bfc8e1408 req-28d30781-de24-47c9-a823-e372f7da6beb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6e486821-32b0-426e-aa16-aefb86e003a0", "tag": "29188354-03b6-4df3-a00b-31e4cd21cb62"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:03.732811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-545fc8f1-03b8-48c0-bf9c-b42bfc8e1408 req-28d30781-de24-47c9-a823-e372f7da6beb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:03.735740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-545fc8f1-03b8-48c0-bf9c-b42bfc8e1408 req-28d30781-de24-47c9-a823-e372f7da6beb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:03.736713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-545fc8f1-03b8-48c0-bf9c-b42bfc8e1408 req-28d30781-de24-47c9-a823-e372f7da6beb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:03.752059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-545fc8f1-03b8-48c0-bf9c-b42bfc8e1408 req-28d30781-de24-47c9-a823-e372f7da6beb service nova] Creating event network-changed:29188354-03b6-4df3-a00b-31e4cd21cb62 for instance 6e486821-32b0-426e-aa16-aefb86e003a0 on nch1-master-1 Oct 15 20:57:03.757904 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-545fc8f1-03b8-48c0-bf9c-b42bfc8e1408 req-28d30781-de24-47c9-a823-e372f7da6beb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052028 Oct 15 20:57:03.758814 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1198/2312] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:03.767723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:03.768652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:03.776339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:03.777116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:03.777798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:03.779077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:03.779709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:03.780362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:03.995517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:04.403828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:04.416037 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-000be671-7864-4d85-93f3-b721e3e5a5e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1315 microversion: 2.1 time: 0.655739 Oct 15 20:57:04.416660 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1199/2313] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:03 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1315 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:04.430021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73bc9a49-751b-44ef-82dc-14c424d7f511 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe75846e280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:04.508181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73bc9a49-751b-44ef-82dc-14c424d7f511 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.081845 Oct 15 20:57:04.509029 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1200/2314] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:04 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:04.514938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7058eca4-65a4-428e-9590-3f0bfc83750d tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Calling method '.version_select at 0x7fe74f9addc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:04.610378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-7058eca4-65a4-428e-9590-3f0bfc83750d tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP 67001078-a957-4ad0-9aba-db66e410a878 could not be found", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:57:04.611095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7058eca4-65a4-428e-9590-3f0bfc83750d tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] HTTP exception thrown: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7058eca4-65a4-428e-9590-3f0bfc83750d tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Request method failure captured: Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878 HTTP/1.1 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6JoLN_tk79cNc6ukQZcuovnJvLZ8A8Dtm104LYVct8OcQwU9fWYPOXJiDdyFapq3NigjukU-zOreBUJwqCT6aWfj5CMzHsuTA9PuWuR6K0ZSitn7cQqgMj2ZZKv77YCsA5GhSR6Sct9_w7RhnBae1ipBw5FhYq4wahJrnBnuQVzwk Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 77dd9d0796b845a4b66069ed09b88f12 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsTestJSON-1012343994 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsTestJSON-1012343994 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 77dd9d0796b845a4b66069ed09b88f12 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsTestJSON-1012343994 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsTestJSON-1012343994-project Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1fe50f300f3d4fa295033618d73faa7e Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsTestJSON-1012343994-project Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '67001078-a957-4ad0-9aba-db66e410a878'} Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP 67001078-a957-4ad0-9aba-db66e410a878 could not be found Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-54bf60db-b6ed-4375-bbc0-eccda2bd7dcf'] Oct 15 20:57:04.649461 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 94, in show Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878. Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 97, in show Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:04.653603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7058eca4-65a4-428e-9590-3f0bfc83750d tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Returning 404 to user: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:04.655795 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7058eca4-65a4-428e-9590-3f0bfc83750d tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878" status: 404 len: 111 microversion: 2.1 time: 0.144282 Oct 15 20:57:04.659425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1201/2315] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:57:04 2022] GET /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878 => generated 111 bytes in 147 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:04.677570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:04.695378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:04.696930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Searching by: {'deleted': False, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:57:04.712161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:04.712830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:04.714170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:04.715581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:04.717616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:04.718204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:04.771658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:57:04.871915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:04.879944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:04.880682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:04.881329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:05.114978 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40c2f958-671d-4bde-8e62-28fd4355167d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 1712 microversion: 2.1 time: 0.453884 Oct 15 20:57:05.129064 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1202/2316] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:57:04 2022] GET /compute/v2.1/servers/detail => generated 1712 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:05.129064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a66d31df-4db9-4330-9607-e78ce945344e req-35fcd288-344e-4e42-aacd-c3506312bb2d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "73402931-6f69-461a-84ae-e0e91c34b2e5", "name": "network-vif-deleted", "tag": "998f23e5-d301-418e-aa99-242986c4578b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:05.146944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a66d31df-4db9-4330-9607-e78ce945344e req-35fcd288-344e-4e42-aacd-c3506312bb2d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:05.147532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a66d31df-4db9-4330-9607-e78ce945344e req-35fcd288-344e-4e42-aacd-c3506312bb2d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:05.147985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a66d31df-4db9-4330-9607-e78ce945344e req-35fcd288-344e-4e42-aacd-c3506312bb2d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:05.170389 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a66d31df-4db9-4330-9607-e78ce945344e req-35fcd288-344e-4e42-aacd-c3506312bb2d service nova] Creating event network-vif-deleted:998f23e5-d301-418e-aa99-242986c4578b for instance 73402931-6f69-461a-84ae-e0e91c34b2e5 on nch1-master-1 Oct 15 20:57:05.175873 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a66d31df-4db9-4330-9607-e78ce945344e req-35fcd288-344e-4e42-aacd-c3506312bb2d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058538 Oct 15 20:57:05.176983 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1203/2317] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:05.186136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-401e8a68-04d0-4b82-b5ca-22d24a6c815f req-26801b71-6ca8-435a-add4-a7451f83fa47 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "352695e9-705f-447b-b213-141c9512ce65", "name": "network-vif-plugged", "status": "completed", "tag": "ead09e8f-0550-4641-b47e-949943f91f0a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:05.211704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-401e8a68-04d0-4b82-b5ca-22d24a6c815f req-26801b71-6ca8-435a-add4-a7451f83fa47 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:05.212196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-401e8a68-04d0-4b82-b5ca-22d24a6c815f req-26801b71-6ca8-435a-add4-a7451f83fa47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:05.212629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-401e8a68-04d0-4b82-b5ca-22d24a6c815f req-26801b71-6ca8-435a-add4-a7451f83fa47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:05.227335 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-401e8a68-04d0-4b82-b5ca-22d24a6c815f req-26801b71-6ca8-435a-add4-a7451f83fa47 service nova] Creating event network-vif-plugged:ead09e8f-0550-4641-b47e-949943f91f0a for instance 352695e9-705f-447b-b213-141c9512ce65 on nch1-master-1 Oct 15 20:57:05.233631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-401e8a68-04d0-4b82-b5ca-22d24a6c815f req-26801b71-6ca8-435a-add4-a7451f83fa47 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054701 Oct 15 20:57:05.234721 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1204/2318] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:05.241903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:05.243142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:05.262284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:05.263901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:05.264727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:05.265681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:05.266294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:05.267011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:05.540197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:05.551911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a40627e7-13d5-4028-9a20-217fe4a94183 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 200 len: 1309 microversion: 2.1 time: 0.315764 Oct 15 20:57:05.571977 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1205/2319] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:05 2022] GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 1309 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:05.581682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9861cbd-e040-4881-af84-57324d5a2c33 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] Calling method '.version_select at 0x7fe74fa8f9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:07.673546 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9861cbd-e040-4881-af84-57324d5a2c33 tempest-FloatingIPDetailsTestJSON-1240664955 tempest-FloatingIPDetailsTestJSON-1240664955-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/43893157-3a05-40e3-811b-44f5a47a4b5a" status: 202 len: 0 microversion: 2.1 time: 2.099579 Oct 15 20:57:07.679109 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1206/2320] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:57:05 2022] DELETE /compute/v2.1/os-floating-ips/43893157-3a05-40e3-811b-44f5a47a4b5a => generated 0 bytes in 2106 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:07.684602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e28665ad-3f9a-4db4-9d95-02b723dabae1 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Calling method '.version_select at 0x7fe75850f670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:07.790710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-e28665ad-3f9a-4db4-9d95-02b723dabae1 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP 67001078-a957-4ad0-9aba-db66e410a878 could not be found", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:57:07.792498 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e28665ad-3f9a-4db4-9d95-02b723dabae1 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] HTTP exception thrown: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e28665ad-3f9a-4db4-9d95-02b723dabae1 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Request method failure captured: Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878 HTTP/1.1 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6JoLN_tk79cNc6ukQZcuovnJvLZ8A8Dtm104LYVct8OcQwU9fWYPOXJiDdyFapq3NigjukU-zOreBUJwqCT6aWfj5CMzHsuTA9PuWuR6K0ZSitn7cQqgMj2ZZKv77YCsA5GhSR6Sct9_w7RhnBae1ipBw5FhYq4wahJrnBnuQVzwk Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 77dd9d0796b845a4b66069ed09b88f12 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsTestJSON-1012343994 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/77dd9d0796b845a4b66069ed09b88f12"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsTestJSON-1012343994 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 77dd9d0796b845a4b66069ed09b88f12 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsTestJSON-1012343994 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsTestJSON-1012343994-project Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1fe50f300f3d4fa295033618d73faa7e Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsTestJSON-1012343994-project Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '67001078-a957-4ad0-9aba-db66e410a878'} Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP 67001078-a957-4ad0-9aba-db66e410a878 could not be found Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-4b89bb45-d940-421e-89a1-f68264000e58'] Oct 15 20:57:07.795217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 158, in delete Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878. Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 161, in delete Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 Oct 15 20:57:07.798876 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:07.805741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e28665ad-3f9a-4db4-9d95-02b723dabae1 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] Returning 404 to user: Floating IP not found for ID 67001078-a957-4ad0-9aba-db66e410a878 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:07.806879 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e28665ad-3f9a-4db4-9d95-02b723dabae1 tempest-FloatingIPsTestJSON-1012343994 tempest-FloatingIPsTestJSON-1012343994-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878" status: 404 len: 111 microversion: 2.1 time: 0.126266 Oct 15 20:57:07.807731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1207/2321] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:57:07 2022] DELETE /compute/v2.1/os-floating-ips/67001078-a957-4ad0-9aba-db66e410a878 => generated 111 bytes in 128 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:07.827771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a082bcd9-96af-4a08-bb36-bf4e01785126 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:07.864541 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a082bcd9-96af-4a08-bb36-bf4e01785126 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.053940 Oct 15 20:57:07.875018 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1208/2322] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:57:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:07.881199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:07.881908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:07.890392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:07.891154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:07.891898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:07.893120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:07.893781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:07.894420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:08.878619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:08.895932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-029e4fc3-8dc9-4d05-9d2a-9b010c4ec6fa tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1315 microversion: 2.1 time: 1.018572 Oct 15 20:57:08.897254 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1209/2323] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:07 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1315 bytes in 1021 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:08.902225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ffec2ca-9967-4d3a-8d93-e52fb96e1761 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7544185e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:08.953175 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ffec2ca-9967-4d3a-8d93-e52fb96e1761 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.054312 Oct 15 20:57:08.954729 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1210/2324] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:08 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:08.959831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:08.960706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:08.973659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:08.974417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:08.974876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:08.975804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:08.976289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:08.976732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:09.075216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:09.100039 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d7102c7f-bdc3-4be9-a8a9-68e87e50d535 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 200 len: 1288 microversion: 2.1 time: 0.142671 Oct 15 20:57:09.103607 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1211/2325] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:08 2022] GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 1288 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:09.110472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4efd8318-e112-49c4-83e9-94e4536f756d req-4c3a434a-39b4-41f7-be26-f6183286edaa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6e486821-32b0-426e-aa16-aefb86e003a0", "name": "network-vif-plugged", "status": "completed", "tag": "29188354-03b6-4df3-a00b-31e4cd21cb62"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:09.121458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4efd8318-e112-49c4-83e9-94e4536f756d req-4c3a434a-39b4-41f7-be26-f6183286edaa service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:09.122178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4efd8318-e112-49c4-83e9-94e4536f756d req-4c3a434a-39b4-41f7-be26-f6183286edaa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:09.123371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4efd8318-e112-49c4-83e9-94e4536f756d req-4c3a434a-39b4-41f7-be26-f6183286edaa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:09.142119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4efd8318-e112-49c4-83e9-94e4536f756d req-4c3a434a-39b4-41f7-be26-f6183286edaa service nova] Creating event network-vif-plugged:29188354-03b6-4df3-a00b-31e4cd21cb62 for instance 6e486821-32b0-426e-aa16-aefb86e003a0 on nch1-master-1 Oct 15 20:57:09.151169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4efd8318-e112-49c4-83e9-94e4536f756d req-4c3a434a-39b4-41f7-be26-f6183286edaa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044888 Oct 15 20:57:09.152825 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1212/2326] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:09.163130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-adafae66-9562-462d-bbef-d5c88695067d req-1433462e-7174-4c79-b0d8-5a421592e492 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "352695e9-705f-447b-b213-141c9512ce65", "name": "network-vif-deleted", "tag": "ead09e8f-0550-4641-b47e-949943f91f0a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:09.179294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-adafae66-9562-462d-bbef-d5c88695067d req-1433462e-7174-4c79-b0d8-5a421592e492 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:09.180346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-adafae66-9562-462d-bbef-d5c88695067d req-1433462e-7174-4c79-b0d8-5a421592e492 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:09.180677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-adafae66-9562-462d-bbef-d5c88695067d req-1433462e-7174-4c79-b0d8-5a421592e492 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:09.197749 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adafae66-9562-462d-bbef-d5c88695067d req-1433462e-7174-4c79-b0d8-5a421592e492 service nova] Creating event network-vif-deleted:ead09e8f-0550-4641-b47e-949943f91f0a for instance 352695e9-705f-447b-b213-141c9512ce65 on nch1-master-1 Oct 15 20:57:09.203965 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-adafae66-9562-462d-bbef-d5c88695067d req-1433462e-7174-4c79-b0d8-5a421592e492 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048829 Oct 15 20:57:09.204682 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1213/2327] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:09.586347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ec4c004-6b56-4841-8178-cef658ebe4ce tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] Action: 'create', calling method: .version_select at 0x7fe74fa9fa60>, body: {"security_group": {"name": "tempest-securitygroup-968651075", "description": "tempest-description-1257634494"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:10.517183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ec4c004-6b56-4841-8178-cef658ebe4ce tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 220 microversion: 2.1 time: 1.310814 Oct 15 20:57:10.517903 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1214/2328] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:57:09 2022] POST /compute/v2.1/os-security-groups => generated 220 bytes in 1313 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:10.522995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:10.523695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:10.540913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:10.541446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:10.544981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:10.546330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:10.546821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:10.547753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:10.581096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:57:10.735876 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21821b26-2108-4e77-a4f3-e240cb06083d tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 204 len: 0 microversion: 2.1 time: 0.216575 Oct 15 20:57:10.736861 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1215/2329] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:57:10 2022] DELETE /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:57:10.766630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:10.770497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:10.785331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:10.788050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:10.788587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:10.789365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:10.795979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:10.799642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:11.101293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:11.112205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35698881-ed2b-44d7-bb08-a7568e8d3988 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1315 microversion: 2.1 time: 0.370457 Oct 15 20:57:11.118437 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1216/2330] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:10 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1315 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:11.150301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ded67bd-760f-4c25-ba7e-9c85c7a99ec8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe75533da60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:11.292776 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ded67bd-760f-4c25-ba7e-9c85c7a99ec8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.161752 Oct 15 20:57:11.293630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1217/2331] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:11 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:11.309910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:11.314204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:11.338617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:11.340795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:11.344299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:11.347986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:11.350056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:11.352013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:11.405453 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 HTTP/1.1 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '352695e9-705f-447b-b213-141c9512ce65'} Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:11.409594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 20:57:11.412852 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:11.417147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:11.421623 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-299db983-5d2a-415a-b7fd-a3b656d10e20 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 404 len: 111 microversion: 2.1 time: 0.125632 Oct 15 20:57:11.432956 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1218/2332] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:11 2022] GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 111 bytes in 138 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:11.448007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27fd4bec-2e8d-4e7e-a325-a9dc33b3a195 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] Action: 'create', calling method: .version_select at 0x7fe74fa9fee0>, body: {"security_group": {"name": "tempest-securitygroup-64231851", "description": "tempest-description-1298024187"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:11.848551 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27fd4bec-2e8d-4e7e-a325-a9dc33b3a195 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 219 microversion: 2.1 time: 0.412721 Oct 15 20:57:11.849623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1219/2333] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:57:11 2022] POST /compute/v2.1/os-security-groups => generated 219 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:11.854267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:11.855041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:11.874630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=6,nova_api:UPDATE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:11.879845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:11.883496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:11.887196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:11.888582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:11.890804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:11.891356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:12.540279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:12.573354 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53106290-8ea8-44cd-ae51-89733eb8c88e tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1715 microversion: 2.1 time: 0.722531 Oct 15 20:57:12.587878 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1220/2334] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:11 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1715 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:12.603754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1255933696", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "accessIPv6": "2001:2001::3", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:12.719613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:14.746689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:14.868207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:15.001176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:15.001867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:15.002460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:15.002981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:15.003504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:15.028872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:15.034220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:15.035111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:15.036150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:15.036901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:15.038252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:15.062920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:15.082375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:15.129927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:15.132736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:15.237832 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b9d9e9e-b05f-4080-a356-0ea2b887817a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.648389 Oct 15 20:57:15.239977 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1221/2335] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:12 2022] POST /compute/v2.1/servers => generated 380 bytes in 2651 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:15.459104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-23d2c659-1d8e-4f1c-ba60-b4a259333b31 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Action: 'create', calling method: .version_select at 0x7fe74fc09310>, body: {"security_group": {"name": "tempest-securitygroup-515886035", "description": "tempest-description-1993553814"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:16.959195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-23d2c659-1d8e-4f1c-ba60-b4a259333b31 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 220 microversion: 2.1 time: 1.717359 Oct 15 20:57:16.960859 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1222/2336] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:57:15 2022] POST /compute/v2.1/os-security-groups => generated 220 bytes in 1720 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:16.965817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:16.967531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:17.008647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:17.009165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:17.009613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:17.010331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:17.023157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:17.031112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:17.339751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:17.348255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73809073-593a-4788-929b-abd742e4a2e1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1502 microversion: 2.1 time: 0.386006 Oct 15 20:57:17.357538 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1223/2337] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:16 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1502 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:17.369408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a331c61c-7a76-412c-8f61-caa16861014b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74fefc280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:17.442919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a331c61c-7a76-412c-8f61-caa16861014b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.083867 Oct 15 20:57:17.443522 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1224/2338] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:17 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:17.467743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:17.471270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:17.490837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:17.491904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:17.492881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:17.494274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:17.494999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:17.495750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:17.513819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:17.559798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:17.563128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:17.564138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:17.565861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lazy-loading 'fault' on Instance uuid 8e20e4c9-1939-425a-a7a8-d284327057bc {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:57:17.908300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'de4d5d0008324d7a807a8f8361a60a4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a0d3787e2a842029da9fb7d0a6715b0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:17.925906 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b819e52d-cfd9-4d89-a723-2a561a4ef160 tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 200 len: 1367 microversion: 2.1 time: 0.481320 Oct 15 20:57:17.927208 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1225/2339] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:17 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 1367 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:17.932298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b5835b63-d4a8-4eba-bb90-3aa0ec383ccc req-3cbcdff1-ad0d-452b-a654-8c3362b73f1d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e20e4c9-1939-425a-a7a8-d284327057bc", "name": "network-vif-unplugged", "status": "completed", "tag": "5d6409b1-05df-4a6a-8356-24ea8f253baa"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:17.954945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5835b63-d4a8-4eba-bb90-3aa0ec383ccc req-3cbcdff1-ad0d-452b-a654-8c3362b73f1d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:17.955681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5835b63-d4a8-4eba-bb90-3aa0ec383ccc req-3cbcdff1-ad0d-452b-a654-8c3362b73f1d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:17.956296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5835b63-d4a8-4eba-bb90-3aa0ec383ccc req-3cbcdff1-ad0d-452b-a654-8c3362b73f1d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:18.002512 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b5835b63-d4a8-4eba-bb90-3aa0ec383ccc req-3cbcdff1-ad0d-452b-a654-8c3362b73f1d service nova] Creating event network-vif-unplugged:5d6409b1-05df-4a6a-8356-24ea8f253baa for instance 8e20e4c9-1939-425a-a7a8-d284327057bc on nch2-master-1 Oct 15 20:57:18.016657 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b5835b63-d4a8-4eba-bb90-3aa0ec383ccc req-3cbcdff1-ad0d-452b-a654-8c3362b73f1d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.086909 Oct 15 20:57:18.023370 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1226/2340] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:17 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:18.035869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:18.036833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:18.061101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:18.062150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:18.062767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:18.063913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:18.064532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:18.065806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:18.208678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:18.232092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37617b5f-5381-4707-ae64-1994a7a836a3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1279 microversion: 2.1 time: 0.206837 Oct 15 20:57:18.246153 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1227/2341] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:18 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1279 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:18.260469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1be8605a-e344-40ea-b14b-97a85bc23851 req-902ab747-2583-441b-9936-c0a2526782c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e20e4c9-1939-425a-a7a8-d284327057bc", "name": "network-vif-deleted", "tag": "5d6409b1-05df-4a6a-8356-24ea8f253baa"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:18.279345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1be8605a-e344-40ea-b14b-97a85bc23851 req-902ab747-2583-441b-9936-c0a2526782c6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:18.279892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1be8605a-e344-40ea-b14b-97a85bc23851 req-902ab747-2583-441b-9936-c0a2526782c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:18.280338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1be8605a-e344-40ea-b14b-97a85bc23851 req-902ab747-2583-441b-9936-c0a2526782c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:18.302309 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1be8605a-e344-40ea-b14b-97a85bc23851 req-902ab747-2583-441b-9936-c0a2526782c6 service nova] Creating event network-vif-deleted:5d6409b1-05df-4a6a-8356-24ea8f253baa for instance 8e20e4c9-1939-425a-a7a8-d284327057bc on nch2-master-1 Oct 15 20:57:18.311566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1be8605a-e344-40ea-b14b-97a85bc23851 req-902ab747-2583-441b-9936-c0a2526782c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063829 Oct 15 20:57:18.312599 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1228/2342] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:18.327333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0dce79d-00ff-44ca-8590-b4d807d2b213 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Action: 'create', calling method: .version_select at 0x7fe74fb51f70>, body: {"security_group": {"name": "tempest-securitygroup-241268713", "description": "tempest-description-298780784"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:19.121187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0dce79d-00ff-44ca-8590-b4d807d2b213 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 219 microversion: 2.1 time: 0.806792 Oct 15 20:57:19.123657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1229/2343] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 20:57:18 2022] POST /compute/v2.1/os-security-groups => generated 219 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:19.129517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:19.130491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:19.185271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:19.193864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:19.194506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:19.195377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:19.198895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:19.202006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:19.313486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lazy-loading 'flavor' on Instance uuid 6e486821-32b0-426e-aa16-aefb86e003a0 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:57:19.736943 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-856fff0c-5bc1-41a8-a5fe-66563f4b59c4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0/action" status: 202 len: 0 microversion: 2.1 time: 0.611400 Oct 15 20:57:19.737826 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1230/2344] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:57:19 2022] POST /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0/action => generated 0 bytes in 613 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:19.763355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d6b6618-1bb7-44cc-b715-9b50e9992055 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74fa9c1f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:19.887207 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d6b6618-1bb7-44cc-b715-9b50e9992055 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1413 microversion: 2.1 time: 0.135696 Oct 15 20:57:19.899164 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1231/2345] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:19 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1413 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:19.918700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:19.921682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] [instance: 8e20e4c9-1939-425a-a7a8-d284327057bc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:19.947029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:19.952542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:19.954087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:19.960381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:19.964142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:19.965469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:20.019184 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] HTTP exception thrown: Instance 8e20e4c9-1939-425a-a7a8-d284327057bc could not be found. Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Request method failure captured: Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc HTTP/1.1 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4n7xlkdq8s1EnhRmzlV08Es3koGMM-hwYpBSAplt-4FDU5nV3ng32-pI3mYusKzRqnxL69LW7uq2YUadjILLxD5vcrsKG6Be4c_DIYdyMWmboH6-EqzEqiVbVPHfjoTrXxVFsFhOluJVpzz4ty50n9SY7XM-A_6e63DNCuQ_XVB9w Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 0a0d3787e2a842029da9fb7d0a6715b0 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestManualDisk-830689221 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0a0d3787e2a842029da9fb7d0a6715b0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0a0d3787e2a842029da9fb7d0a6715b0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0a0d3787e2a842029da9fb7d0a6715b0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0a0d3787e2a842029da9fb7d0a6715b0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestManualDisk-830689221 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 0a0d3787e2a842029da9fb7d0a6715b0 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestManualDisk-830689221 Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestManualDisk-830689221-project Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: de4d5d0008324d7a807a8f8361a60a4e Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestManualDisk-830689221-project Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8e20e4c9-1939-425a-a7a8-d284327057bc could not be found. Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8e20e4c9-1939-425a-a7a8-d284327057bc'} Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e20e4c9-1939-425a-a7a8-d284327057bc could not be found. Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e20e4c9-1939-425a-a7a8-d284327057bc could not be found. Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:20.029653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e20e4c9-1939-425a-a7a8-d284327057bc could not be found. Oct 15 20:57:20.033406 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:20.037229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Returning 404 to user: Instance 8e20e4c9-1939-425a-a7a8-d284327057bc could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:20.040085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a947383e-55aa-4933-94a4-52f892f28dfc tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc" status: 404 len: 111 microversion: 2.1 time: 0.138473 Oct 15 20:57:20.045149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1232/2346] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:19 2022] GET /compute/v2.1/servers/8e20e4c9-1939-425a-a7a8-d284327057bc => generated 111 bytes in 144 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:20.063107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Calling method '.version_select at 0x7fe74fb07ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:21.073749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 07d82454-9c28-4b02-9f07-a2775a1a27d6 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:57:21.074848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Neutron security group 07d82454-9c28-4b02-9f07-a2775a1a27d6 not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:57:21.076929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.security_groups [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Security Group ID 07d82454-9c28-4b02-9f07-a2775a1a27d6 does not exist {{(pid=138688) _format_security_group_rule /opt/stack/nova/nova/api/openstack/compute/security_groups.py:75}} Oct 15 20:57:21.107844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 07d82454-9c28-4b02-9f07-a2775a1a27d6 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:57:21.109097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Neutron security group 07d82454-9c28-4b02-9f07-a2775a1a27d6 not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:57:21.109695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.security_groups [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Security Group ID 07d82454-9c28-4b02-9f07-a2775a1a27d6 does not exist {{(pid=138688) _format_security_group_rule /opt/stack/nova/nova/api/openstack/compute/security_groups.py:75}} Oct 15 20:57:23.467664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group df381234-c681-4691-868c-22eb8fe1328e does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:57:23.469742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Neutron security group df381234-c681-4691-868c-22eb8fe1328e not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:57:23.470307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.security_groups [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Security Group ID df381234-c681-4691-868c-22eb8fe1328e does not exist {{(pid=138688) _format_security_group_rule /opt/stack/nova/nova/api/openstack/compute/security_groups.py:75}} Oct 15 20:57:23.530173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group df381234-c681-4691-868c-22eb8fe1328e does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:57:23.531225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Neutron security group df381234-c681-4691-868c-22eb8fe1328e not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:57:23.535250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.security_groups [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Security Group ID df381234-c681-4691-868c-22eb8fe1328e does not exist {{(pid=138688) _format_security_group_rule /opt/stack/nova/nova/api/openstack/compute/security_groups.py:75}} Oct 15 20:57:23.766028 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0ec00c32-3ab7-4d22-8c65-0937a4cd09d4 tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] 10.222.0.16 "GET /compute/v2.1/os-security-groups?all_tenants=true" status: 200 len: 8649 microversion: 2.1 time: 3.714566 Oct 15 20:57:23.781605 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1233/2347] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Oct 15 20:57:20 2022] GET /compute/v2.1/os-security-groups?all_tenants=true => generated 8649 bytes in 3735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:23.788335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:23.789199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:23.796115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:23.796833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:23.797500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:23.798362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:23.798969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:23.799544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:24.403272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:24.413490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-300a1e5c-f752-4116-8ad6-51a65500b37f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1321 microversion: 2.1 time: 0.630567 Oct 15 20:57:24.414087 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1234/2348] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:23 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1321 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:24.424147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:24.424834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:24.431428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:24.432138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:24.432593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:24.433336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:24.433956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:24.434402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:24.743800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:24.758414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86dea88b-1be9-4fa5-8d85-80e8516cdec7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1508 microversion: 2.1 time: 0.339748 Oct 15 20:57:24.760030 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1235/2349] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:24 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1508 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:24.767785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca9dd838-c7da-4667-b4ff-ca8eb44ef23a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] Calling method '.version_select at 0x7fe74faa0c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:24.816360 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca9dd838-c7da-4667-b4ff-ca8eb44ef23a tempest-ServersTestManualDisk-830689221 tempest-ServersTestManualDisk-830689221-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-681472305" status: 202 len: 0 microversion: 2.1 time: 0.054822 Oct 15 20:57:24.817553 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1236/2350] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 20:57:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-681472305 => generated 0 bytes in 57 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:24.828230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4aa44933-3139-476e-8e8f-5b59c99958a4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f9eb820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:24.897572 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4aa44933-3139-476e-8e8f-5b59c99958a4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1422 microversion: 2.1 time: 0.078695 Oct 15 20:57:24.898534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1237/2351] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:24 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1422 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:25.052457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:25.118513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:25.232989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a2634ae-8a13-45c5-9124-4aa2d9b9c637 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74f9adb80>, body: {"keypair": {"name": "key_/.\\@:", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKn0IoJB5a9Ae4mPNVWWpfww/HEbtzbqVbLfimY/vRgar8boHjNyRX1x1Q8gwy76UzIouOh4HxXN2yq6hW5gsMxsovbNTjkZ8pM/hOTSlpmnlHHL/mWvUXJQKjxQb9nTYQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a2634ae-8a13-45c5-9124-4aa2d9b9c637 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 257 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "key_/.\\@:", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKn0IoJB5a9Ae4mPNVWWpfww/HEbtzbqVbLfimY/vRgar8boHjNyRX1x1Q8gwy76UzIouOh4HxXN2yq6hW5gsMxsovbNTjkZ8pM/hOTSlpmnlHHL/mWvUXJQKjxQb9nTYQ=="}} Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: exception: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': 'key_/.\\@:', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKn0IoJB5a9Ae4mPNVWWpfww/HEbtzbqVbLfimY/vRgar8boHjNyRX1x1Q8gwy76UzIouOh4HxXN2yq6hW5gsMxsovbNTjkZ8pM/hOTSlpmnlHHL/mWvUXJQKjxQb9nTYQ=='}}} Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 263, in check Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = func(instance) Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 161, in _validate_keypair_name_20 Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _validate_keypair_name(keypair_name, safe_chars) Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 175, in _validate_keypair_name Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidName(reason=reason) Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidName: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'key_/.\\@:' is not a 'keypair_name_20' Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['keypair']['properties']['name']['allOf'][1]: Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'keypair_name_20', Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'maxLength': 255, Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minLength': 1, Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'} Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['keypair']['name']: Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'key_/.\\@:' Oct 15 20:57:25.237311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Oct 15 20:57:25.241491 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:25.244483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a2634ae-8a13-45c5-9124-4aa2d9b9c637 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 400 to user: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:25.248383 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a2634ae-8a13-45c5-9124-4aa2d9b9c637 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 193 microversion: 2.1 time: 0.345818 Oct 15 20:57:25.248961 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1238/2352] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:24 2022] POST /compute/v2.1/os-keypairs => generated 193 bytes in 350 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:57:25.260595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2c03314e-1de6-4c08-a63d-b9ee133fc7ac req-b823c43f-8ce7-438c-9fdd-f82611e7f56a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d38ead8c-d325-4fda-8df9-acff8948f988", "tag": "e0ac614b-b366-4d72-b72d-eb57eb12f440"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:25.286106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2c03314e-1de6-4c08-a63d-b9ee133fc7ac req-b823c43f-8ce7-438c-9fdd-f82611e7f56a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:25.289864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2c03314e-1de6-4c08-a63d-b9ee133fc7ac req-b823c43f-8ce7-438c-9fdd-f82611e7f56a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:25.290320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2c03314e-1de6-4c08-a63d-b9ee133fc7ac req-b823c43f-8ce7-438c-9fdd-f82611e7f56a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:25.319684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2c03314e-1de6-4c08-a63d-b9ee133fc7ac req-b823c43f-8ce7-438c-9fdd-f82611e7f56a service nova] Creating event network-changed:e0ac614b-b366-4d72-b72d-eb57eb12f440 for instance d38ead8c-d325-4fda-8df9-acff8948f988 on nch2-master-1 Oct 15 20:57:25.327405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2c03314e-1de6-4c08-a63d-b9ee133fc7ac req-b823c43f-8ce7-438c-9fdd-f82611e7f56a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074385 Oct 15 20:57:25.328016 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1239/2353] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:25.333510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-edb58673-8d38-49ac-9d79-350e3a95efe4 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] Calling method '.version_select at 0x7fe74faa08b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:26.056626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-edb58673-8d38-49ac-9d79-350e3a95efe4 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups?all_tenants=true" status: 200 len: 1110 microversion: 2.1 time: 0.726230 Oct 15 20:57:26.066335 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1240/2354] 10.222.0.16 () {58 vars in 1191 bytes} [Sat Oct 15 20:57:25 2022] GET /compute/v2.1/os-security-groups?all_tenants=true => generated 1110 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:26.072757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f76bcde4-cbd4-4b69-8118-b0809df1d0b2 req-6d33a8cb-417b-4f6b-ae39-746c213cdcdc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4da129cd-0c8c-4e46-bcc6-beb0550690da", "name": "network-vif-unplugged", "status": "completed", "tag": "ab0aa7bc-7367-426b-a481-53223b9795b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:26.084761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f76bcde4-cbd4-4b69-8118-b0809df1d0b2 req-6d33a8cb-417b-4f6b-ae39-746c213cdcdc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:26.085489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f76bcde4-cbd4-4b69-8118-b0809df1d0b2 req-6d33a8cb-417b-4f6b-ae39-746c213cdcdc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:26.086083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f76bcde4-cbd4-4b69-8118-b0809df1d0b2 req-6d33a8cb-417b-4f6b-ae39-746c213cdcdc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:26.098208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f76bcde4-cbd4-4b69-8118-b0809df1d0b2 req-6d33a8cb-417b-4f6b-ae39-746c213cdcdc service nova] Creating event network-vif-unplugged:ab0aa7bc-7367-426b-a481-53223b9795b8 for instance 4da129cd-0c8c-4e46-bcc6-beb0550690da on nch2-master-1 Oct 15 20:57:26.113362 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f76bcde4-cbd4-4b69-8118-b0809df1d0b2 req-6d33a8cb-417b-4f6b-ae39-746c213cdcdc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045411 Oct 15 20:57:26.116895 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1241/2355] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:26 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:26.128994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c7c853e-c3e4-466e-b0ca-1d0a4cad509c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74fa12040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:26.179376 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c7c853e-c3e4-466e-b0ca-1d0a4cad509c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 200 len: 1422 microversion: 2.1 time: 0.061136 Oct 15 20:57:26.180166 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1242/2356] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:57:26 2022] GET /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 1422 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:26.186641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74f9ad280>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-105606389", "public_key": "ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:26.225443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:26.231173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for user 353e4db513a94d96bd82b85742624fc2 and project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:26.253347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 2166 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-105606389", "public_key": "ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu"}} Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:26.277168 ncdv-master-1 devstack@n-api.service[138688]: exception: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-105606389', 'public_key': 'ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu'}}} Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 666, in load_ssh_public_key Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid key format") Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ValueError: Invalid key format Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6774, in import_key_pair Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6847, in _generate_fingerprint Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:26.280193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create(req, body) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 130, in _create Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:26.282456 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:26.284257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:26.285340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f176ebd-0585-4e60-b12f-f36b928a6bdc tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.103992 Oct 15 20:57:26.286447 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1243/2357] 10.222.0.16 () {60 vars in 1160 bytes} [Sat Oct 15 20:57:26 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 105 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:26.291531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:26.292495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:26.304121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:26.304625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:26.305093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:26.306218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:26.306691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:26.307125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:26.471153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:26.480928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b395532-b5c9-4faa-9573-1399b6c820df tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1321 microversion: 2.1 time: 0.193229 Oct 15 20:57:26.482368 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1244/2358] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:26 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1321 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:26.487498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:26.488240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:26.494638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:26.495465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:26.496216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:26.497223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:26.498531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:26.499069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:26.650584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:26.660514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0577269f-8015-4f36-8515-60e41b528535 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1508 microversion: 2.1 time: 0.176813 Oct 15 20:57:26.661566 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1245/2359] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:26 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1508 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:26.794299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationNegativeTest-server-765064388", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:26.861882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:27.096193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:27.096765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:27.097230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:27.097730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:27.098313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:27.110808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:27.111344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:27.111791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:27.112538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:27.113005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:27.113461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:27.134539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Getting quotas for project c44841db4e33412e805da46ba4262fe7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:27.140323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Getting quotas for user 500cba6141d34c4081d38a7b1be1775e and project c44841db4e33412e805da46ba4262fe7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:27.154737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:27.158689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:27.223419 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3c7c00d-4dfe-4376-8698-98d824525f34 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.559158 Oct 15 20:57:27.231263 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1246/2360] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 569 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:27.237462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d3e43e1-e960-441d-8e1f-6f4bd1daf67b tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Calling method '.version_select at 0x7fe74faa0280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:27.489897 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d3e43e1-e960-441d-8e1f-6f4bd1daf67b tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/952e6806-0767-48ea-8dec-818dc0527718" status: 202 len: 0 microversion: 2.1 time: 0.256558 Oct 15 20:57:27.491707 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1247/2361] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:57:27 2022] DELETE /compute/v2.1/os-security-groups/952e6806-0767-48ea-8dec-818dc0527718 => generated 0 bytes in 259 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:27.508835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26a1bed5-a140-4a48-8c29-08fcf21fc35d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74fa12670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:28.419089 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26a1bed5-a140-4a48-8c29-08fcf21fc35d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 204 len: 0 microversion: 2.1 time: 0.926025 Oct 15 20:57:28.420418 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1248/2362] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:27 2022] DELETE /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:57:28.427532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-93a46143-a1e5-4908-a419-9d7e357189d2 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74fa98280>, body: {"keypair": {"name": "tempest-keypair-1537670939", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD40sT+9VrYnWZGkLtylQm5d0BuM7iO4UDppogobCx1ttWsQsov+7HOYplFosFcfhxe761wqzjOiSrEJSMvjPE1YdZH7SO807QsRQ9VPqI88KIZB3CV0WAKdtbAHF179Uw=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:28.471237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-93a46143-a1e5-4908-a419-9d7e357189d2 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:28.478782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-93a46143-a1e5-4908-a419-9d7e357189d2 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for user 353e4db513a94d96bd82b85742624fc2 and project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:28.520046 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-93a46143-a1e5-4908-a419-9d7e357189d2 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.098294 Oct 15 20:57:28.521198 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1249/2363] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:28.543837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:28.546258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:28.553997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:28.554736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:28.555322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:28.556216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:28.556894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:28.557536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:28.645473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:28.656088 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7c6d866-b7cd-4a40-bcaf-777c8b25fe55 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1270 microversion: 2.1 time: 0.133424 Oct 15 20:57:28.673918 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1250/2364] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:28 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1270 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:28.678928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3afa85d9-794b-4254-948b-f48d1b5a6ede tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] Calling method '.version_select at 0x7fe754706f70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:29.162646 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3afa85d9-794b-4254-948b-f48d1b5a6ede tempest-SecurityGroupsTestAdminJSON-979153963 tempest-SecurityGroupsTestAdminJSON-979153963-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/1c69e3a7-59fc-4370-ae86-bb03f850981e" status: 202 len: 0 microversion: 2.1 time: 0.487161 Oct 15 20:57:29.166389 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1251/2365] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:57:28 2022] DELETE /compute/v2.1/os-security-groups/1c69e3a7-59fc-4370-ae86-bb03f850981e => generated 0 bytes in 492 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:29.172272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:29.173831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:29.189154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:29.193970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:29.194643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:29.204268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:29.215279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:29.217955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:29.588209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:29.600205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f371d39-dc72-4095-8436-7c2854b8249b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1321 microversion: 2.1 time: 0.429529 Oct 15 20:57:29.601079 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1252/2366] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:29 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1321 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:29.628454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:29.638475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:29.648809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:29.657576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:29.657928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:29.658871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:29.663728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:29.664121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:30.033707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:30.050005 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e3bd2e93-7fa2-4aa0-b6e5-169084eb7347 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1508 microversion: 2.1 time: 0.425106 Oct 15 20:57:30.050893 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1253/2367] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:29 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1508 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:30.063265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:30.063784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:30.084531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:30.085063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:30.089872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:30.091517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:30.092354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:30.094253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:30.168624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:57:30.314031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:30.371802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=7,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:30.375528 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b718ded7-bec2-4f0f-8783-7a8f07fe593a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 204 len: 0 microversion: 2.1 time: 0.323138 Oct 15 20:57:30.378708 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1254/2368] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:57:30 2022] DELETE /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 0 bytes in 327 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:57:30.392557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74f9adb80>, body: {"keypair": {"name": "tempest-keypair-1537670939", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM37xNVmwL5MQyFfkLDjrMcYQF1z2KXQoP6TX70SSzqlYzYcNav8QNMXnl+0rjxMIvzZ1SjjCOyNQawaUqmduXGpe+3sY5RBuKcj3Cam9pJzmH1k0AyUCPd9p5kS2u09rA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:30.436807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:30.444753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for user 353e4db513a94d96bd82b85742624fc2 and project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:30.492722 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] HTTP exception thrown: Key pair 'tempest-keypair-1537670939' already exists. Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 273 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "tempest-keypair-1537670939", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM37xNVmwL5MQyFfkLDjrMcYQF1z2KXQoP6TX70SSzqlYzYcNav8QNMXnl+0rjxMIvzZ1SjjCOyNQawaUqmduXGpe+3sY5RBuKcj3Cam9pJzmH1k0AyUCPd9p5kS2u09rA=="}} Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: exception: Key pair 'tempest-keypair-1537670939' already exists. Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': 'tempest-keypair-1537670939', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM37xNVmwL5MQyFfkLDjrMcYQF1z2KXQoP6TX70SSzqlYzYcNav8QNMXnl+0rjxMIvzZ1SjjCOyNQawaUqmduXGpe+3sY5RBuKcj3Cam9pJzmH1k0AyUCPd9p5kS2u09rA=='}}} Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = self._query(query) Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi conn.query(q) Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result.read() Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry '353e4db513a94d96bd82b85742624fc2-tempest-keypair-1537670939' for key 'key_pairs.uniq_key_pairs0user_id0name'") Oct 15 20:57:30.531777 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 69, in _create_in_db Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi kp.save(context.session) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi session.flush() Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._flush(objects) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi compat.raise_( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi flush_context.execute() Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi rec.execute(self) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = self._query(query) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi conn.query(q) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result.read() Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Oct 15 20:57:30.537334 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '353e4db513a94d96bd82b85742624fc2-tempest-keypair-1537670939' for key 'key_pairs.uniq_key_pairs0user_id0name'") Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO key_pairs (created_at, updated_at, name, user_id, fingerprint, public_key, type) VALUES (%(created_at)s, %(updated_at)s, %(name)s, %(user_id)s, %(fingerprint)s, %(public_key)s, %(type)s)] Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 10, 15, 20, 57, 30, 478728), 'updated_at': None, 'name': 'tempest-keypair-1537670939', 'user_id': '353e4db513a94d96bd82b85742624fc2', 'fingerprint': '20:b1:6d:ab:20:c9:e7:29:c5:82:49:1b:6c:03:04:bb', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM37xNVmwL5MQyFfkLDjrMcYQF1z2KXQoP6TX70SSzqlYzYcNav8QNMXnl+0rjxMIvzZ1SjjCOyNQawaUqmduXGpe+3sY5RBuKcj3Cam9pJzmH1k0AyUCPd9p5kS2u09rA==', 'type': 'ssh'}] Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6777, in import_key_pair Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi keypair.create() Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 158, in create Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._create() Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 162, in _create Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_keypair = self._create_in_db(self._context, updates) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 134, in _create_in_db Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _create_in_db(context, values) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 71, in _create_in_db Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.KeyPairExists(key_name=values['name']) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.KeyPairExists: Key pair 'tempest-keypair-1537670939' already exists. Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create(req, body) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 132, in _create Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=exc.format_message()) Oct 15 20:57:30.542525 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Key pair 'tempest-keypair-1537670939' already exists. Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Key pair 'tempest-keypair-1537670939' already exists. Oct 15 20:57:30.549310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:30.550274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 409 to user: Key pair 'tempest-keypair-1537670939' already exists. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:30.551245 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44cbcbc7-791b-41b4-b950-ae402f0d9c61 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 409 len: 105 microversion: 2.1 time: 0.171392 Oct 15 20:57:30.557092 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1255/2369] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:30 2022] POST /compute/v2.1/os-keypairs => generated 105 bytes in 173 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:57:30.560697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-49bdbda7-32cf-4e0a-bd6c-b7f49bd5baf8 req-331d4398-a6a2-4c50-b5db-4f516c1967e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d38ead8c-d325-4fda-8df9-acff8948f988", "name": "network-vif-plugged", "status": "completed", "tag": "e0ac614b-b366-4d72-b72d-eb57eb12f440"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:30.599714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-49bdbda7-32cf-4e0a-bd6c-b7f49bd5baf8 req-331d4398-a6a2-4c50-b5db-4f516c1967e7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:30.601502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-49bdbda7-32cf-4e0a-bd6c-b7f49bd5baf8 req-331d4398-a6a2-4c50-b5db-4f516c1967e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:30.603324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-49bdbda7-32cf-4e0a-bd6c-b7f49bd5baf8 req-331d4398-a6a2-4c50-b5db-4f516c1967e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:30.629752 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-49bdbda7-32cf-4e0a-bd6c-b7f49bd5baf8 req-331d4398-a6a2-4c50-b5db-4f516c1967e7 service nova] Creating event network-vif-plugged:e0ac614b-b366-4d72-b72d-eb57eb12f440 for instance d38ead8c-d325-4fda-8df9-acff8948f988 on nch2-master-1 Oct 15 20:57:30.642591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-49bdbda7-32cf-4e0a-bd6c-b7f49bd5baf8 req-331d4398-a6a2-4c50-b5db-4f516c1967e7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089157 Oct 15 20:57:30.659868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1256/2370] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:30.670377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddf1310a-98f8-4931-ae56-8b18fed70365 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] Calling method '.version_select at 0x7fe74f9a34c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:31.032009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddf1310a-98f8-4931-ae56-8b18fed70365 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/ca569938-c9c1-4cf4-a18a-b10950e42579" status: 202 len: 0 microversion: 2.1 time: 0.370145 Oct 15 20:57:31.032623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1257/2371] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:57:30 2022] DELETE /compute/v2.1/os-security-groups/ca569938-c9c1-4cf4-a18a-b10950e42579 => generated 0 bytes in 372 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:31.037189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:31.037904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:31.047682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:31.049433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:31.050079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:31.051182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:31.051910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:31.056792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:31.281697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:31.289833 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d0010e5-fcc9-4081-b04c-70ba13931c16 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 0.256103 Oct 15 20:57:31.291858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1258/2372] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:31 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:31.297944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da6082ce-1417-49e8-acc1-5dbaecfd75b5 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Calling method '.version_select at 0x7fe74faa0f70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:31.330057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da6082ce-1417-49e8-acc1-5dbaecfd75b5 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1537670939" status: 202 len: 0 microversion: 2.1 time: 0.036909 Oct 15 20:57:31.337194 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1259/2373] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 20:57:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1537670939 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:31.414137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationNegativeTestJSON-server-1707398566", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "76243f48-ac57-4169-afa3-4182fbe1350d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:31.481131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] validate_networks() for [('76243f48-ac57-4169-afa3-4182fbe1350d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:32.659730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:32.660282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:32.660700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:32.661177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:32.661603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:32.672787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:32.673509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:32.674111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:32.675256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:32.675731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:32.676163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:32.697838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Getting quotas for project 11bdafc438884621944db399da5528f8. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:32.704666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Getting quotas for user 78d679a10c9e4bf2bc35786235aeaf2a and project 11bdafc438884621944db399da5528f8. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:32.720093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:32.723274 ncdv-master-1 devstack@n-api.service[138688]: WARNING nova.utils [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Hostname tempest-FloatingIPsAssociationNegativeTestJSON-server-1707398566 is longer than 63, truncate it to tempest-FloatingIPsAssociationNegativeTestJSON-server-170739856 Oct 15 20:57:32.724576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:32.790656 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da9a4213-b019-4c43-a791-cf80534ad982 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.452086 Oct 15 20:57:32.791343 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1260/2374] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1453 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:32.804270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:32.805146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:32.815069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:32.815747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:32.816421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:32.817415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:32.818443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:32.819092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:33.187610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:33.198794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-34914086-6be6-4331-83e8-677bc3fbdd74 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1321 microversion: 2.1 time: 0.399533 Oct 15 20:57:33.202282 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1261/2375] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:32 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1321 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:33.212408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-1110355591", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c130cb51-e60d-43b7-af80-10e77fbe9595"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:33.282197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] validate_networks() for [('c130cb51-e60d-43b7-af80-10e77fbe9595', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:34.229713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:34.230248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:34.230606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:34.231020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:34.231435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:34.246629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:34.247630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:34.248151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:34.248914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:34.249402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:34.249843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:34.284865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:34.293835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for user 5d93a680570f4d23b1f6df534bc082b2 and project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:34.309408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:34.311980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:34.453967 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6fece08e-7d10-4a5d-96a1-3fd29248cd65 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.248907 Oct 15 20:57:34.459236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1262/2376] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:33 2022] POST /compute/v2.1/servers => generated 380 bytes in 1256 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:34.470871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f866c9cc-3764-4e3b-a782-5a0aac9dfe67 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] Calling method '.version_select at 0x7fe74fa12820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:35.166407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f866c9cc-3764-4e3b-a782-5a0aac9dfe67 tempest-SecurityGroupsTestAdminJSON-1963488644 tempest-SecurityGroupsTestAdminJSON-1963488644-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/ddc3f22a-0dc6-43da-a25d-b81f23c44941" status: 202 len: 0 microversion: 2.1 time: 0.705508 Oct 15 20:57:35.167975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1263/2377] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:57:34 2022] DELETE /compute/v2.1/os-security-groups/ddc3f22a-0dc6-43da-a25d-b81f23c44941 => generated 0 bytes in 708 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:57:35.174551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:35.175424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:35.190501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.192882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.195201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.196146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.199042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.199704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.496188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:35.519137 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d218a3a-7623-4d02-9bd4-e70256cd7ee4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1529 microversion: 2.1 time: 0.348483 Oct 15 20:57:35.521529 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1264/2378] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:35 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1529 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:35.528428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5701288e-5617-47c5-872e-a6df8b0a1305 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74f8bf5e0>, body: {"keypair": {"name": "", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH3OwSJUhiTX/QG67LSnXyc+ZqrO+PUYwSWicJu76gnEsWqB94gvH13u5hmC7ulcP1FZGVxX4Q+Le2cKoFeBeiKSI+LAkiMacRW7SuJyw44rSTlPpx0o7N+eFD9EqbEa0A=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5701288e-5617-47c5-872e-a6df8b0a1305 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 247 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH3OwSJUhiTX/QG67LSnXyc+ZqrO+PUYwSWicJu76gnEsWqB94gvH13u5hmC7ulcP1FZGVxX4Q+Le2cKoFeBeiKSI+LAkiMacRW7SuJyw44rSTlPpx0o7N+eFD9EqbEa0A=="}} Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: . '' is too short Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': '', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH3OwSJUhiTX/QG67LSnXyc+ZqrO+PUYwSWicJu76gnEsWqB94gvH13u5hmC7ulcP1FZGVxX4Q+Le2cKoFeBeiKSI+LAkiMacRW7SuJyw44rSTlPpx0o7N+eFD9EqbEa0A=='}}} Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['keypair']['properties']['name']['allOf'][0]: Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['keypair']: Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '' Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:57:35.532897 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Oct 15 20:57:35.537066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:35.540660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5701288e-5617-47c5-872e-a6df8b0a1305 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:35.541794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5701288e-5617-47c5-872e-a6df8b0a1305 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 108 microversion: 2.1 time: 0.018869 Oct 15 20:57:35.548496 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1265/2379] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:35 2022] POST /compute/v2.1/os-keypairs => generated 108 bytes in 26 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:57:35.558294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:35.559636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:35.579127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.579714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.580209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.581131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.581626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.582396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.746793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:35.759530 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83589074-4d00-41af-8780-89473e0e884a tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 0.209229 Oct 15 20:57:35.760783 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1266/2380] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:35 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:35.766649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-442f92f0-134e-476e-b050-7be347e6907c req-db165487-1b74-443c-b3c1-6d1f00a52554 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4da129cd-0c8c-4e46-bcc6-beb0550690da", "name": "network-vif-deleted", "tag": "ab0aa7bc-7367-426b-a481-53223b9795b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:35.776820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-442f92f0-134e-476e-b050-7be347e6907c req-db165487-1b74-443c-b3c1-6d1f00a52554 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.777570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-442f92f0-134e-476e-b050-7be347e6907c req-db165487-1b74-443c-b3c1-6d1f00a52554 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.778290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-442f92f0-134e-476e-b050-7be347e6907c req-db165487-1b74-443c-b3c1-6d1f00a52554 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.790476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-442f92f0-134e-476e-b050-7be347e6907c req-db165487-1b74-443c-b3c1-6d1f00a52554 service nova] Creating event network-vif-deleted:ab0aa7bc-7367-426b-a481-53223b9795b8 for instance 4da129cd-0c8c-4e46-bcc6-beb0550690da on nch2-master-1 Oct 15 20:57:35.796027 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-442f92f0-134e-476e-b050-7be347e6907c req-db165487-1b74-443c-b3c1-6d1f00a52554 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033709 Oct 15 20:57:35.797810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1267/2381] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:35.803226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:35.804230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:35.811727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.812504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.813140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.814102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.814777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.815444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.907120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:35.925724 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b942b0d-9a3a-4da2-9573-97b130f9c313 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1290 microversion: 2.1 time: 0.126429 Oct 15 20:57:35.926985 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1268/2382] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:35 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1290 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:35.932177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8fa14ddf-4d94-416d-bca1-855ac2bdb882 req-e0d38a48-ea61-4a9c-b4b4-86ec41b0d1dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a708d875-0764-45ee-9b65-d5c576abd402", "tag": "5570eac9-7f3b-4f14-8252-089d27fe947b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:35.942738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8fa14ddf-4d94-416d-bca1-855ac2bdb882 req-e0d38a48-ea61-4a9c-b4b4-86ec41b0d1dc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.943474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8fa14ddf-4d94-416d-bca1-855ac2bdb882 req-e0d38a48-ea61-4a9c-b4b4-86ec41b0d1dc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.944253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8fa14ddf-4d94-416d-bca1-855ac2bdb882 req-e0d38a48-ea61-4a9c-b4b4-86ec41b0d1dc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.957123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8fa14ddf-4d94-416d-bca1-855ac2bdb882 req-e0d38a48-ea61-4a9c-b4b4-86ec41b0d1dc service nova] Creating event network-changed:5570eac9-7f3b-4f14-8252-089d27fe947b for instance a708d875-0764-45ee-9b65-d5c576abd402 on nch2-master-1 Oct 15 20:57:35.962711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8fa14ddf-4d94-416d-bca1-855ac2bdb882 req-e0d38a48-ea61-4a9c-b4b4-86ec41b0d1dc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034356 Oct 15 20:57:35.963516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1269/2383] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:35.969625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:35.970588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:35.978278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.979000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.979591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:35.980453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:35.981087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:35.982206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.164246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:36.183427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78a97710-5c99-4e05-ad4a-4060cdad4141 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1503 microversion: 2.1 time: 0.218508 Oct 15 20:57:36.184937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1270/2384] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:35 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1503 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:36.204814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:36.206709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:36.224943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.226140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.228690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.230613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.232338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.234633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.336217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:36.353303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7626718-26f9-41c9-b885-712ccb4af0f3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1212 microversion: 2.1 time: 0.165626 Oct 15 20:57:36.370080 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1271/2385] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:36 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1212 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:36.380921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74fa3cb80>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-27263573", "public_key": " "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:36.398695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:36.413500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for user 353e4db513a94d96bd82b85742624fc2 and project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:36.435155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 93 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-27263573", "public_key": " "}} Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: exception: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-27263573', 'public_key': ' '}}} Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 650, in load_ssh_public_key Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid line format") Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ValueError: Invalid line format Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6774, in import_key_pair Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6847, in _generate_fingerprint Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:36.438672 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create(req, body) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 130, in _create Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:36.443475 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b97e68f-da26-4e81-be08-c111f7c1e350 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.067588 Oct 15 20:57:36.453582 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1272/2386] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Oct 15 20:57:36 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 82 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:36.460314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:36.461236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:36.474186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.474831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.475475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.476204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.476663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.477164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.651563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:36.660501 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c0c34d1f-5413-482e-8487-4d312127f657 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1503 microversion: 2.1 time: 0.205580 Oct 15 20:57:36.661097 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1273/2387] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:36 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1503 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:36.673885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9357f8d6-91fa-4d9b-a8d2-69671cea06eb tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe74f9eb280>, body: {"keypair": {"name": "keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHqlS9zI2U9Ma3edb0J+Rk6srGveZRKfwUkT+7wWgvY9aMiEXk/wMoSSqxTzDHjsy9pWw26RbuTbQ6/zMjq5GqPbnWznn87ni28MNfRbonxx0wxxmFy3an3dn25ZND6P2A=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9357f8d6-91fa-4d9b-a8d2-69671cea06eb tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 507 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHqlS9zI2U9Ma3edb0J+Rk6srGveZRKfwUkT+7wWgvY9aMiEXk/wMoSSqxTzDHjsy9pWw26RbuTbQ6/zMjq5GqPbnWznn87ni28MNfRbonxx0wxxmFy3an3dn25ZND6P2A=="}} Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHqlS9zI2U9Ma3edb0J+Rk6srGveZRKfwUkT+7wWgvY9aMiEXk/wMoSSqxTzDHjsy9pWw26RbuTbQ6/zMjq5GqPbnWznn87ni28MNfRbonxx0wxxmFy3an3dn25ZND6P2A=='}}} Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['keypair']['properties']['name']['allOf'][0]: Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['keypair']: Oct 15 20:57:36.679182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9357f8d6-91fa-4d9b-a8d2-69671cea06eb tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 400 to user: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9357f8d6-91fa-4d9b-a8d2-69671cea06eb tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 627 microversion: 2.1 time: 0.018564 Oct 15 20:57:36.682392 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1274/2388] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:36 2022] POST /compute/v2.1/os-keypairs => generated 627 bytes in 20 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:57:36.687926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:36.688725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:36.696380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.697120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.698102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.699022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.699638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.700146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.928007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:36.940594 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6639d1e1-7012-4b47-b6d3-c1a6eecab8b9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1529 microversion: 2.1 time: 0.257495 Oct 15 20:57:36.941976 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1275/2389] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:36 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1529 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:36.947375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:36.947998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:36.956385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.957191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.957729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:36.958676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:36.959240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:36.959783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:37.016031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:57:37.147318 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6054bfa4-09f1-4a91-93d8-e42e9a4f0628 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 204 len: 0 microversion: 2.1 time: 0.203709 Oct 15 20:57:37.149112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1276/2390] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:57:36 2022] DELETE /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:57:37.157612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Action: 'create', calling method: .version_select at 0x7fe758578700>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1365267053", "public_key": "ssh-rsa JUNK nova@ubuntu"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:37.181364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:37.200980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Getting quotas for user 353e4db513a94d96bd82b85742624fc2 and project 2e241b6abf12428ba07a6f052bc00c70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:37.221701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=7,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:37.233772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 118 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1365267053", "public_key": "ssh-rsa JUNK nova@ubuntu"}} Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: exception: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-1365267053', 'public_key': 'ssh-rsa JUNK nova@ubuntu'}}} Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 664, in load_ssh_public_key Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi inner_key_type, data = _get_sshstr(data) Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 134, in _get_sshstr Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi n, data = _get_u32(data) Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 121, in _get_u32 Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid data") Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ValueError: Invalid data Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6774, in import_key_pair Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6847, in _generate_fingerprint Oct 15 20:57:37.239977 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create(req, body) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 130, in _create Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:37.244622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:37.259324 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99da77e7-cbe1-47b3-9430-bddd39f3528e tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.101689 Oct 15 20:57:37.266099 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1277/2391] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:37 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 116 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 20:57:37.272012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:37.272998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:37.282763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:37.283255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:37.283694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:37.284455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:37.284912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:37.285708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:37.528086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:37.537738 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c4e1141-d0b5-4743-8410-aa15a04b1212 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 0.270047 Oct 15 20:57:37.538792 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1278/2392] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:37 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:37.547815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:37.547815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:37.552135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:37.552628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:37.553078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:37.553778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:37.554237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:37.554659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:37.742157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:37.751699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca69b332-8ca1-4e79-bef0-3030c664cdcf tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1332 microversion: 2.1 time: 0.211748 Oct 15 20:57:37.765062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1279/2393] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:37 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1332 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:37.777711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:37.779228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:37.790688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:37.791772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:37.792725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:37.795873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:37.796530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:37.798812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:38.049807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:38.059460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-840a2fe1-cf22-4963-9095-cfd6159081e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1509 microversion: 2.1 time: 0.301943 Oct 15 20:57:38.061440 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1280/2394] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:37 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1509 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:38.081230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b1e348b-6081-4af9-bdcb-b9a65779be49 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Calling method '.version_select at 0x7fe74fa3cca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:38.113254 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7b1e348b-6081-4af9-bdcb-b9a65779be49 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] HTTP exception thrown: Keypair tempest-keypair-non-existent-530151382 not found for user 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b1e348b-6081-4af9-bdcb-b9a65779be49 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Request method failure captured: Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-530151382 HTTP/1.1 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6xa5L3CBoT0A4rrOlOLQF9LsIN1KzAsCudrATy54_6p4cR_WBrxoW6qmvMZ6DDFkJakk8xO3RdOhopTEZx-GsnI5XBEGvsby6PQzi_3uPgl7S4XRO8hCR-jXtfkMggUU7qeUiqmZlsWU49biKXkRm68vXC8E9F0GpHFYYJftLYBuw Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e241b6abf12428ba07a6f052bc00c70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 2e241b6abf12428ba07a6f052bc00c70 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1412209103 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1412209103-project Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: exception: Keypair tempest-keypair-non-existent-530151382 not found for user 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'tempest-keypair-non-existent-530151382'} Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-non-existent-530151382 not found for user 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:38.115163 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req, id) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-non-existent-530151382 not found for user 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-non-existent-530151382 not found for user 353e4db513a94d96bd82b85742624fc2 Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b1e348b-6081-4af9-bdcb-b9a65779be49 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] Returning 404 to user: Keypair tempest-keypair-non-existent-530151382 not found for user 353e4db513a94d96bd82b85742624fc2 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:38.118437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b1e348b-6081-4af9-bdcb-b9a65779be49 tempest-KeyPairsNegativeTestJSON-1412209103 tempest-KeyPairsNegativeTestJSON-1412209103-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-530151382" status: 404 len: 144 microversion: 2.1 time: 0.053326 Oct 15 20:57:38.121298 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1281/2395] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 20:57:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-530151382 => generated 144 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:38.136568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:38.140474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:38.156409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:38.156909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:38.158286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:38.159038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:38.163566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:38.163974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:38.177246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:38.291591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:38.304137 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b46d2fb-916a-4607-83ef-f7a08e2babc8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1204 microversion: 2.1 time: 0.180773 Oct 15 20:57:38.305452 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1282/2396] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:38 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1204 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:38.313158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:38.314107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:38.322081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:38.323044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:38.323718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:38.324758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:38.325495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:38.326531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:39.238756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:39.251772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5430eee-78f2-4251-bc1d-b0f51f1576e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1524 microversion: 2.1 time: 0.944398 Oct 15 20:57:39.255112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1283/2397] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:38 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1524 bytes in 948 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:39.269286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:39.273934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:39.306632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:39.307690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:39.312926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:39.318394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:39.319567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:39.320030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:40.565302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:40.575115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d67e92b-d83e-4663-bf0d-80505c05fb3e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 1.317141 Oct 15 20:57:40.598961 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1284/2398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:39 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 1340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:40.616441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:40.620394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:40.646037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:40.648359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:40.649678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:40.658599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:40.659099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:40.659543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:41.612297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:41.631220 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25eb3f96-bfea-4427-bef4-dede18982461 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1332 microversion: 2.1 time: 1.029795 Oct 15 20:57:41.633915 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1285/2399] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:40 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1332 bytes in 1034 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:41.651302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:41.655400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:41.667474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:41.671007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:41.672353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:41.675652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:41.676263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:41.676899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:42.012837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:42.021240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48c3b2c0-60d5-4d2a-a996-e0908a25d446 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 200 len: 1467 microversion: 2.1 time: 0.385791 Oct 15 20:57:42.029277 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1286/2400] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:41 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 1467 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:42.034408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:42.046048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:42.071283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:42.076369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:42.085379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:42.086147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:42.086616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:42.090223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:42.512536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:42.531836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5ec317e-db60-41f9-8a99-5c5bee9001f0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1266 microversion: 2.1 time: 0.498806 Oct 15 20:57:42.532401 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1287/2401] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:42 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1266 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:42.551139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2ed93977-be73-4a13-aef7-06b9fa57f2ba req-72897ad7-a8e2-4fdb-87cd-90b5a43eb8bb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8", "tag": "48fb08ef-453d-47d0-90ba-6f116d9f1cda"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:42.586601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2ed93977-be73-4a13-aef7-06b9fa57f2ba req-72897ad7-a8e2-4fdb-87cd-90b5a43eb8bb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:42.589496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2ed93977-be73-4a13-aef7-06b9fa57f2ba req-72897ad7-a8e2-4fdb-87cd-90b5a43eb8bb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:42.590629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2ed93977-be73-4a13-aef7-06b9fa57f2ba req-72897ad7-a8e2-4fdb-87cd-90b5a43eb8bb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:42.622609 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2ed93977-be73-4a13-aef7-06b9fa57f2ba req-72897ad7-a8e2-4fdb-87cd-90b5a43eb8bb service nova] Creating event network-changed:48fb08ef-453d-47d0-90ba-6f116d9f1cda for instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 on nch2-master-1 Oct 15 20:57:42.655939 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2ed93977-be73-4a13-aef7-06b9fa57f2ba req-72897ad7-a8e2-4fdb-87cd-90b5a43eb8bb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.116729 Oct 15 20:57:42.657124 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1288/2402] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:43.331789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiagnosticsNegativeTest-server-1039244731", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:43.451172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:44.272735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:45.180384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:45.180820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:45.181664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:45.182049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:45.182810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:45.201719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:45.202211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:45.206326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:45.212555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:45.213088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:45.213521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:45.244742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Getting quotas for project c12effac060d4c23a3a3f0b4019a9bf5. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:45.251144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Getting quotas for user d2fb28a271f343b6b21960f82c213406 and project c12effac060d4c23a3a3f0b4019a9bf5. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:45.283777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:45.286990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:45.344592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79996262-0ecc-4c7d-b418-ed93dbfbdfa9 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.685753 Oct 15 20:57:45.351997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1289/2403] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 2694 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:45.357640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:45.358638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:45.366929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:45.367664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:45.368333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:45.370070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:45.370733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:45.371318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:45.892107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:45.910991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c788efba-6ca8-4bff-8371-01e4bd1e33c7 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1524 microversion: 2.1 time: 0.557553 Oct 15 20:57:45.915393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1290/2404] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:45 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1524 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:45.941738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:45.945525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:45.964172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:45.965714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:45.966733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:45.969594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:45.971961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:45.974469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.343538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:46.352206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b2c734b8-a415-4942-b743-e4b64e3619fd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 0.433278 Oct 15 20:57:46.363568 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1291/2405] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:45 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:46.375722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9ec79335-0dbb-4b80-b8f8-72d3095aa304 req-211a959f-8afc-4b40-ab68-05874a893dfa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d38ead8c-d325-4fda-8df9-acff8948f988", "name": "network-vif-deleted", "tag": "e0ac614b-b366-4d72-b72d-eb57eb12f440"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:46.419679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9ec79335-0dbb-4b80-b8f8-72d3095aa304 req-211a959f-8afc-4b40-ab68-05874a893dfa service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.420164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9ec79335-0dbb-4b80-b8f8-72d3095aa304 req-211a959f-8afc-4b40-ab68-05874a893dfa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.420690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9ec79335-0dbb-4b80-b8f8-72d3095aa304 req-211a959f-8afc-4b40-ab68-05874a893dfa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.448487 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9ec79335-0dbb-4b80-b8f8-72d3095aa304 req-211a959f-8afc-4b40-ab68-05874a893dfa service nova] Creating event network-vif-deleted:e0ac614b-b366-4d72-b72d-eb57eb12f440 for instance d38ead8c-d325-4fda-8df9-acff8948f988 on nch2-master-1 Oct 15 20:57:46.467391 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9ec79335-0dbb-4b80-b8f8-72d3095aa304 req-211a959f-8afc-4b40-ab68-05874a893dfa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101615 Oct 15 20:57:46.469973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1292/2406] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:46.485277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:46.489535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:46.501694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.502190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.502632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.503950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.504411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.504839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.820904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:46.831022 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-42bb1032-057d-4056-8ab7-9db6c90d3b2d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1332 microversion: 2.1 time: 0.358399 Oct 15 20:57:46.832136 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1293/2407] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:46 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1332 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:46.837158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:46.837988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:46.845247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.845986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.848647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.849861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.850409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.850853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.867642 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 HTTP/1.1 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd38ead8c-d325-4fda-8df9-acff8948f988'} Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:57:46.869636 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 20:57:46.874790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:57:46.877148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:57:46.878318 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5028b1d5-c1d0-464d-921f-fa809bc83ffc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 404 len: 111 microversion: 2.1 time: 0.044997 Oct 15 20:57:46.879464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1294/2408] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:46 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:57:46.893375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:46.894248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:46.904441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.905345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.906012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:46.907311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:46.908070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:46.908675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:47.141994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:47.159215 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfb4d8b5-0f02-4354-add7-19e7289c472d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1308 microversion: 2.1 time: 0.273076 Oct 15 20:57:47.163723 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1295/2409] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:46 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1308 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:47.188080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b5af80d4-c8fc-4aa0-8f13-cf7124579855 req-3bd14c0f-5c8b-4c80-9636-edaedb2f66cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6e486821-32b0-426e-aa16-aefb86e003a0", "name": "network-vif-plugged", "status": "completed", "tag": "29188354-03b6-4df3-a00b-31e4cd21cb62"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:47.217969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5af80d4-c8fc-4aa0-8f13-cf7124579855 req-3bd14c0f-5c8b-4c80-9636-edaedb2f66cd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:47.218774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5af80d4-c8fc-4aa0-8f13-cf7124579855 req-3bd14c0f-5c8b-4c80-9636-edaedb2f66cd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:47.219850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b5af80d4-c8fc-4aa0-8f13-cf7124579855 req-3bd14c0f-5c8b-4c80-9636-edaedb2f66cd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:47.232858 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b5af80d4-c8fc-4aa0-8f13-cf7124579855 req-3bd14c0f-5c8b-4c80-9636-edaedb2f66cd service nova] Creating event network-vif-plugged:29188354-03b6-4df3-a00b-31e4cd21cb62 for instance 6e486821-32b0-426e-aa16-aefb86e003a0 on nch1-master-1 Oct 15 20:57:47.253991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b5af80d4-c8fc-4aa0-8f13-cf7124579855 req-3bd14c0f-5c8b-4c80-9636-edaedb2f66cd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077408 Oct 15 20:57:47.255267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1296/2410] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:47.508955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Calling method '.version_select at 0x7fe74f1b0700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:47.533437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:47.537893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:47.538344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:47.539139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:47.539600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:47.540026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:47.572895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-68bb7ab4-901d-4521-9e5a-71b696e29921 tempest-AbsoluteLimitsTestJSON-193301635 tempest-AbsoluteLimitsTestJSON-193301635-project] 10.222.0.16 "GET /compute/v2.1/limits" status: 200 len: 511 microversion: 2.1 time: 0.313486 Oct 15 20:57:47.586058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1297/2411] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 20:57:47 2022] GET /compute/v2.1/limits => generated 511 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:47.683632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsNegativeTestJSON-server-1936601901", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "efcdede0-bcb2-4a81-a47d-7e71d95b2c4a"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:47.781126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] validate_networks() for [('efcdede0-bcb2-4a81-a47d-7e71d95b2c4a', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:48.681016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:48.681613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:48.681955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:48.682375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:48.682797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:48.702925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:48.703420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:48.703868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:48.704607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:48.705100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:48.705532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:48.757142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Getting quotas for project 974bd50eb06e453e9bb20218b9635555. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:48.775381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Getting quotas for user 79e244a9ce2343dbb903a4722da86eaf and project 974bd50eb06e453e9bb20218b9635555. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:48.812553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:48.837792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:48.985073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f0bc57e4-b0cd-4948-9600-cd5d99505761 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.396737 Oct 15 20:57:48.990314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1298/2412] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 1403 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:49.017590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:49.020026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:49.051180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:49.055669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:49.057231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:49.060195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:49.063512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:49.064658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:49.399172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:49.411679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4915d90b-6e1d-4dd0-99e5-cad9e371f2ec tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1323 microversion: 2.1 time: 0.419893 Oct 15 20:57:49.412769 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1299/2413] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:48 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1323 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:49.418876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27feb403-7671-4252-98f8-f4d02b17f6a8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: .version_select at 0x7fe74fb07040>, body: {"keypair": {"name": "tempest-key-152352319", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNhaL9hj0Ud0BS0z1My/E0k9Ag0XcSq7GFv7Qz24Zm2/CJNx5D3tiXlbulJiVFfovF0YDy0NhuBT8sWWYRs03ZSCZJOV1erLr+R1wdDNhTXbibN72Bd0sbNygXuII4x2hQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:49.446828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-27feb403-7671-4252-98f8-f4d02b17f6a8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:49.460395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-27feb403-7671-4252-98f8-f4d02b17f6a8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:49.535207 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27feb403-7671-4252-98f8-f4d02b17f6a8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 381 microversion: 2.1 time: 0.120896 Oct 15 20:57:49.535829 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1300/2414] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:57:49 2022] POST /compute/v2.1/os-keypairs => generated 381 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:49.563305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:49.564550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:49.631853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:49.633968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:49.634648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:49.636575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:49.639275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:49.639973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:49.970415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:49.980131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-74503ec5-35da-4356-872f-c39d178a1735 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1524 microversion: 2.1 time: 0.437114 Oct 15 20:57:49.982972 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1301/2415] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:49 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1524 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:49.988693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:49.989338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:50.010474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:50.011309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:50.018976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:50.021367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:50.022057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:50.025666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:50.979716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:50.993658 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f25cf0b4-a26c-4d22-b709-bba939859ebc tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 1.008822 Oct 15 20:57:51.009795 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1302/2416] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:49 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 1025 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:51.030413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:51.031300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:51.052487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:51.055685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:51.057150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:51.059051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:51.063144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:51.063874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:51.659437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:51.669880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a1dd562e-9b58-46d1-a78f-927d0b85c37e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1332 microversion: 2.1 time: 0.657336 Oct 15 20:57:51.673346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1303/2417] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:51 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1332 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:51.686657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:51.687547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:51.706706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:51.707651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:51.708451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:51.710632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:51.711395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:51.712798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:51.755191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:52.134263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:52.151475 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63d3abfc-7d46-411b-8b56-dbc560971b8e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1308 microversion: 2.1 time: 0.476794 Oct 15 20:57:52.152239 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1304/2418] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:51 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1308 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:52.159009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c0b67e08-935d-4576-93b2-d1f1c49c9901 req-5e6ebbe5-7a01-441f-bb32-241dbc3de31b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e14662ce-cafb-4dcf-add1-d49193e3ed0f", "tag": "15c191de-11bf-42b3-b218-d8913de071bc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:52.182576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c0b67e08-935d-4576-93b2-d1f1c49c9901 req-5e6ebbe5-7a01-441f-bb32-241dbc3de31b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:52.183417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c0b67e08-935d-4576-93b2-d1f1c49c9901 req-5e6ebbe5-7a01-441f-bb32-241dbc3de31b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:52.186717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c0b67e08-935d-4576-93b2-d1f1c49c9901 req-5e6ebbe5-7a01-441f-bb32-241dbc3de31b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:52.214556 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c0b67e08-935d-4576-93b2-d1f1c49c9901 req-5e6ebbe5-7a01-441f-bb32-241dbc3de31b service nova] Creating event network-changed:15c191de-11bf-42b3-b218-d8913de071bc for instance e14662ce-cafb-4dcf-add1-d49193e3ed0f on nch2-master-1 Oct 15 20:57:52.226363 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c0b67e08-935d-4576-93b2-d1f1c49c9901 req-5e6ebbe5-7a01-441f-bb32-241dbc3de31b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072530 Oct 15 20:57:52.230880 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1305/2419] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:52.253433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:52.261808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:52.267091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:52.288870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:52.293212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:52.294800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:52.302963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:52.303855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:52.304533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:52.478144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:52.502810 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7cf04da-a894-4b8a-b91c-bb65e9443e10 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1283 microversion: 2.1 time: 0.269444 Oct 15 20:57:52.503829 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1306/2420] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:52 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1283 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:52.512013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a708d875-0764-45ee-9b65-d5c576abd402", "name": "network-vif-plugged", "status": "completed", "tag": "5570eac9-7f3b-4f14-8252-089d27fe947b"}, {"server_uuid": "ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8", "name": "network-vif-plugged", "status": "completed", "tag": "48fb08ef-453d-47d0-90ba-6f116d9f1cda"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:52.523617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:52.524376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:52.525415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:52.543505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] Creating event network-vif-plugged:5570eac9-7f3b-4f14-8252-089d27fe947b for instance a708d875-0764-45ee-9b65-d5c576abd402 on nch2-master-1 Oct 15 20:57:52.544862 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] Creating event network-vif-plugged:48fb08ef-453d-47d0-90ba-6f116d9f1cda for instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 on nch2-master-1 Oct 15 20:57:52.553456 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fda235a7-565c-4a85-bcab-8ae3dea7e0a6 req-5335f460-2ca9-45f2-a744-3db52d946d80 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.047501 Oct 15 20:57:52.554750 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1307/2421] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:52 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:52.562992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ce64e64-a274-40bc-ac98-7ecc3bc76b23 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '.version_select at 0x7fe74fa3c9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:52.569802 ncdv-master-1 devstack@n-api.service[138688]: WARNING oslo_db.sqlalchemy.utils [None req-9ce64e64-a274-40bc-ac98-7ecc3bc76b23 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Unique keys not in sort_keys. The sorting order may be unstable. Oct 15 20:57:52.578317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ce64e64-a274-40bc-ac98-7ecc3bc76b23 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 350 microversion: 2.1 time: 0.022139 Oct 15 20:57:52.579170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1308/2422] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:57:52 2022] GET /compute/v2.1/os-keypairs => generated 350 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:52.584312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:52.585424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:52.595225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:52.595891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:52.596498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:52.597436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:52.598051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:52.599095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:53.547273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:53.562142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d071b8b-6b4d-4231-b38c-1dc1ec824c5f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1323 microversion: 2.1 time: 0.978131 Oct 15 20:57:53.569671 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1309/2423] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:52 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1323 bytes in 990 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:53.577550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:53.588283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:53.605638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:53.610571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:53.611168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:53.612179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:53.612767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:53.613339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:53.982021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:53.994756 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82b61b2e-af40-4ee5-9be5-77f3852bbf54 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1524 microversion: 2.1 time: 0.422374 Oct 15 20:57:54.009713 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1310/2424] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:53 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1524 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:54.016389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:54.017346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:54.026261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:54.027361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:54.028103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:54.029221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:54.029899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:54.030527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:54.389742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:54.400361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f236a55-5bc8-4754-bcfd-cce32a43e7bd tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 0.389314 Oct 15 20:57:54.401511 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1311/2425] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:54 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:54.407226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:54.408023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:54.416237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:54.419389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:54.420703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:54.421637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:54.422262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:54.422854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:54.632431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:54.642192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67605a22-a99e-4456-b643-0b8268097423 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1332 microversion: 2.1 time: 0.238998 Oct 15 20:57:54.653985 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1312/2426] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:54 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1332 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:54.663764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-674308000", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "key_name": "tempest-key-152352319", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:54.768236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:57:55.398910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:57:55.399875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:57:55.400361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:57:55.400835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:57:55.401362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:57:55.412335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=57 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:57:55.416242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:55.417256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:55.417856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:55.418786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:55.420226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:55.421217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:55.446765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:57:55.459222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:57:55.478904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:57:55.483136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:57:55.587803 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a640102-7a72-4934-ae6f-e22fbedb0161 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.931545 Oct 15 20:57:55.598902 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1313/2427] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:57:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 943 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:57:55.604835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:55.605656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:55.614713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:55.615523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:55.621366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:55.624875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:55.628244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:55.628452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:55.900420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:55.909676 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b73328cb-83d2-4faa-8f6f-d90d495f94d9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1308 microversion: 2.1 time: 0.308972 Oct 15 20:57:55.910476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1314/2428] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:55 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1308 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:55.916472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:55.916895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:55.918479 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 20:57:55.920818 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 20:57:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 20:57:55.924796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:55.925426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:55.925899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:55.926651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:55.927118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:55.928289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:56.216522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:56.231452 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2445db99-6b4f-40e3-91ce-b275dd878161 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1325 microversion: 2.1 time: 0.316332 Oct 15 20:57:56.235189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1315/2429] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:55 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1325 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:56.254817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d972c3f1-24c5-42bf-81d9-fc5819c81c53 req-e901d5fd-fb19-45c3-a235-113ae3fb2b36 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b10aaffc-b90f-47aa-b7a6-cc5dd65e190c", "tag": "af93c5b9-3c0d-44df-8e24-ad5985828b53"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:56.292150 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 20:57:56.292373 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 20:57:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 20:57:56.308359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d972c3f1-24c5-42bf-81d9-fc5819c81c53 req-e901d5fd-fb19-45c3-a235-113ae3fb2b36 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:56.311015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d972c3f1-24c5-42bf-81d9-fc5819c81c53 req-e901d5fd-fb19-45c3-a235-113ae3fb2b36 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:56.313734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d972c3f1-24c5-42bf-81d9-fc5819c81c53 req-e901d5fd-fb19-45c3-a235-113ae3fb2b36 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:56.350670 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d972c3f1-24c5-42bf-81d9-fc5819c81c53 req-e901d5fd-fb19-45c3-a235-113ae3fb2b36 service nova] Creating event network-changed:af93c5b9-3c0d-44df-8e24-ad5985828b53 for instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c on nch2-master-1 Oct 15 20:57:56.365320 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d972c3f1-24c5-42bf-81d9-fc5819c81c53 req-e901d5fd-fb19-45c3-a235-113ae3fb2b36 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.128602 Oct 15 20:57:56.373560 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1316/2430] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:56.389200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6d9d2f6d-40aa-45f2-86a9-d70536fd43d9 req-51a888e0-626f-405e-8c74-540c3ff3bcb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e14662ce-cafb-4dcf-add1-d49193e3ed0f", "name": "network-vif-plugged", "status": "completed", "tag": "15c191de-11bf-42b3-b218-d8913de071bc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:56.408628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d9d2f6d-40aa-45f2-86a9-d70536fd43d9 req-51a888e0-626f-405e-8c74-540c3ff3bcb8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:56.409206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d9d2f6d-40aa-45f2-86a9-d70536fd43d9 req-51a888e0-626f-405e-8c74-540c3ff3bcb8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:56.409645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d9d2f6d-40aa-45f2-86a9-d70536fd43d9 req-51a888e0-626f-405e-8c74-540c3ff3bcb8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:56.426852 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6d9d2f6d-40aa-45f2-86a9-d70536fd43d9 req-51a888e0-626f-405e-8c74-540c3ff3bcb8 service nova] Creating event network-vif-plugged:15c191de-11bf-42b3-b218-d8913de071bc for instance e14662ce-cafb-4dcf-add1-d49193e3ed0f on nch2-master-1 Oct 15 20:57:56.432709 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6d9d2f6d-40aa-45f2-86a9-d70536fd43d9 req-51a888e0-626f-405e-8c74-540c3ff3bcb8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057631 Oct 15 20:57:56.438433 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1317/2431] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:56.447997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:56.448754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:56.458854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:56.459583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:56.460279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:56.461097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:56.461639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:56.462207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:56.741935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:56.751246 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3ecead3-abf2-4f6f-a617-0d4b915c75be tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1323 microversion: 2.1 time: 0.308718 Oct 15 20:57:56.765947 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1318/2432] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:56 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1323 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:56.774107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:56.774107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:56.783105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:56.783663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:56.784104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:56.784965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:56.786271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:56.786414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:57.069994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:57.080515 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-56148d34-1ef4-42f7-a2c6-17ed2f9c76a8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1524 microversion: 2.1 time: 0.313222 Oct 15 20:57:57.085792 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1319/2433] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:56 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1524 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:57.096583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6b737839-d865-4f61-8519-dc03e7f603f1 req-a0fae47d-833d-4519-a4d5-37ac390e3b08 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6", "tag": "6f4c02d0-4987-4f8a-a1e1-917c3c2665bd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:57:57.124121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6b737839-d865-4f61-8519-dc03e7f603f1 req-a0fae47d-833d-4519-a4d5-37ac390e3b08 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:57.132387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6b737839-d865-4f61-8519-dc03e7f603f1 req-a0fae47d-833d-4519-a4d5-37ac390e3b08 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:57.133751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6b737839-d865-4f61-8519-dc03e7f603f1 req-a0fae47d-833d-4519-a4d5-37ac390e3b08 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:57.173542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6b737839-d865-4f61-8519-dc03e7f603f1 req-a0fae47d-833d-4519-a4d5-37ac390e3b08 service nova] Creating event network-changed:6f4c02d0-4987-4f8a-a1e1-917c3c2665bd for instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 on nch1-master-1 Oct 15 20:57:57.189989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6b737839-d865-4f61-8519-dc03e7f603f1 req-a0fae47d-833d-4519-a4d5-37ac390e3b08 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098654 Oct 15 20:57:57.190763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1320/2434] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:57:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:57:57.203772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:57.206616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:57.227873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:57.229643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:57.230111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:57.236588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:57.237133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:57.237579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:57.880175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:57.898744 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b98c4f62-9ada-422a-9e77-0576e26641a0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1318 microversion: 2.1 time: 0.696219 Oct 15 20:57:57.899726 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1321/2435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:57 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1318 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:57.908372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:57.908994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:57.930577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:57.931078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:57.931540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:57.932236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:57.932701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:57.933163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:58.643001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:58.660047 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e08b269-585d-43aa-a5eb-45b2cec8e981 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1279 microversion: 2.1 time: 0.759189 Oct 15 20:57:58.661080 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1322/2436] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:57 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1279 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:58.678813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:58.679930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:58.686911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:58.687728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:58.687859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:58.688534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:58.688992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:58.689722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:59.204206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:59.222159 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a18e9f7e-3802-4505-a78c-f9e0ada35ad7 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1535 microversion: 2.1 time: 0.560394 Oct 15 20:57:59.223081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1323/2437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:58 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1535 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:59.227619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:59.228445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:59.236542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:59.237065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:59.243621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:59.245212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:59.245863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:59.246811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:59.536079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:57:59.548374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1d82b36d-09ff-490a-8db2-f2502970d26f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1308 microversion: 2.1 time: 0.321280 Oct 15 20:57:59.562189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1324/2438] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:59 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1308 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:57:59.567244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:57:59.568083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:57:59.575787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:59.576496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:59.581788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:57:59.582572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:57:59.590411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:57:59.591394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:00.086383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:00.096479 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ba0bf46-cee7-480c-94f6-f6250483d2c5 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1325 microversion: 2.1 time: 0.532891 Oct 15 20:58:00.097750 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1325/2439] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:57:59 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1325 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:00.102811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:00.103596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:00.110611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:00.111173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:00.111685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:00.112460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:00.112985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:00.113702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:00.118392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:00.558336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:00.575250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b39ef41f-f01e-4276-bf57-e82bb59cb27a tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1323 microversion: 2.1 time: 0.468586 Oct 15 20:58:00.578224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1326/2440] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:00 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1323 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:00.594124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:00.596265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:00.615856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:00.616279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:00.616726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:00.621080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:00.622850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:00.623322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:01.160316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:01.171664 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e85be60a-f338-414f-a5d0-000789285383 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1506 microversion: 2.1 time: 0.591947 Oct 15 20:58:01.173199 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1327/2441] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:00 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1506 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:01.198364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:01.199259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:01.206458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:01.207283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:01.208488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:01.209265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:01.209929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:01.210356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:01.571426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:01.579987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50ece6fa-fff7-4760-8955-b0c806e2f31e tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1468 microversion: 2.1 time: 0.391561 Oct 15 20:58:01.580561 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1328/2442] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:01 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1468 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:01.585217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-188130fd-a2d4-46f7-944c-75f1dc9b36fb tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'create', calling method: .version_select at 0x7fe74fa3c0d0>, body: {"pool": "dd4fc1bf-26ef-4766-8df5-50a70c32acab"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:04.100104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-188130fd-a2d4-46f7-944c-75f1dc9b36fb tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.518288 Oct 15 20:58:04.101008 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1329/2443] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:58:01 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2520 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:04.105670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:04.107586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:04.115240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.115816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.116332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.117222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.117757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.118196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.254261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:04.262029 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0868784-274a-463a-82d1-567dc2e1f99c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1327 microversion: 2.1 time: 0.159932 Oct 15 20:58:04.262640 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1330/2444] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:04 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1327 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:04.267127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:04.267988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:04.274059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.274614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.275127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.275901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.276445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.276944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.425827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:04.434249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c93dec02-7a57-4bd5-9d0f-869c873baa03 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1489 microversion: 2.1 time: 0.170554 Oct 15 20:58:04.434876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1331/2445] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:04 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1489 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:04.440160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:04.440781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:04.447063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.447631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.447984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.448694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.449313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.449751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.603805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:04.612881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-876ae615-5f1c-4bc6-8962-cfa30111d2a6 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1325 microversion: 2.1 time: 0.176296 Oct 15 20:58:04.613504 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1332/2446] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:04 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1325 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:04.618169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-702a745f-1833-4299-889c-a573ce94ed38 req-0ff1d97c-e6d6-4160-a066-f716f6372963 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b10aaffc-b90f-47aa-b7a6-cc5dd65e190c", "name": "network-vif-plugged", "status": "completed", "tag": "af93c5b9-3c0d-44df-8e24-ad5985828b53"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:04.628091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-702a745f-1833-4299-889c-a573ce94ed38 req-0ff1d97c-e6d6-4160-a066-f716f6372963 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.628831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-702a745f-1833-4299-889c-a573ce94ed38 req-0ff1d97c-e6d6-4160-a066-f716f6372963 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.629464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-702a745f-1833-4299-889c-a573ce94ed38 req-0ff1d97c-e6d6-4160-a066-f716f6372963 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.642737 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-702a745f-1833-4299-889c-a573ce94ed38 req-0ff1d97c-e6d6-4160-a066-f716f6372963 service nova] Creating event network-vif-plugged:af93c5b9-3c0d-44df-8e24-ad5985828b53 for instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c on nch2-master-1 Oct 15 20:58:04.647998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-702a745f-1833-4299-889c-a573ce94ed38 req-0ff1d97c-e6d6-4160-a066-f716f6372963 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033144 Oct 15 20:58:04.648819 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1333/2447] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:04.654766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:04.655630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:04.663421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.664166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.664798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.665723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:04.666368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:04.666962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:04.889641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:04.897797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef7d7396-00e7-4fea-9ec9-9e912a8df227 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1323 microversion: 2.1 time: 0.247639 Oct 15 20:58:04.898390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1334/2448] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:04 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1323 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:05.082798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "tempest-host-1211506809", "block_migration": true, "disk_over_commit": false}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:05.085348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:05.091697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.092185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.092625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.093357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.093817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.094240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.119936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lazy-loading 'flavor' on Instance uuid a708d875-0764-45ee-9b65-d5c576abd402 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:05.133008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Going to try to live migrate instance to tempest-host-1211506809 {{(pid=138688) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 20:58:05.141128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] HTTP exception thrown: Compute host tempest-host-1211506809 could not be found. Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Request method failure captured: Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402/action HTTP/1.1 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 107 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx61rQpQ7cWwAUg0hYyLNHjZvbjOg7S13V8ZQJys9Z2x7JC4hGJaLnlRA2r0gDNomc6qMC5R19LJSI2GNceqoP3FO9gY-gpIdDOexPvliLQyZD288wrZkWTbUq3D-Ji03ZUsD7E4QzipG2PaMYtc8PNo5KnN2mZ42qmgSRiVBNje8sw Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: af0b8f89094f42fb9ae9829807d9f6d4 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationNegativeTest-1766745771 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationNegativeTest-1766745771 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: af0b8f89094f42fb9ae9829807d9f6d4 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1766745771 Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationNegativeTest-1766745771-project-admin Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 623abfad80794a058a678b8b50009e3e Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationNegativeTest-1766745771-project-admin Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: {"os-migrateLive": {"host": "tempest-host-1211506809", "block_migration": true, "disk_over_commit": false}} Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: exception: Compute host tempest-host-1211506809 could not be found. Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'a708d875-0764-45ee-9b65-d5c576abd402', 'body': {'os-migrateLive': {'host': 'tempest-host-1211506809', 'block_migration': True, 'disk_over_commit': False}}} Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 123, in _migrate_live Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.live_migrate(context, instance, block_migration, Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 259, in inner Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5488, in live_migrate Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nodes = objects.ComputeNodeList.get_all_by_host(context, host_name) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/compute_node.py", line 476, in get_all_by_host Oct 15 20:58:05.170439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_computes = cls._db_compute_node_get_all_by_host(context, host, Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 179, in wrapper Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/compute_node.py", line 472, in _db_compute_node_get_all_by_host Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return db.compute_node_get_all_by_host(context, host) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 241, in wrapper Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 740, in compute_node_get_all_by_host Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host tempest-host-1211506809 could not be found. Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 150, in _migrate_live Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Compute host tempest-host-1211506809 could not be found. Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Compute host tempest-host-1211506809 could not be found. Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Returning 400 to user: Compute host tempest-host-1211506809 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3639060-4fca-40bd-bb90-81d7daf73826 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402/action" status: 400 len: 100 microversion: 2.1 time: 0.273055 Oct 15 20:58:05.176501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1335/2449] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Oct 15 20:58:04 2022] POST /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402/action => generated 100 bytes in 274 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:58:05.182747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:05.182747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:05.186403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.186896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.187393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.188098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.188563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.188998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.330122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:05.338063 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-17a1d7d1-6a65-46f4-8f10-47ec70e8b146 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 200 len: 1449 microversion: 2.1 time: 0.164081 Oct 15 20:58:05.338954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1336/2450] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:05 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 1449 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:05.392901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3954631-b7fd-4c06-949c-8ad3b8a68e6f tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Action: 'create', calling method: .version_select at 0x7fe74fa12670>, body: {"keypair": {"name": "tempest-keypair-763033871", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMEJIDQi3j53zinSoxivLOtNZ5FbU2Z4FImPtGc2ivEjUesXCThWLRTetl8gd5gwz59i+gePLl7p7mhLxkylF8UweTyTHM9qXLd7qfrP59TDl/NLbb1qNTwTpH7G/6aHWA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:05.412298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c3954631-b7fd-4c06-949c-8ad3b8a68e6f tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Getting quotas for project 7e45e841218949e688b3deee07fd25ed. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:05.418309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c3954631-b7fd-4c06-949c-8ad3b8a68e6f tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Getting quotas for user a8d437aff8aa486ca43edb2605a57b4e and project 7e45e841218949e688b3deee07fd25ed. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:05.436153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:05.445437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3954631-b7fd-4c06-949c-8ad3b8a68e6f tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.105380 Oct 15 20:58:05.446046 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1337/2451] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 20:58:05 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:05.451248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-65beee0a-1a9b-4406-bfd9-b9d49f789329 req-452c0470-deb8-49d1-b561-98552f34bdf6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8712cdbc-97cd-492a-865d-81a3d542b7a3", "tag": "e58a1b34-85fe-4f04-bfa1-8ac563b675da"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:05.461925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-65beee0a-1a9b-4406-bfd9-b9d49f789329 req-452c0470-deb8-49d1-b561-98552f34bdf6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.462410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-65beee0a-1a9b-4406-bfd9-b9d49f789329 req-452c0470-deb8-49d1-b561-98552f34bdf6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.463018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-65beee0a-1a9b-4406-bfd9-b9d49f789329 req-452c0470-deb8-49d1-b561-98552f34bdf6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.474560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-65beee0a-1a9b-4406-bfd9-b9d49f789329 req-452c0470-deb8-49d1-b561-98552f34bdf6 service nova] Creating event network-changed:e58a1b34-85fe-4f04-bfa1-8ac563b675da for instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 on nch1-master-1 Oct 15 20:58:05.479940 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-65beee0a-1a9b-4406-bfd9-b9d49f789329 req-452c0470-deb8-49d1-b561-98552f34bdf6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032681 Oct 15 20:58:05.480646 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1338/2452] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:05.486663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'action', calling method: .version_select at 0x7fe7556bee50>, body: {"addFloatingIp": {"address": "172.35.4.173"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:05.489208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:05.497145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.497595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.498114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.498840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:05.499301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:05.499729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:05.583487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 20:58:05.583823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquired lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 20:58:07.664342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 20:58:08.235236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Updating instance_info_cache with network_info: [{"id": "48fb08ef-453d-47d0-90ba-6f116d9f1cda", "address": "fa:16:3e:b8:f1:48", "network": {"id": "76243f48-ac57-4169-afa3-4182fbe1350d", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-394308189-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.4.173", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "11bdafc438884621944db399da5528f8", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap48fb08ef-45", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 20:58:08.249196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Releasing lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 20:58:08.250062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-34c36b0b-cd3b-4e90-b0c8-fe3d6f74b9a9 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action" status: 202 len: 0 microversion: 2.1 time: 2.768071 Oct 15 20:58:08.254123 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1339/2453] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:05 2022] POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action => generated 0 bytes in 2770 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 20:58:08.256486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:08.257088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:08.263920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.264410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.264856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.265618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.266091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.266525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.290524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lazy-loading 'flavor' on Instance uuid e14662ce-cafb-4dcf-add1-d49193e3ed0f {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:08.358061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3222825-5c85-4f92-8e1f-116d587b6f6c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f/action" status: 202 len: 0 microversion: 2.1 time: 0.105351 Oct 15 20:58:08.358687 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1340/2454] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:08 2022] POST /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f/action => generated 0 bytes in 107 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:58:08.364572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:08.365317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:08.372254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.373211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.373699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.374434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.374900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.375336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.578943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:08.587214 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-406f28c2-4234-43c7-b938-b10067bd00b0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1468 microversion: 2.1 time: 0.227272 Oct 15 20:58:08.587797 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1341/2455] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:08 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1468 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:08.597251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:08.597860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:08.604041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.604697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.605453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.606372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.606967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.607521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.748038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:08.757788 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a3916d7-38ec-4077-9647-41bef4448443 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1327 microversion: 2.1 time: 0.168869 Oct 15 20:58:08.758296 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1342/2456] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:08 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1327 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:08.767966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:08.768534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:08.774673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.775232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.775674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.776413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:08.776810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:08.777272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:08.792988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=3,nova_cell1:RELEASE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:08.806216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:08.813772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.compute.api [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Working on deleting snapshot 33adb9c7-f044-4315-9600-a24381427f4d from shelved instance... Oct 15 20:58:09.415551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:09.420206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Doing local delete in cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=138688) _delete /opt/stack/nova/nova/compute/api.py:2546}} Oct 15 20:58:09.420917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:09.421428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:09.421863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:09.422287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.compute.api [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Oct 15 20:58:09.426195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] deallocate_for_instance() {{(pid=138688) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1780}} Oct 15 20:58:09.906764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Updating instance_info_cache with network_info: [] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 20:58:10.206869 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fdce3cc-c09b-4709-aa71-07dac23b6c87 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 204 len: 0 microversion: 2.1 time: 1.442622 Oct 15 20:58:10.207486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1343/2457] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:08 2022] DELETE /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 0 bytes in 1449 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:10.211913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:10.212515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:10.218367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:10.218843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:10.219276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:10.219960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:10.220419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:10.220839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:10.351421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:10.359802 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3da5d9d3-a8d4-4fd8-b20a-fcea88a2bae9 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1325 microversion: 2.1 time: 0.151364 Oct 15 20:58:10.360665 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1344/2458] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:10 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1325 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:10.365187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:10.365800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:10.371359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:10.371841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:10.372276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:10.373518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:10.374119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:10.374572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:10.507362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:10.516275 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-157570e5-8649-437b-8c11-536c67b4b275 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1472 microversion: 2.1 time: 0.154472 Oct 15 20:58:10.516857 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1345/2459] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:10 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1472 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:10.522206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6", "name": "network-vif-plugged", "status": "completed", "tag": "6f4c02d0-4987-4f8a-a1e1-917c3c2665bd"}, {"server_uuid": "6e486821-32b0-426e-aa16-aefb86e003a0", "name": "network-vif-unplugged", "status": "completed", "tag": "29188354-03b6-4df3-a00b-31e4cd21cb62"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:10.533343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:10.533829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:10.534265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:10.547937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] Creating event network-vif-plugged:6f4c02d0-4987-4f8a-a1e1-917c3c2665bd for instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 on nch1-master-1 Oct 15 20:58:10.548471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_external_events [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] Unable to find a host for instance 6e486821-32b0-426e-aa16-aefb86e003a0. Dropping event network-vif-unplugged {{(pid=138688) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 15 20:58:10.553388 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-44a2ac20-606d-44b9-b2ff-5ba32e28b36b req-9afa3cee-9536-4e13-8ccd-5e1c836a8b2b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 353 microversion: 2.1 time: 0.035339 Oct 15 20:58:10.553991 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1346/2460] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:10 2022] POST /compute/v2.1/os-server-external-events => generated 353 bytes in 37 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 15 20:58:10.559383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1884753178", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "29d511ab-25e6-4bb9-a341-70a08a7248cc"}], "security_groups": [{"name": "tempest-securitygroup--1196649737"}], "key_name": "tempest-keypair-763033871", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "OS-DCF:diskConfig": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:10.703832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] validate_networks() for [('29d511ab-25e6-4bb9-a341-70a08a7248cc', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:58:11.117297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Getting quotas for project 7e45e841218949e688b3deee07fd25ed. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:11.122554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Getting quotas for user a8d437aff8aa486ca43edb2605a57b4e and project 7e45e841218949e688b3deee07fd25ed. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:11.134993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:58:11.135407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:58:11.135878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:58:11.136283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:58:11.136688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:58:11.144807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:11.145340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:11.145772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:11.146489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:11.146938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:11.147357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:11.163995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Getting quotas for project 7e45e841218949e688b3deee07fd25ed. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:11.169118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Getting quotas for user a8d437aff8aa486ca43edb2605a57b4e and project 7e45e841218949e688b3deee07fd25ed. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:11.181944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:58:11.184345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:58:11.234073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbd544b9-9539-4a41-8779-ffa37c250797 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 0.678836 Oct 15 20:58:11.234639 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1347/2461] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:10 2022] POST /compute/v2.1/servers => generated 407 bytes in 680 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:58:11.239250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5f2a12e7-c6ff-4743-a08b-ddf7af1da207 req-98c5b4b4-9505-445c-a4b0-7957c7ebfff4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8", "tag": "48fb08ef-453d-47d0-90ba-6f116d9f1cda"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:11.249674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5f2a12e7-c6ff-4743-a08b-ddf7af1da207 req-98c5b4b4-9505-445c-a4b0-7957c7ebfff4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:11.250152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5f2a12e7-c6ff-4743-a08b-ddf7af1da207 req-98c5b4b4-9505-445c-a4b0-7957c7ebfff4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:11.250667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5f2a12e7-c6ff-4743-a08b-ddf7af1da207 req-98c5b4b4-9505-445c-a4b0-7957c7ebfff4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:11.262205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5f2a12e7-c6ff-4743-a08b-ddf7af1da207 req-98c5b4b4-9505-445c-a4b0-7957c7ebfff4 service nova] Creating event network-changed:48fb08ef-453d-47d0-90ba-6f116d9f1cda for instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 on nch2-master-1 Oct 15 20:58:11.267525 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5f2a12e7-c6ff-4743-a08b-ddf7af1da207 req-98c5b4b4-9505-445c-a4b0-7957c7ebfff4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031521 Oct 15 20:58:11.268121 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1348/2462] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:11.273026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c073135-af6a-4a04-8749-2c29eac82c46 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'create', calling method: .version_select at 0x7fe74ee5c8b0>, body: {"pool": "dd4fc1bf-26ef-4766-8df5-50a70c32acab"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:13.277296 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c073135-af6a-4a04-8749-2c29eac82c46 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.007875 Oct 15 20:58:13.278650 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1349/2463] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:58:11 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2009 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:13.282952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:13.283685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:13.290808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:13.291304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:13.291748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:13.292454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:13.292925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:13.293387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:13.517379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:13.525114 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33f0e61e-fb8b-4da9-b275-5949dc966462 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 200 len: 1480 microversion: 2.1 time: 0.245906 Oct 15 20:58:13.541337 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1350/2464] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:13 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 1480 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:13.541337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationNegativeTest-server-843717786", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:13.588386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:58:13.976944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:58:13.977437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:58:13.977953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:58:13.978369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:58:13.978789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:58:13.989827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:13.989827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:13.989827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:13.990526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:13.990987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:13.991415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.008773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Getting quotas for project c44841db4e33412e805da46ba4262fe7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:14.014158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Getting quotas for user 500cba6141d34c4081d38a7b1be1775e and project c44841db4e33412e805da46ba4262fe7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:14.026881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:58:14.029790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:58:14.075574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1bcffca-6ec7-44a8-8671-408af817cee0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.548748 Oct 15 20:58:14.082777 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1351/2465] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 550 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:58:14.082777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:14.082777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:14.088547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.089816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.089816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.090189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.090649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.091084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.311678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:14.320241 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce87b86c-7433-4bc7-af10-0374441b538f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1327 microversion: 2.1 time: 0.242906 Oct 15 20:58:14.321355 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1352/2466] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:14 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1327 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:14.326823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:14.327621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:14.333784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.334457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.335038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.335767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.336487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.337173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.354534 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 HTTP/1.1 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6e486821-32b0-426e-aa16-aefb86e003a0'} Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:14.356518 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 20:58:14.360799 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.362160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:14.363138 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bb0690e-9c1e-4604-8c1f-69967dfd979d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 404 len: 111 microversion: 2.1 time: 0.040705 Oct 15 20:58:14.363850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1353/2467] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:14 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:14.369226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:14.369794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:14.376629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.377115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.377579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.378419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.378891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.379435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.408223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-server-diagnostics failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Request method failure captured: Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c/diagnostics HTTP/1.1 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx7C4uUMeyiMVRSUgRt9LpBxE_NK3999CCSQOoiiaxvM3OqWH4I-eqPXA--P_yhPOJ7gvycw3-YCxhYZ4OOVAXKAf22v32QWUmv3kc7HrwIJ0l5Hx_zDvf-A2QKVIDvfzBJ853CS_UQoyyHAWePy8buWBK1JYsrMROWBQipHL79gmwQ Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c12effac060d4c23a3a3f0b4019a9bf5 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiagnosticsNegativeTest-925702978 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiagnosticsNegativeTest-925702978 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c12effac060d4c23a3a3f0b4019a9bf5 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiagnosticsNegativeTest-925702978 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiagnosticsNegativeTest-925702978-project Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d2fb28a271f343b6b21960f82c213406 Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiagnosticsNegativeTest-925702978-project Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: exception: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': 'b10aaffc-b90f-47aa-b7a6-cc5dd65e190c'} Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_diagnostics.py", line 38, in index Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi context.can(sd_policies.BASE_POLICY_NAME, Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.force_reraise() Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise self.value Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.enforce( Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.413674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:14.417783 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:14.417783 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:14.417783 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Oct 15 20:58:14.417783 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Oct 15 20:58:14.417783 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Oct 15 20:58:14.417783 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:14.419185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:14.423219 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8804c48e-00dc-4145-a5d3-c6c5342b478f tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c/diagnostics" status: 403 len: 117 microversion: 2.1 time: 0.054258 Oct 15 20:58:14.429895 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1354/2468] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:58:14 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c/diagnostics => generated 117 bytes in 59 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:14.429895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:14.429895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:14.436956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.437510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.438221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.438641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.439099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.439904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.676205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:14.685853 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e31d241-290d-4cfc-968b-353d39f22eb8 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1395 microversion: 2.1 time: 0.260163 Oct 15 20:58:14.686757 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1355/2469] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:14 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1395 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:14.691349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:14.692182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:14.698579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.699249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.699847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.700708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.701921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.702343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.946719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:14.956759 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-630b71f8-42cb-40d3-a910-3ead1dd75048 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1522 microversion: 2.1 time: 0.268778 Oct 15 20:58:14.957354 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1356/2470] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:14 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1522 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:14.961916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8c21a21e-2c2b-4e4f-ac20-5cd1d51219a7 req-014eebc5-1899-4e8f-ab07-0a4d112142c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8712cdbc-97cd-492a-865d-81a3d542b7a3", "name": "network-vif-plugged", "status": "completed", "tag": "e58a1b34-85fe-4f04-bfa1-8ac563b675da"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:14.971987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8c21a21e-2c2b-4e4f-ac20-5cd1d51219a7 req-014eebc5-1899-4e8f-ab07-0a4d112142c3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:14.972479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8c21a21e-2c2b-4e4f-ac20-5cd1d51219a7 req-014eebc5-1899-4e8f-ab07-0a4d112142c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:14.973023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8c21a21e-2c2b-4e4f-ac20-5cd1d51219a7 req-014eebc5-1899-4e8f-ab07-0a4d112142c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:14.985683 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8c21a21e-2c2b-4e4f-ac20-5cd1d51219a7 req-014eebc5-1899-4e8f-ab07-0a4d112142c3 service nova] Creating event network-vif-plugged:e58a1b34-85fe-4f04-bfa1-8ac563b675da for instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 on nch1-master-1 Oct 15 20:58:14.990695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8c21a21e-2c2b-4e4f-ac20-5cd1d51219a7 req-014eebc5-1899-4e8f-ab07-0a4d112142c3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032193 Oct 15 20:58:14.991295 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1357/2471] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:14.996460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a2de33a1-3577-4559-b2ec-094fb589ed08 req-19d68d57-718f-4e3c-8ddb-709b64ccc5c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6e486821-32b0-426e-aa16-aefb86e003a0", "name": "network-vif-deleted", "tag": "29188354-03b6-4df3-a00b-31e4cd21cb62"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:15.006640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a2de33a1-3577-4559-b2ec-094fb589ed08 req-19d68d57-718f-4e3c-8ddb-709b64ccc5c1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:15.007156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a2de33a1-3577-4559-b2ec-094fb589ed08 req-19d68d57-718f-4e3c-8ddb-709b64ccc5c1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:15.007571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a2de33a1-3577-4559-b2ec-094fb589ed08 req-19d68d57-718f-4e3c-8ddb-709b64ccc5c1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:15.020746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_external_events [req-a2de33a1-3577-4559-b2ec-094fb589ed08 req-19d68d57-718f-4e3c-8ddb-709b64ccc5c1 service nova] Unable to find a host for instance 6e486821-32b0-426e-aa16-aefb86e003a0. Dropping event network-vif-deleted {{(pid=138688) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 15 20:58:15.021965 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a2de33a1-3577-4559-b2ec-094fb589ed08 req-19d68d57-718f-4e3c-8ddb-709b64ccc5c1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.029310 Oct 15 20:58:15.023004 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1358/2472] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:14 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 31 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 15 20:58:15.028119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.4.106"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:15.030715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:15.036622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:15.037136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:15.037570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:15.038254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:15.038705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:15.039127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:15.065006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 20:58:15.065984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquired lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 20:58:15.961542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Error message: {"NeutronError": {"type": "FloatingIPPortAlreadyAssociated", "message": "Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab.", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:58:15.962167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Releasing lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 20:58:15.962713 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] HTTP exception thrown: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.962713 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Request method failure captured: Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action HTTP/1.1 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 46 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx66JOVrxpwugtcXazHYTtxH3c6dtmp_CEZKPkSziQRU6QQW05DhnQcZB0oeCGJ762xkIkP96GGY1t2123LVOQyDJTeNEbIryDADbMw92rukqgogMBfhT5MBdsTupzUnri85iRvLkY7IMFk5sXrsWGHIGmf6_Z4pAIlKVJdSISiRhts Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_11bdafc438884621944db399da5528f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/11bdafc438884621944db399da5528f8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 78d679a10c9e4bf2bc35786235aeaf2a Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: {"addFloatingIp": {"address": "172.35.4.106"}} Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8', 'body': {'addFloatingIp': {'address': '172.35.4.106'}}} Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2703, in associate_floating_ip Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi client.update_floatingip(fip['id'], {'floatingip': param}) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1016, in update_floatingip Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._update_resource(self.floatingip_path % (floatingip), Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 2548, in _update_resource Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.put(path, **kwargs) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 365, in put Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("PUT", action, body=body, Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:58:15.964823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 239, in _add_floating_ip Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.network_api.associate_floating_ip(context, instance, Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 165, in wrapper Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = f(self, context, *args, **kwargs) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2705, in associate_floating_ip Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpAssociateFailed(str(e)) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpAssociateFailed: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 246, in _add_floating_ip Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=e.format_message()) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] Oct 15 20:58:15.969030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:15.972052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Returning 400 to user: Cannot associate floating IP 172.35.4.106 (7f5b60a5-f7fe-4518-b648-63de2c7900da) with port 48fb08ef-453d-47d0-90ba-6f116d9f1cda using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network dd4fc1bf-26ef-4766-8df5-50a70c32acab. Oct 15 20:58:15.972052 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-5c6bcf3b-706d-4d5e-8e20-2bc3923b1c0c'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:15.972052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6ba9d8b-ef6c-40f8-9f91-649e7044016d tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action" status: 400 len: 381 microversion: 2.1 time: 0.942079 Oct 15 20:58:15.972052 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1359/2473] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:15 2022] POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action => generated 381 bytes in 944 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:58:15.975461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1235114305"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:15.977760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:15.985152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:15.985806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:15.986245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:15.986976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:15.987463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:15.987919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:15.992236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=16,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:16.202936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e638cf80-5aa4-41b5-804a-f6819d811461 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f/action" status: 202 len: 0 microversion: 2.1 time: 0.233531 Oct 15 20:58:16.209259 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1360/2474] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:15 2022] POST /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f/action => generated 0 bytes in 241 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:58:16.215600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:16.216992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:16.222844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:16.223340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:16.223780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:16.224519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:16.224982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:16.225642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:16.327283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:16.345589 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39998cca-e963-4939-bc44-823df74b3dd0 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1276 microversion: 2.1 time: 0.134882 Oct 15 20:58:16.346808 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1361/2475] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:16 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1276 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:16.355463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:16.356813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:16.366084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:16.366625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:16.367134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:16.368746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:16.369360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:16.369857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:16.403495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:16.488821 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-825c8e6c-ae35-47dd-a3e5-d33abc300804 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 204 len: 0 microversion: 2.1 time: 0.139535 Oct 15 20:58:16.493784 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1362/2476] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:16 2022] DELETE /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:16.499284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1060006712", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:16.555798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:58:17.063888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:58:17.064544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:58:17.064767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:58:17.065208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:58:17.065563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:58:17.076645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.077167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.078056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.078512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.078969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.079392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.097241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:17.102507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:17.116580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:58:17.119122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:58:17.130366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:17.170120 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30e223a6-6bce-4a9b-af83-3f8a489f443f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.674462 Oct 15 20:58:17.171236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1363/2477] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 677 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:58:17.181315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:17.182050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:17.200607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.202337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.202337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.204189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.204372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.204885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.239937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] HTTP exception thrown: Action 999 not found Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Request method failure captured: Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6/os-instance-actions/999 HTTP/1.1 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx7H2r-AHe3h2S7RP4tUoB7gU7FKaSL6UClMzly0cSnNieodRCaW2OUk4PLqjIACNQAEe4GXllTr_eWQ72zGckzsgaejS5HXDZGvv_eJf3MrpyseKBUObujwZiDyHku4NRIT1_fXTdGEQtjY1iEU962zussq9UtdDOKEXs99IJ29hx4 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 974bd50eb06e453e9bb20218b9635555 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_974bd50eb06e453e9bb20218b9635555"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 974bd50eb06e453e9bb20218b9635555 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-InstanceActionsNegativeTestJSON-1975732239-project Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 79e244a9ce2343dbb903a4722da86eaf Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-1975732239-project Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: exception: Action 999 not found Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6', 'id': '999'} Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 152, in show Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Action 999 not found Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Action 999 not found Oct 15 20:58:17.247007 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:17.251295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Returning 404 to user: Action 999 not found {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:17.251724 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ca18ec9-f5a3-4881-b98d-5e5dafc51894 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6/os-instance-actions/999" status: 404 len: 66 microversion: 2.1 time: 0.079171 Oct 15 20:58:17.253159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1364/2478] 10.222.0.16 () {58 vars in 1308 bytes} [Sat Oct 15 20:58:17 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6/os-instance-actions/999 => generated 66 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:58:17.259714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:17.260424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:17.269180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.269180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.269427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.271118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.271599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.271996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.533000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:17.541229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc326d67-6adf-448a-a8c5-9734799e00e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1509 microversion: 2.1 time: 0.286795 Oct 15 20:58:17.559949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1365/2479] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:17 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1509 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:17.565076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:17.565738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:17.571742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.572200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.572631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.573406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:17.573864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:17.574286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:17.778743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:17.791678 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3831760-8d71-4765-b8df-cf1e030f20dc tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1395 microversion: 2.1 time: 0.230347 Oct 15 20:58:17.792472 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1366/2480] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:17 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1395 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:17.818547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14a95b8c-13a3-4869-91df-182250953db2 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '.version_select at 0x7fe74ee001f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.035502 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14a95b8c-13a3-4869-91df-182250953db2 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/7f5b60a5-f7fe-4518-b648-63de2c7900da" status: 202 len: 0 microversion: 2.1 time: 2.224244 Oct 15 20:58:20.037081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1367/2481] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:58:17 2022] DELETE /compute/v2.1/os-floating-ips/7f5b60a5-f7fe-4518-b648-63de2c7900da => generated 0 bytes in 2244 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:58:20.044061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6712420-b651-4f78-b5bf-f3ee31150ab8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ee00670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.114694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6712420-b651-4f78-b5bf-f3ee31150ab8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.076176 Oct 15 20:58:20.116285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1368/2482] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:20.121997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.123507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:20.130833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.132161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.132776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.134222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.134893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.136130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.265260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2fb28a271f343b6b21960f82c213406', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c12effac060d4c23a3a3f0b4019a9bf5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:20.277522 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7748757-0d48-4711-8b8d-f5eb57c2b57c tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 200 len: 1436 microversion: 2.1 time: 0.159674 Oct 15 20:58:20.278854 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1369/2483] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 1436 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:20.287240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-41ae376f-001e-4aef-977d-1189fcb3107d req-dcb5503d-619c-4c96-8a98-466dc2f24fca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e970fbbc-4044-415f-833d-21f6bd4d33a2", "tag": "dba907e7-a00a-4b1c-813e-5237b472bb44"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:20.324603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-41ae376f-001e-4aef-977d-1189fcb3107d req-dcb5503d-619c-4c96-8a98-466dc2f24fca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.325439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-41ae376f-001e-4aef-977d-1189fcb3107d req-dcb5503d-619c-4c96-8a98-466dc2f24fca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.326996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-41ae376f-001e-4aef-977d-1189fcb3107d req-dcb5503d-619c-4c96-8a98-466dc2f24fca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.362842 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-41ae376f-001e-4aef-977d-1189fcb3107d req-dcb5503d-619c-4c96-8a98-466dc2f24fca service nova] Creating event network-changed:dba907e7-a00a-4b1c-813e-5237b472bb44 for instance e970fbbc-4044-415f-833d-21f6bd4d33a2 on nch1-master-1 Oct 15 20:58:20.371091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-41ae376f-001e-4aef-977d-1189fcb3107d req-dcb5503d-619c-4c96-8a98-466dc2f24fca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090753 Oct 15 20:58:20.372737 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1370/2484] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:20.379179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.380616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:20.387764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.388668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.389501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.390454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.391215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.391812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.621380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:20.630343 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0daf8d71-8721-4611-bca7-b86c3a6d443a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1315 microversion: 2.1 time: 0.255430 Oct 15 20:58:20.631125 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1371/2485] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1315 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:20.636153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34137b22-7d73-4113-88d0-1dc4046be028 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '.version_select at 0x7fe74ee42f70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.636719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-34137b22-7d73-4113-88d0-1dc4046be028 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 1a8b257d-4c9c-478f-99b4-7cf3c4e98084] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:20.643384 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-34137b22-7d73-4113-88d0-1dc4046be028 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] HTTP exception thrown: Instance 1a8b257d-4c9c-478f-99b4-7cf3c4e98084 could not be found. Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34137b22-7d73-4113-88d0-1dc4046be028 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Request method failure captured: Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1a8b257d-4c9c-478f-99b4-7cf3c4e98084/os-instance-actions HTTP/1.1 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx7H2r-AHe3h2S7RP4tUoB7gU7FKaSL6UClMzly0cSnNieodRCaW2OUk4PLqjIACNQAEe4GXllTr_eWQ72zGckzsgaejS5HXDZGvv_eJf3MrpyseKBUObujwZiDyHku4NRIT1_fXTdGEQtjY1iEU962zussq9UtdDOKEXs99IJ29hx4 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 974bd50eb06e453e9bb20218b9635555 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_974bd50eb06e453e9bb20218b9635555"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 974bd50eb06e453e9bb20218b9635555 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-InstanceActionsNegativeTestJSON-1975732239-project Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 79e244a9ce2343dbb903a4722da86eaf Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-1975732239-project Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1a8b257d-4c9c-478f-99b4-7cf3c4e98084 could not be found. Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '1a8b257d-4c9c-478f-99b4-7cf3c4e98084'} Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1a8b257d-4c9c-478f-99b4-7cf3c4e98084 could not be found. Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 89, in index Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(req, context, server_id) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 77, in _get_instance Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return common.get_instance(self.compute_api, context, server_id) Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:20.645066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1a8b257d-4c9c-478f-99b4-7cf3c4e98084 could not be found. Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1a8b257d-4c9c-478f-99b4-7cf3c4e98084 could not be found. Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34137b22-7d73-4113-88d0-1dc4046be028 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Returning 404 to user: Instance 1a8b257d-4c9c-478f-99b4-7cf3c4e98084 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-34137b22-7d73-4113-88d0-1dc4046be028 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/1a8b257d-4c9c-478f-99b4-7cf3c4e98084/os-instance-actions" status: 404 len: 111 microversion: 2.1 time: 0.014406 Oct 15 20:58:20.648725 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1372/2486] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/servers/1a8b257d-4c9c-478f-99b4-7cf3c4e98084/os-instance-actions => generated 111 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:20.658269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.658894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:20.674882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.675704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.676290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.677192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.678078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.678680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.818565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:20.826578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c55cbb48-8c2a-4dd7-aefa-c5f2366f4d15 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1318 microversion: 2.1 time: 0.174302 Oct 15 20:58:20.827219 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1373/2487] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1318 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:20.831633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.832244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:20.838159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.838643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.839084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.839781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:20.840242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:20.840675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:20.981398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:20.989542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-62b00072-c5e9-4589-af65-08ac80bd0f24 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1509 microversion: 2.1 time: 0.161303 Oct 15 20:58:20.990251 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1374/2488] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1509 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:20.994679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:20.995431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:21.001331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:21.001816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:21.002250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:21.002942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:21.003393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:21.003815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:21.136941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:21.145677 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80430cfa-10b2-4b38-b366-9dd83e820395 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1395 microversion: 2.1 time: 0.154596 Oct 15 20:58:21.146246 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1375/2489] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:20 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1395 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:21.150906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'action', calling method: .version_select at 0x7fe7556beee0>, body: {"removeFloatingIp": {"address": "172.35.4.173"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:21.469489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:21.477301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:21.478218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:21.479175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:21.479994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:21.480662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:21.481225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:22.189289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 20:58:22.190881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquired lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 20:58:23.432858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 20:58:24.456769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Updating instance_info_cache with network_info: [{"id": "48fb08ef-453d-47d0-90ba-6f116d9f1cda", "address": "fa:16:3e:b8:f1:48", "network": {"id": "76243f48-ac57-4169-afa3-4182fbe1350d", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-394308189-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "11bdafc438884621944db399da5528f8", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap48fb08ef-45", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 20:58:24.483286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Releasing lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 20:58:24.484514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-11433c75-38af-44a7-abcd-db26f46f73fa tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action" status: 202 len: 0 microversion: 2.1 time: 3.337241 Oct 15 20:58:24.485681 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1376/2490] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:21 2022] POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action => generated 0 bytes in 3339 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 20:58:24.491089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:24.491802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:24.501307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:24.502149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:24.502931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:24.504599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:24.505097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:24.505617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:24.513094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=3,nova_cell1:INSERT=2,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:24.547685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:24.657589 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5da4a712-322f-49f0-978c-eb76055b392d tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "DELETE /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 204 len: 0 microversion: 2.1 time: 0.170329 Oct 15 20:58:24.658290 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1377/2491] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:24 2022] DELETE /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:24.664491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:24.665432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:24.672347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:24.672847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:24.673389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:24.674162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:24.674799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:24.675236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:24.706740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:24.801932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32e0e2c3-ba6c-4e02-9ce9-44185c9b85d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 204 len: 0 microversion: 2.1 time: 0.141529 Oct 15 20:58:24.802732 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1378/2492] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:24 2022] DELETE /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:24.808835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae7ca92d-6b1b-4a04-ac3f-c6277cb96d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edd6af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:24.927478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae7ca92d-6b1b-4a04-ac3f-c6277cb96d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.123056 Oct 15 20:58:24.930933 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1379/2493] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:24 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:24.936371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:24.937301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] [instance: b10aaffc-b90f-47aa-b7a6-cc5dd65e190c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:24.949359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:24.950823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:24.951432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:24.952358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:24.953780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:24.956576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:24.977608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] HTTP exception thrown: Instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c could not be found. Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Request method failure captured: Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c HTTP/1.1 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx7C4uUMeyiMVRSUgRt9LpBxE_NK3999CCSQOoiiaxvM3OqWH4I-eqPXA--P_yhPOJ7gvycw3-YCxhYZ4OOVAXKAf22v32QWUmv3kc7HrwIJ0l5Hx_zDvf-A2QKVIDvfzBJ853CS_UQoyyHAWePy8buWBK1JYsrMROWBQipHL79gmwQ Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c12effac060d4c23a3a3f0b4019a9bf5 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiagnosticsNegativeTest-925702978 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c12effac060d4c23a3a3f0b4019a9bf5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiagnosticsNegativeTest-925702978 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c12effac060d4c23a3a3f0b4019a9bf5 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiagnosticsNegativeTest-925702978 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiagnosticsNegativeTest-925702978-project Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d2fb28a271f343b6b21960f82c213406 Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiagnosticsNegativeTest-925702978-project Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c could not be found. Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b10aaffc-b90f-47aa-b7a6-cc5dd65e190c'} Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c could not be found. Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c could not be found. Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:24.979868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c could not be found. Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] Returning 404 to user: Instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b144e963-e506-4b65-91b5-011b330e2a38 tempest-ServerDiagnosticsNegativeTest-925702978 tempest-ServerDiagnosticsNegativeTest-925702978-project] 10.222.0.16 "GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c" status: 404 len: 111 microversion: 2.1 time: 0.049231 Oct 15 20:58:24.985329 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1380/2494] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:24 2022] GET /compute/v2.1/servers/b10aaffc-b90f-47aa-b7a6-cc5dd65e190c => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:24.994322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:24.995150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:25.002024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.002518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.002956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.003660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.004280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.004859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.172213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:25.183839 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e3942048-6f9e-4dc1-91f1-6e22b964f405 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1315 microversion: 2.1 time: 0.199819 Oct 15 20:58:25.184411 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1381/2495] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:24 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1315 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:25.204096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:25.204708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:25.211320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.212131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.212569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.213293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.213873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.214308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.413493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:25.426365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8ca493dd-06e0-4290-9ac6-f33d34cfb74c tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1318 microversion: 2.1 time: 0.240890 Oct 15 20:58:25.441637 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1382/2496] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:25 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1318 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:25.448335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fbe6798f-9eb9-478f-8a25-46e58ecbdbb1 req-d45132a1-16e0-4a97-9630-8caa5c073730 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e970fbbc-4044-415f-833d-21f6bd4d33a2", "name": "network-vif-plugged", "status": "completed", "tag": "dba907e7-a00a-4b1c-813e-5237b472bb44"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:25.463929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fbe6798f-9eb9-478f-8a25-46e58ecbdbb1 req-d45132a1-16e0-4a97-9630-8caa5c073730 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.465120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fbe6798f-9eb9-478f-8a25-46e58ecbdbb1 req-d45132a1-16e0-4a97-9630-8caa5c073730 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.465653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fbe6798f-9eb9-478f-8a25-46e58ecbdbb1 req-d45132a1-16e0-4a97-9630-8caa5c073730 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.483178 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fbe6798f-9eb9-478f-8a25-46e58ecbdbb1 req-d45132a1-16e0-4a97-9630-8caa5c073730 service nova] Creating event network-vif-plugged:dba907e7-a00a-4b1c-813e-5237b472bb44 for instance e970fbbc-4044-415f-833d-21f6bd4d33a2 on nch1-master-1 Oct 15 20:58:25.490971 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fbe6798f-9eb9-478f-8a25-46e58ecbdbb1 req-d45132a1-16e0-4a97-9630-8caa5c073730 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047893 Oct 15 20:58:25.491585 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1383/2497] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:25.505863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:25.509699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:25.528004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.528745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.529577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.530567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.531069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.532498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.742365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:25.754608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b374df53-c93a-43e1-ada7-e16c8cc8b798 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1395 microversion: 2.1 time: 0.258655 Oct 15 20:58:25.771254 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1384/2498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:25 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1395 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:25.776857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ca7be734-48df-44ab-9b11-275028d8f513", "tag": "c84eb606-33ff-47f4-9f41-040ccf41d631"}, {"server_uuid": "b10aaffc-b90f-47aa-b7a6-cc5dd65e190c", "name": "network-vif-deleted", "tag": "af93c5b9-3c0d-44df-8e24-ad5985828b53"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:25.790353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:25.790847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:25.791678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:25.813412 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] Creating event network-changed:c84eb606-33ff-47f4-9f41-040ccf41d631 for instance ca7be734-48df-44ab-9b11-275028d8f513 on nch1-master-1 Oct 15 20:58:25.814021 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] Creating event network-vif-deleted:af93c5b9-3c0d-44df-8e24-ad5985828b53 for instance b10aaffc-b90f-47aa-b7a6-cc5dd65e190c on nch2-master-1 Oct 15 20:58:25.831772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-66432e98-730b-460b-b120-34bcda2990c9 req-0d12837a-7de0-479e-8613-ba115c5f156b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.058641 Oct 15 20:58:25.833661 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1385/2499] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:25 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:25.840415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c8fcd8d-2f26-4e0c-ad26-6d4f03d187b0 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '.version_select at 0x7fe74ee42790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:26.930492 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c8fcd8d-2f26-4e0c-ad26-6d4f03d187b0 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/f63eaade-9100-41ee-8eaf-cf1b05bec212" status: 202 len: 0 microversion: 2.1 time: 1.095394 Oct 15 20:58:26.931103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1386/2500] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:58:25 2022] DELETE /compute/v2.1/os-floating-ips/f63eaade-9100-41ee-8eaf-cf1b05bec212 => generated 0 bytes in 1097 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:58:26.937262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:26.938004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:26.955959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:26.956451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:26.956894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:26.957628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:26.958084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:26.958511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:27.087873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:27.109298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=31 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:27.149625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:27.157819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ee9febd-b348-4396-99b8-49897625a5b1 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1528 microversion: 2.1 time: 0.225735 Oct 15 20:58:27.162242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1387/2501] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:26 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1528 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:27.174158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:27.178072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:27.193949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:27.195471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:27.206852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:27.214527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:27.218093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:27.218745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:27.901131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:27.908703 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67c5a21a-8a79-4251-98ed-005a05c8a39c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1515 microversion: 2.1 time: 0.738341 Oct 15 20:58:27.909398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1388/2502] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:27 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1515 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:27.939915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e7ee870-399f-41b4-8377-f2adc43fbbfe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edd6a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:28.042979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e7ee870-399f-41b4-8377-f2adc43fbbfe tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.115221 Oct 15 20:58:28.043952 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1389/2503] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:27 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:28.055718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:28.063579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:28.089348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:28.089847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:28.090294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:28.091016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:28.092026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:28.092564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:28.692742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:28.702191 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1598bd7-c960-4664-88fc-539c6f910767 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1315 microversion: 2.1 time: 0.656826 Oct 15 20:58:28.708508 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1390/2504] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:28 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1315 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:28.713687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:28.714641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:28.725811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:28.726805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:28.727480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:28.728447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:28.729167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:28.729807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.114220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:29.132284 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-65c3cf9d-5ecf-41af-9c81-48467605204d tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1318 microversion: 2.1 time: 0.422289 Oct 15 20:58:29.139115 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1391/2505] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:28 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1318 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:29.167054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:29.173012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:29.197419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.215474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.216139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.217614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.218165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.227781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.527481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:29.540699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ee17fea-ebfb-49da-88c0-d5de89bed132 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1575 microversion: 2.1 time: 0.398562 Oct 15 20:58:29.542711 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1392/2506] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:29 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1575 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:29.548701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b69925b-4704-4d63-8efd-56d5659a121e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": ""}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b69925b-4704-4d63-8efd-56d5659a121e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Request method failure captured: Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action HTTP/1.1 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 34 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx66JOVrxpwugtcXazHYTtxH3c6dtmp_CEZKPkSziQRU6QQW05DhnQcZB0oeCGJ762xkIkP96GGY1t2123LVOQyDJTeNEbIryDADbMw92rukqgogMBfhT5MBdsTupzUnri85iRvLkY7IMFk5sXrsWGHIGmf6_Z4pAIlKVJdSISiRhts Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_11bdafc438884621944db399da5528f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/11bdafc438884621944db399da5528f8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 78d679a10c9e4bf2bc35786235aeaf2a Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: {"addFloatingIp": {"address": ""}} Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8', 'body': {'addFloatingIp': {'address': ''}}} Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is not valid under any of the given schemas Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'oneOf' in schema['properties']['addFloatingIp']['properties']['address']: Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'oneOf': [{'format': 'ipv4'}, {'format': 'ipv6'}], 'type': 'string'} Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['addFloatingIp']: Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '' Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:58:29.558397 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Oct 15 20:58:29.562982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:29.565603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b69925b-4704-4d63-8efd-56d5659a121e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Returning 400 to user: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:29.582912 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b69925b-4704-4d63-8efd-56d5659a121e tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action" status: 400 len: 142 microversion: 2.1 time: 0.037620 Oct 15 20:58:29.586186 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1393/2507] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:29 2022] POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action => generated 142 bytes in 43 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:58:29.595795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e14662ce-cafb-4dcf-add1-d49193e3ed0f", "name": "network-vif-plugged", "status": "completed", "tag": "15c191de-11bf-42b3-b218-d8913de071bc"}, {"server_uuid": "ca7be734-48df-44ab-9b11-275028d8f513", "name": "network-vif-plugged", "status": "completed", "tag": "c84eb606-33ff-47f4-9f41-040ccf41d631"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:29.628529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.629252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.629904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.656040 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] Creating event network-vif-plugged:15c191de-11bf-42b3-b218-d8913de071bc for instance e14662ce-cafb-4dcf-add1-d49193e3ed0f on nch2-master-1 Oct 15 20:58:29.656779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] Creating event network-vif-plugged:c84eb606-33ff-47f4-9f41-040ccf41d631 for instance ca7be734-48df-44ab-9b11-275028d8f513 on nch1-master-1 Oct 15 20:58:29.675009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-691fabca-ebf9-4b0d-91b6-948a5d8cd407 req-85f05e93-03d4-40c4-ad34-ba4ac76e491b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.086968 Oct 15 20:58:29.677797 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1394/2508] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:29 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:29.698276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "15bdf694-ed96-4052-b284-784636eefd0a", "tag": "929e53c0-85c1-4e31-a351-d67f070c1486"}, {"name": "network-changed", "server_uuid": "ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8", "tag": "48fb08ef-453d-47d0-90ba-6f116d9f1cda"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:29.730752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.732109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.734169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.750912 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] Creating event network-changed:929e53c0-85c1-4e31-a351-d67f070c1486 for instance 15bdf694-ed96-4052-b284-784636eefd0a on nch1-master-1 Oct 15 20:58:29.751428 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] Creating event network-changed:48fb08ef-453d-47d0-90ba-6f116d9f1cda for instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 on nch2-master-1 Oct 15 20:58:29.763028 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cd94734a-4e97-4e57-a496-2cba2c00a1e2 req-cae101e0-608a-4eca-93db-5ad796bd2221 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.082391 Oct 15 20:58:29.763979 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1395/2509] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:29 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:29.771864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:29.772584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:29.780986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.781522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.781959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.784218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.784218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.784218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.848678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:29.849200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:29.849638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:29.850093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lazy-loading 'fault' on Instance uuid 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:30.014496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '79e244a9ce2343dbb903a4722da86eaf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '974bd50eb06e453e9bb20218b9635555', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:30.024830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6bee158c-0598-43a6-818a-7284362a25d7 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 200 len: 1315 microversion: 2.1 time: 0.258777 Oct 15 20:58:30.068737 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1396/2510] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:29 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 1315 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:30.084881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:30.086482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:30.102018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:30.105693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:30.106689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:30.107715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:30.108368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:30.111993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:30.164167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:30.197099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:30.198608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:30.202223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:30.203356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lazy-loading 'fault' on Instance uuid 8712cdbc-97cd-492a-865d-81a3d542b7a3 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:30.833020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:30.840823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90c691af-25ac-4381-9a40-c50184b3718b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 200 len: 1317 microversion: 2.1 time: 0.769330 Oct 15 20:58:30.841465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1397/2511] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:30 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 1317 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:30.854029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d036b41d-3184-4311-bff7-e81d6982b8d3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edc5ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:30.939988 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d036b41d-3184-4311-bff7-e81d6982b8d3 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.089668 Oct 15 20:58:30.946444 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1398/2512] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:30 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:30.957348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:30.958188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:30.972592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:30.973513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:30.979311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:30.980416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:30.983229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:30.983867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:31.310259 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43485aa5-5b33-4d40-acdc-ad82a6ef1547 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.362312 Oct 15 20:58:31.311489 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1399/2513] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 20:58:30 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2/os-interface => generated 284 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:31.316280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "0.0.0.0"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:31.319904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:31.333549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:31.334066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:31.334493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:31.335485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:31.335670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:31.336083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:31.398538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 20:58:31.398772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquired lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 20:58:31.658936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Releasing lock "refresh_cache-ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 20:58:31.659751 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] HTTP exception thrown: floating IP not found Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Request method failure captured: Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action HTTP/1.1 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 41 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx66JOVrxpwugtcXazHYTtxH3c6dtmp_CEZKPkSziQRU6QQW05DhnQcZB0oeCGJ762xkIkP96GGY1t2123LVOQyDJTeNEbIryDADbMw92rukqgogMBfhT5MBdsTupzUnri85iRvLkY7IMFk5sXrsWGHIGmf6_Z4pAIlKVJdSISiRhts Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_11bdafc438884621944db399da5528f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/11bdafc438884621944db399da5528f8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 78d679a10c9e4bf2bc35786235aeaf2a Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: {"addFloatingIp": {"address": "0.0.0.0"}} Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: exception: floating IP not found Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8', 'body': {'addFloatingIp': {'address': '0.0.0.0'}}} Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 239, in _add_floating_ip Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.network_api.associate_floating_ip(context, instance, Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 165, in wrapper Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = f(self, context, *args, **kwargs) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2699, in associate_floating_ip Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fip = self._get_floating_ip_by_address(client, floating_address) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 3011, in _get_floating_ip_by_address Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFoundForAddress(address=address) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFoundForAddress: Floating IP not found for address 0.0.0.0. Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 252, in _add_floating_ip Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: floating IP not found Oct 15 20:58:31.661247 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: floating IP not found Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Returning 404 to user: floating IP not found {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3544a3ca-d8f6-489a-8df7-c12225f0e5e8 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action" status: 404 len: 67 microversion: 2.1 time: 0.350077 Oct 15 20:58:31.665888 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1400/2514] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:31 2022] POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action => generated 67 bytes in 351 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:58:31.673459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:31.674079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:31.683962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:31.684464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:31.684908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:31.685739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:31.686197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:31.697348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:32.214082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:32.222700 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c0842f4-604b-4cae-9a36-ebf8510da53b tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1315 microversion: 2.1 time: 0.558121 Oct 15 20:58:32.226206 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1401/2515] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:31 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1315 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:32.241563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:32.242615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:32.250628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:32.251354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:32.251968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:32.252854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:32.253394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:32.253827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:32.639701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:32.648208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ff3020ea-e0ca-49de-a5ad-d7b8cacaaf02 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1468 microversion: 2.1 time: 0.420166 Oct 15 20:58:32.677819 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1402/2516] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:32 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1468 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:32.704403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:32.705082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] [instance: 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:32.738232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:32.738724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:32.739161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:32.739879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:32.740333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:32.740757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:32.790821 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] HTTP exception thrown: Instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 could not be found. Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Request method failure captured: Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 HTTP/1.1 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx7H2r-AHe3h2S7RP4tUoB7gU7FKaSL6UClMzly0cSnNieodRCaW2OUk4PLqjIACNQAEe4GXllTr_eWQ72zGckzsgaejS5HXDZGvv_eJf3MrpyseKBUObujwZiDyHku4NRIT1_fXTdGEQtjY1iEU962zussq9UtdDOKEXs99IJ29hx4 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 974bd50eb06e453e9bb20218b9635555 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_974bd50eb06e453e9bb20218b9635555"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/974bd50eb06e453e9bb20218b9635555"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 974bd50eb06e453e9bb20218b9635555 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-1975732239 Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-InstanceActionsNegativeTestJSON-1975732239-project Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 79e244a9ce2343dbb903a4722da86eaf Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-1975732239-project Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 could not be found. Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6'} Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 could not be found. Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 could not be found. Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:32.799661 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 could not be found. Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] Returning 404 to user: Instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:32.804104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0de2b18-dce7-43ac-8d2c-164b6486ba16 tempest-InstanceActionsNegativeTestJSON-1975732239 tempest-InstanceActionsNegativeTestJSON-1975732239-project] 10.222.0.16 "GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6" status: 404 len: 111 microversion: 2.1 time: 0.122384 Oct 15 20:58:32.808535 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1403/2517] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:32 2022] GET /compute/v2.1/servers/560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 => generated 111 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:32.827555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5de5d888-fe7f-4e6f-b8d8-3de9fcf35854 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "0.0.0.0"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:33.053738 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5de5d888-fe7f-4e6f-b8d8-3de9fcf35854 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] HTTP exception thrown: Floating IP not found for address 0.0.0.0. Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5de5d888-fe7f-4e6f-b8d8-3de9fcf35854 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Request method failure captured: Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action HTTP/1.1 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 44 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx66JOVrxpwugtcXazHYTtxH3c6dtmp_CEZKPkSziQRU6QQW05DhnQcZB0oeCGJ762xkIkP96GGY1t2123LVOQyDJTeNEbIryDADbMw92rukqgogMBfhT5MBdsTupzUnri85iRvLkY7IMFk5sXrsWGHIGmf6_Z4pAIlKVJdSISiRhts Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_11bdafc438884621944db399da5528f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/11bdafc438884621944db399da5528f8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 78d679a10c9e4bf2bc35786235aeaf2a Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: {"removeFloatingIp": {"address": "0.0.0.0"}} Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP not found for address 0.0.0.0. Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8', 'body': {'removeFloatingIp': {'address': '0.0.0.0'}}} Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 58, in get_instance_by_floating_ip_addr Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.network_api.get_instance_id_by_floating_address( Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2927, in get_instance_id_by_floating_address Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fip = self._get_floating_ip_by_address(client, address) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 3011, in _get_floating_ip_by_address Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFoundForAddress(address=address) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFoundForAddress: Floating IP not found for address 0.0.0.0. Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 277, in _remove_floating_ip Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = get_instance_by_floating_ip_addr(self, context, address) Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 61, in get_instance_by_floating_ip_addr Oct 15 20:58:33.055624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=ex.format_message()) Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for address 0.0.0.0. Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for address 0.0.0.0. Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5de5d888-fe7f-4e6f-b8d8-3de9fcf35854 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Returning 404 to user: Floating IP not found for address 0.0.0.0. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:33.059136 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5de5d888-fe7f-4e6f-b8d8-3de9fcf35854 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action" status: 404 len: 88 microversion: 2.1 time: 0.247413 Oct 15 20:58:33.061609 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1404/2518] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:32 2022] POST /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8/action => generated 88 bytes in 252 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:58:33.314347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-39015b53-b660-433d-91c2-628df4d9b59f req-b7cef7b0-d4f7-40c4-91c9-5679c388e0d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15bdf694-ed96-4052-b284-784636eefd0a", "name": "network-vif-plugged", "status": "completed", "tag": "929e53c0-85c1-4e31-a351-d67f070c1486"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:33.329955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-39015b53-b660-433d-91c2-628df4d9b59f req-b7cef7b0-d4f7-40c4-91c9-5679c388e0d0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.330683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-39015b53-b660-433d-91c2-628df4d9b59f req-b7cef7b0-d4f7-40c4-91c9-5679c388e0d0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.331750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-39015b53-b660-433d-91c2-628df4d9b59f req-b7cef7b0-d4f7-40c4-91c9-5679c388e0d0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.348826 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-39015b53-b660-433d-91c2-628df4d9b59f req-b7cef7b0-d4f7-40c4-91c9-5679c388e0d0 service nova] Creating event network-vif-plugged:929e53c0-85c1-4e31-a351-d67f070c1486 for instance 15bdf694-ed96-4052-b284-784636eefd0a on nch1-master-1 Oct 15 20:58:33.356335 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-39015b53-b660-433d-91c2-628df4d9b59f req-b7cef7b0-d4f7-40c4-91c9-5679c388e0d0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.293457 Oct 15 20:58:33.358323 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1405/2519] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:33.383653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:33.385114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:33.394935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.396167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.396772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.398171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.399366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.399949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.434015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 HTTP/1.1 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8712cdbc-97cd-492a-865d-81a3d542b7a3'} Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:33.436042 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 20:58:33.440132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:33.443747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:33.444765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6f8e8e4d-caa7-47d0-b923-62c65f9ea227 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 404 len: 111 microversion: 2.1 time: 0.084541 Oct 15 20:58:33.446557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1406/2520] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:33 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:33.458894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8712cdbc-97cd-492a-865d-81a3d542b7a3", "name": "network-vif-deleted", "tag": "e58a1b34-85fe-4f04-bfa1-8ac563b675da"}, {"server_uuid": "560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6", "name": "network-vif-deleted", "tag": "6f4c02d0-4987-4f8a-a1e1-917c3c2665bd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:33.477718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.478492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.479073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.501083 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] Creating event network-vif-deleted:e58a1b34-85fe-4f04-bfa1-8ac563b675da for instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 on nch1-master-1 Oct 15 20:58:33.502080 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] Creating event network-vif-deleted:6f4c02d0-4987-4f8a-a1e1-917c3c2665bd for instance 560042dc-2ee5-4fdd-967d-ad0cc8aeb1b6 on nch1-master-1 Oct 15 20:58:33.515331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-28be41a4-0898-4fae-8754-7cac0c99efdb req-b9d9316b-98ac-4dbf-a6f7-9a51cd6f0a32 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.064770 Oct 15 20:58:33.516745 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1407/2521] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:33 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:33.523343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4318c569-cffd-400e-844e-c53bdb0f6c8d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edf3310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:33.582858 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4318c569-cffd-400e-844e-c53bdb0f6c8d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.062482 Oct 15 20:58:33.582858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1408/2522] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:33 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:33.590246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:33.590901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:33.609056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.609559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.612385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.612385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.612662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.613467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.652388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lazy-loading 'flavor' on Instance uuid ca7be734-48df-44ab-9b11-275028d8f513 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:33.759365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5998faee-cdf2-479d-956a-3b2ef6efa18f tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513/action" status: 202 len: 0 microversion: 2.1 time: 0.176020 Oct 15 20:58:33.760472 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1409/2523] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:33 2022] POST /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513/action => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:58:33.768027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:33.769605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:33.779164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.779917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.780556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:33.784469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:33.785134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:33.785635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:34.246817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:34.257278 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abe62ab7-f3ec-4924-b3e6-8a94f97121da tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1575 microversion: 2.1 time: 0.495438 Oct 15 20:58:34.276960 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1410/2524] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:33 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1575 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:34.369929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6daeeab4-42aa-46bf-856b-107b2ff33f67 req-65fa18bd-7aea-4012-9780-64f4fdcc00f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e970fbbc-4044-415f-833d-21f6bd4d33a2", "tag": "dba907e7-a00a-4b1c-813e-5237b472bb44"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:34.385226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6daeeab4-42aa-46bf-856b-107b2ff33f67 req-65fa18bd-7aea-4012-9780-64f4fdcc00f0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:34.385719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6daeeab4-42aa-46bf-856b-107b2ff33f67 req-65fa18bd-7aea-4012-9780-64f4fdcc00f0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:34.386159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6daeeab4-42aa-46bf-856b-107b2ff33f67 req-65fa18bd-7aea-4012-9780-64f4fdcc00f0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:34.406208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6daeeab4-42aa-46bf-856b-107b2ff33f67 req-65fa18bd-7aea-4012-9780-64f4fdcc00f0 service nova] Creating event network-changed:dba907e7-a00a-4b1c-813e-5237b472bb44 for instance e970fbbc-4044-415f-833d-21f6bd4d33a2 on nch1-master-1 Oct 15 20:58:34.430598 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6daeeab4-42aa-46bf-856b-107b2ff33f67 req-65fa18bd-7aea-4012-9780-64f4fdcc00f0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.157941 Oct 15 20:58:34.431230 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1411/2525] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:34.452773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:34.453449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:34.487178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:34.487750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:34.488272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:34.489390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:34.491690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:34.491690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:34.538095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:34.657551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=95,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:34.715339 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d2ef6d83-3b2c-49dc-b70c-09b7215cf9dd tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 204 len: 0 microversion: 2.1 time: 0.281333 Oct 15 20:58:34.716949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1412/2526] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:34 2022] DELETE /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:34.725514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:34.726224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:34.746457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:34.746968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:34.747405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:34.748689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:34.749184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:34.749616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:35.210270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:35.225459 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb7e0273-efac-4ce8-87f3-f2d5da2aa97d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1315 microversion: 2.1 time: 0.503918 Oct 15 20:58:35.226233 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1413/2527] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:34 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1315 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:35.243029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2a90925-089d-4f58-8156-0c8b3e07891b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '.version_select at 0x7fe74ee76b80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:35.274741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2a90925-089d-4f58-8156-0c8b3e07891b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-key-152352319" status: 202 len: 0 microversion: 2.1 time: 0.044621 Oct 15 20:58:35.278162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1414/2528] 10.222.0.16 () {58 vars in 1206 bytes} [Sat Oct 15 20:58:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-key-152352319 => generated 0 bytes in 51 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:58:35.288801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:35.291109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:35.309701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:35.310693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:35.311473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:35.313481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:35.314772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:35.315569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:35.805191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:35.821007 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-875a42f0-c532-44cf-8511-598579859ac9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1459 microversion: 2.1 time: 0.539738 Oct 15 20:58:35.822050 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1415/2529] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:35 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1459 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:35.827434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae56fb82-4f4d-4478-bf58-0943f3dfe98a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74eddcdc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:35.882214 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae56fb82-4f4d-4478-bf58-0943f3dfe98a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.058928 Oct 15 20:58:35.885086 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1416/2530] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:35 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:35.895688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:35.896315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:35.903779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:35.904561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:35.905013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:35.905756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:35.906221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:35.906656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:36.452691 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 20:58:36.453570 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 20:58:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 20:58:36.693736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:36.704654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4a5a5d8-a524-4729-8d17-28444a0b7d3a tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1541 microversion: 2.1 time: 0.820386 Oct 15 20:58:36.707323 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1417/2531] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:35 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1541 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:36.723837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-673450032", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:36.856220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:58:38.176369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:58:38.177532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:58:38.178531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:58:38.179683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:58:38.180450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:58:38.224781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:38.233557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:38.234277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:38.235291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:38.240059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:38.240748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:38.301077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:38.351473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:38.405233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:58:38.413249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:58:38.677536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e656466b-182a-4aaa-af3a-2ee083f91b93 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.967376 Oct 15 20:58:38.678783 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1418/2532] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 1971 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:58:38.683539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:38.685602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:38.694109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:38.695052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:38.695705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:38.697134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:38.697812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:38.698430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:39.187361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fdc5dca2-db04-4518-aa6c-db40ecb3f850 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1610 microversion: 2.1 time: 0.507322 Oct 15 20:58:39.201353 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1419/2533] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:38 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1610 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:39.227754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:39.229377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:39.260941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:39.261472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:39.264671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:39.265651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:39.267571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:39.268195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:39.274156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:39.571124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:39.581057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccdb0a75-7669-43b2-b62b-d87687035e89 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1502 microversion: 2.1 time: 0.378407 Oct 15 20:58:39.595479 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1420/2534] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:39 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1502 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:39.600447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3136fa06-30ba-4e12-83b1-ce08c3aa99d7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edae4c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:39.672365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3136fa06-30ba-4e12-83b1-ce08c3aa99d7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1440 microversion: 2.1 time: 0.075485 Oct 15 20:58:39.684148 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1421/2535] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:39 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1440 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:39.697871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:39.701159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:39.725303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:39.726935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:39.727394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:39.728134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:39.728596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:39.731313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:39.846488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:39.847035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:39.847492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:39.847962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lazy-loading 'fault' on Instance uuid ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:39.970325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78d679a10c9e4bf2bc35786235aeaf2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '11bdafc438884621944db399da5528f8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:39.984684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cfb241bb-c3aa-4fb1-b5e3-badf469ed145 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 200 len: 1322 microversion: 2.1 time: 0.299876 Oct 15 20:58:40.006195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1422/2536] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:39 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 1322 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:40.141960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-88fed8ff-1318-454f-a011-b636e473ebb0 req-ad92edf3-95b8-4135-99fc-a3658e06060d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8", "name": "network-vif-deleted", "tag": "48fb08ef-453d-47d0-90ba-6f116d9f1cda"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:40.166577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-88fed8ff-1318-454f-a011-b636e473ebb0 req-ad92edf3-95b8-4135-99fc-a3658e06060d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.170457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-88fed8ff-1318-454f-a011-b636e473ebb0 req-ad92edf3-95b8-4135-99fc-a3658e06060d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.173430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-88fed8ff-1318-454f-a011-b636e473ebb0 req-ad92edf3-95b8-4135-99fc-a3658e06060d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.206806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-88fed8ff-1318-454f-a011-b636e473ebb0 req-ad92edf3-95b8-4135-99fc-a3658e06060d service nova] Creating event network-vif-deleted:48fb08ef-453d-47d0-90ba-6f116d9f1cda for instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 on nch2-master-1 Oct 15 20:58:40.222470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-88fed8ff-1318-454f-a011-b636e473ebb0 req-ad92edf3-95b8-4135-99fc-a3658e06060d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.214776 Oct 15 20:58:40.228200 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1423/2537] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:40.250007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:40.255816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:40.268214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.269010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.269670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.271181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.271920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.272551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.455257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:40.464172 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f13ccfa0-014f-4f9c-af8f-d9f606e0e35e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1204 microversion: 2.1 time: 0.234406 Oct 15 20:58:40.469956 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1424/2538] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:40 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1204 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:40.496089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:40.510341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.512135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.514300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.517236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.519482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.520059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.560551 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6c3cc88-d598-4431-b032-74a3b89dcb43 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.075709 Oct 15 20:58:40.568821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1425/2539] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 20:58:40 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:40.568821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69b94df0-ec9b-4593-8429-d29181a08d1d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:40.575430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69b94df0-ec9b-4593-8429-d29181a08d1d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.576771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69b94df0-ec9b-4593-8429-d29181a08d1d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.577305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69b94df0-ec9b-4593-8429-d29181a08d1d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.607443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69b94df0-ec9b-4593-8429-d29181a08d1d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 20:58:40.678246 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69b94df0-ec9b-4593-8429-d29181a08d1d tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a/action" status: 202 len: 0 microversion: 2.1 time: 0.114400 Oct 15 20:58:40.681871 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1426/2540] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:40 2022] POST /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a/action => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:58:40.687847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:40.692113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:40.705765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.706678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.707331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.710256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.710895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.711585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.745824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Lazy-loading 'flavor' on Instance uuid ca7be734-48df-44ab-9b11-275028d8f513 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:40.763427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] HTTP exception thrown: Cannot 'os-migrateLive' instance ca7be734-48df-44ab-9b11-275028d8f513 while it is in vm_state suspended Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Request method failure captured: Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513/action HTTP/1.1 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 97 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx61rQpQ7cWwAUg0hYyLNHjZvbjOg7S13V8ZQJys9Z2x7JC4hGJaLnlRA2r0gDNomc6qMC5R19LJSI2GNceqoP3FO9gY-gpIdDOexPvliLQyZD288wrZkWTbUq3D-Ji03ZUsD7E4QzipG2PaMYtc8PNo5KnN2mZ42qmgSRiVBNje8sw Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: af0b8f89094f42fb9ae9829807d9f6d4 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationNegativeTest-1766745771 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/af0b8f89094f42fb9ae9829807d9f6d4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationNegativeTest-1766745771 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: af0b8f89094f42fb9ae9829807d9f6d4 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1766745771 Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationNegativeTest-1766745771-project-admin Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 623abfad80794a058a678b8b50009e3e Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationNegativeTest-1766745771-project-admin Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'os-migrateLive' instance ca7be734-48df-44ab-9b11-275028d8f513 while it is in vm_state suspended Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ca7be734-48df-44ab-9b11-275028d8f513', 'body': {'os-migrateLive': {'host': 'nch2-master-1', 'block_migration': True, 'disk_over_commit': False}}} Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 123, in _migrate_live Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.live_migrate(context, instance, block_migration, Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 259, in inner Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance ca7be734-48df-44ab-9b11-275028d8f513 in vm_state suspended. Cannot live_migrate while the instance is in this state. Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:40.765778 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 152, in _migrate_live Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'os-migrateLive' instance ca7be734-48df-44ab-9b11-275028d8f513 while it is in vm_state suspended Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'os-migrateLive' instance ca7be734-48df-44ab-9b11-275028d8f513 while it is in vm_state suspended Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:40.769249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] Returning 409 to user: Cannot 'os-migrateLive' instance ca7be734-48df-44ab-9b11-275028d8f513 while it is in vm_state suspended {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:40.781081 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ff0b932-89ed-4a4c-9825-b501cf172367 tempest-LiveMigrationNegativeTest-1766745771 tempest-LiveMigrationNegativeTest-1766745771-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513/action" status: 409 len: 155 microversion: 2.1 time: 0.099687 Oct 15 20:58:40.788242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1427/2541] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:58:40 2022] POST /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513/action => generated 155 bytes in 104 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:58:40.792042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:40.792659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:40.817824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.820764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.821286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:40.822140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:40.822678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:40.823395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:41.081381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:41.098204 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-285bfadf-569e-4bd1-b8c8-8d62256316f4 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.312273 Oct 15 20:58:41.109117 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1428/2542] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:40 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:41.116995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b718c8ae-a30a-425d-ac5c-88eb6401b231 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edae160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:41.194875 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b718c8ae-a30a-425d-ac5c-88eb6401b231 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1449 microversion: 2.1 time: 0.088149 Oct 15 20:58:41.194875 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1429/2543] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:41 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1449 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:41.205146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:41.209023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:41.229845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:41.230487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:41.231026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:41.232353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:41.232888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:41.233403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:41.307618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:41.482329 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f922ad48-7bfc-4aff-bb81-7e83471e0df5 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 204 len: 0 microversion: 2.1 time: 0.285193 Oct 15 20:58:41.489169 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1430/2544] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:41 2022] DELETE /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:41.496299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:41.497137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] [instance: ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:41.504611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:41.505123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:41.505710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:41.506535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:41.507017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:41.507531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:41.531675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] HTTP exception thrown: Instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 could not be found. Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Request method failure captured: Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 HTTP/1.1 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx66JOVrxpwugtcXazHYTtxH3c6dtmp_CEZKPkSziQRU6QQW05DhnQcZB0oeCGJ762xkIkP96GGY1t2123LVOQyDJTeNEbIryDADbMw92rukqgogMBfhT5MBdsTupzUnri85iRvLkY7IMFk5sXrsWGHIGmf6_Z4pAIlKVJdSISiRhts Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_11bdafc438884621944db399da5528f8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/11bdafc438884621944db399da5528f8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/11bdafc438884621944db399da5528f8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 11bdafc438884621944db399da5528f8 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105 Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 78d679a10c9e4bf2bc35786235aeaf2a Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 could not be found. Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8'} Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 could not be found. Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 could not be found. Oct 15 20:58:41.535942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 could not be found. Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] Returning 404 to user: Instance ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:41.540399 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b985d37-8c56-4f6e-8c94-8e6f7a9f9a76 tempest-FloatingIPsAssociationNegativeTestJSON-30052105 tempest-FloatingIPsAssociationNegativeTestJSON-30052105-project] 10.222.0.16 "GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8" status: 404 len: 111 microversion: 2.1 time: 0.046764 Oct 15 20:58:41.544768 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1431/2545] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:41 2022] GET /compute/v2.1/servers/ed1617cd-c6fc-4c5f-a27c-d11810dfcbc8 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:41.550699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0f63d4a-c857-4edd-8583-7b38b1a622cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ee42790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:41.608964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0f63d4a-c857-4edd-8583-7b38b1a622cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 200 len: 1449 microversion: 2.1 time: 0.062652 Oct 15 20:58:41.610081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1432/2546] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:58:41 2022] GET /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 1449 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:41.682176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2a5c5ad-4aee-4318-b95a-aabb3a1183f9 tempest-FloatingIPDetailsNegativeTestJSON-1218605304 tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project] Calling method '.version_select at 0x7fe74edaef70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:41.756126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-a2a5c5ad-4aee-4318-b95a-aabb3a1183f9 tempest-FloatingIPDetailsNegativeTestJSON-1218605304 tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP 00795cb7-11ee-425e-b78e-f7bbab14f04a could not be found", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:58:41.757102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a2a5c5ad-4aee-4318-b95a-aabb3a1183f9 tempest-FloatingIPDetailsNegativeTestJSON-1218605304 tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project] HTTP exception thrown: Floating IP not found for ID 00795cb7-11ee-425e-b78e-f7bbab14f04a Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2a5c5ad-4aee-4318-b95a-aabb3a1183f9 tempest-FloatingIPDetailsNegativeTestJSON-1218605304 tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project] Request method failure captured: Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-floating-ips/00795cb7-11ee-425e-b78e-f7bbab14f04a HTTP/1.1 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Bmt9E66frdR_n7P0Dd-HFra4At9BtblxSpzmY3n4DxUfD1Ik31-MMztUjYU4ywxfa8GYDN2ZRTB5yPeYZKMKE8imcMZjoaqFQMGNACYYzKvXAQ0qHCFII0_bYWSk6wmJU7xbXPss7wiiiVmA6kMriSpxDzbvQi1tg8HawCsKf7j4 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: bccddd29eb9d4df692e1599e63daffc5 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPDetailsNegativeTestJSON-1218605304 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bccddd29eb9d4df692e1599e63daffc5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bccddd29eb9d4df692e1599e63daffc5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bccddd29eb9d4df692e1599e63daffc5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bccddd29eb9d4df692e1599e63daffc5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPDetailsNegativeTestJSON-1218605304 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: bccddd29eb9d4df692e1599e63daffc5 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPDetailsNegativeTestJSON-1218605304 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: eb0650ccb0ab4d17b795db9e18a6a160 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP not found for ID 00795cb7-11ee-425e-b78e-f7bbab14f04a Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '00795cb7-11ee-425e-b78e-f7bbab14f04a'} Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:58:41.760844 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP 00795cb7-11ee-425e-b78e-f7bbab14f04a could not be found Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c9a8a97d-6411-4628-b903-1a5da964c593'] Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 94, in show Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID 00795cb7-11ee-425e-b78e-f7bbab14f04a. Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 97, in show Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID 00795cb7-11ee-425e-b78e-f7bbab14f04a Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID 00795cb7-11ee-425e-b78e-f7bbab14f04a Oct 15 20:58:41.766011 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:41.770361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2a5c5ad-4aee-4318-b95a-aabb3a1183f9 tempest-FloatingIPDetailsNegativeTestJSON-1218605304 tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project] Returning 404 to user: Floating IP not found for ID 00795cb7-11ee-425e-b78e-f7bbab14f04a {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:41.771420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2a5c5ad-4aee-4318-b95a-aabb3a1183f9 tempest-FloatingIPDetailsNegativeTestJSON-1218605304 tempest-FloatingIPDetailsNegativeTestJSON-1218605304-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/00795cb7-11ee-425e-b78e-f7bbab14f04a" status: 404 len: 111 microversion: 2.1 time: 0.160602 Oct 15 20:58:41.772691 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1433/2547] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:58:41 2022] GET /compute/v2.1/os-floating-ips/00795cb7-11ee-425e-b78e-f7bbab14f04a => generated 111 bytes in 162 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:41.782316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:41.783105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:41.791792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:41.792679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:41.793320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:41.794201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:41.794869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:41.795598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:42.273373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:42.283470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e632be5-a57f-45b2-8622-ed881d03c79f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1266 microversion: 2.1 time: 0.509256 Oct 15 20:58:42.284274 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1434/2548] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:41 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1266 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:42.306843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:42.307722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:42.317171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:42.318039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:42.319259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:42.320146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:42.320767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:42.321386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:42.790448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:42.800637 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d32f95e-9bc8-4fe8-bd44-07c685537957 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1465 microversion: 2.1 time: 0.500902 Oct 15 20:58:42.801949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1435/2549] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:42 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1465 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:42.817338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b7a3ec0d-dc1a-44c9-9f95-04b91ac74bc5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74edf33a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:43.733336 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b7a3ec0d-dc1a-44c9-9f95-04b91ac74bc5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 204 len: 0 microversion: 2.1 time: 0.929883 Oct 15 20:58:43.737766 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1436/2550] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:42 2022] DELETE /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 0 bytes in 931 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:43.745682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:43.746858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:43.765790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:43.767006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:43.769568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:43.770777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:43.771043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:43.773524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:44.535154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:44.545073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-29c0c6b9-26fd-497e-bda6-2e0306cfffb9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.809005 Oct 15 20:58:44.549901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1437/2551] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:43 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 815 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:44.558013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:44.561729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:44.586008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:44.588104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:44.593682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:44.600489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:44.603568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:44.604221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:45.880963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:45.901225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3eca0e98-c53c-4b36-a0b2-cc66d94b981e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1308 microversion: 2.1 time: 1.349963 Oct 15 20:58:45.938056 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1438/2552] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:44 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1308 bytes in 1387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:45.955935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:45.960535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:45.976539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:45.977781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:45.978942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:45.982675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:45.985136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:45.986940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:46.049312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:46.112451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:46.207956 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bf54de48-6613-4989-bee0-361459d02aff tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 204 len: 0 microversion: 2.1 time: 0.263888 Oct 15 20:58:46.211197 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1439/2553] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:45 2022] DELETE /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:46.251701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:46.252509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:46.260494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:46.261084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:46.261450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:46.262171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:46.262624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:46.270099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:46.358381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:46.358869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:46.361848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:46.362489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lazy-loading 'fault' on Instance uuid ca7be734-48df-44ab-9b11-275028d8f513 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:58:46.502917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:46.515199 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-806332ce-47af-4a46-b57f-884dde6a0153 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 200 len: 1308 microversion: 2.1 time: 0.298618 Oct 15 20:58:46.518830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1440/2554] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:46 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 1308 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:46.529145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:46.529764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:46.584125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:46.584637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:46.596140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:46.597474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:46.598177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:46.598993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:47.142217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:47.165123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fbceb3dc-b67b-455b-82d8-f0362315d674 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.636494 Oct 15 20:58:47.165731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1441/2555] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:46 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:47.180865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7d6ed4a1-daa6-4fad-9204-a1a81b414dbc req-1923a03b-9cce-4230-87ce-c1ea5914f001 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ca7be734-48df-44ab-9b11-275028d8f513", "name": "network-vif-deleted", "tag": "c84eb606-33ff-47f4-9f41-040ccf41d631"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:47.204031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d6ed4a1-daa6-4fad-9204-a1a81b414dbc req-1923a03b-9cce-4230-87ce-c1ea5914f001 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:47.204868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d6ed4a1-daa6-4fad-9204-a1a81b414dbc req-1923a03b-9cce-4230-87ce-c1ea5914f001 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:47.205660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d6ed4a1-daa6-4fad-9204-a1a81b414dbc req-1923a03b-9cce-4230-87ce-c1ea5914f001 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:47.227990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7d6ed4a1-daa6-4fad-9204-a1a81b414dbc req-1923a03b-9cce-4230-87ce-c1ea5914f001 service nova] Creating event network-vif-deleted:c84eb606-33ff-47f4-9f41-040ccf41d631 for instance ca7be734-48df-44ab-9b11-275028d8f513 on nch1-master-1 Oct 15 20:58:47.243102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7d6ed4a1-daa6-4fad-9204-a1a81b414dbc req-1923a03b-9cce-4230-87ce-c1ea5914f001 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075547 Oct 15 20:58:47.244081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1442/2556] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:47.253547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-1180940646", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c130cb51-e60d-43b7-af80-10e77fbe9595"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:47.363310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] validate_networks() for [('c130cb51-e60d-43b7-af80-10e77fbe9595', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:58:48.369782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:49.092178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:58:49.093637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:58:49.094589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:58:49.095382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:58:49.096153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:58:49.130032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:49.132075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:49.135935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:49.140024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:49.141252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:49.141882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:49.172976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:49.183526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for user 5d93a680570f4d23b1f6df534bc082b2 and project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:49.212459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:58:49.220235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:58:49.242638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=3,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:49.320508 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd92dc26-20d1-4a84-b18b-01de61c73ecc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.073229 Oct 15 20:58:49.323338 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1443/2557] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 2079 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:58:49.333722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:49.335624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:49.345161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:49.345959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:49.346644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:49.353275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:49.354265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:49.356040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:49.909477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:49.920674 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a52e0692-75fc-4f52-9d8c-5253ce15396f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1308 microversion: 2.1 time: 0.593848 Oct 15 20:58:49.924024 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1444/2558] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:49 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1308 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:49.939624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9be31845-fae3-47e6-88a7-52d0e7fcd207 req-6500015e-d516-494f-bfe5-784581706f0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e14662ce-cafb-4dcf-add1-d49193e3ed0f", "name": "network-vif-unplugged", "status": "completed", "tag": "15c191de-11bf-42b3-b218-d8913de071bc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:49.967545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9be31845-fae3-47e6-88a7-52d0e7fcd207 req-6500015e-d516-494f-bfe5-784581706f0f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:49.970752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9be31845-fae3-47e6-88a7-52d0e7fcd207 req-6500015e-d516-494f-bfe5-784581706f0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:49.971371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9be31845-fae3-47e6-88a7-52d0e7fcd207 req-6500015e-d516-494f-bfe5-784581706f0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.002438 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9be31845-fae3-47e6-88a7-52d0e7fcd207 req-6500015e-d516-494f-bfe5-784581706f0f service nova] Creating event network-vif-unplugged:15c191de-11bf-42b3-b218-d8913de071bc for instance e14662ce-cafb-4dcf-add1-d49193e3ed0f on nch2-master-1 Oct 15 20:58:50.021553 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9be31845-fae3-47e6-88a7-52d0e7fcd207 req-6500015e-d516-494f-bfe5-784581706f0f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.093210 Oct 15 20:58:50.022746 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1445/2559] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:49 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:50.031815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:50.037089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: ca7be734-48df-44ab-9b11-275028d8f513] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:50.053566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.054342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.054956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.055950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.056877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.057622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.081614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] HTTP exception thrown: Instance ca7be734-48df-44ab-9b11-275028d8f513 could not be found. Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Request method failure captured: Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 HTTP/1.1 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6033dUYqOQD15Tawz1vYlA-F5RsXBYArI13IHh55ZlBQNokveysQHkdSLx5Gr6LsGxWNOY9321QVus20adlsSoeryThGv079LQPIHAQJR0tlOu_6JJ0FmSPbt96vVEr4sgPISvRdyHyv_1xvjMEHemZ9yByK1M-Xg6nQfxP5x4Do0 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c44841db4e33412e805da46ba4262fe7 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationNegativeTest-1484496353 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c44841db4e33412e805da46ba4262fe7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c44841db4e33412e805da46ba4262fe7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c44841db4e33412e805da46ba4262fe7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c44841db4e33412e805da46ba4262fe7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationNegativeTest-1484496353 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c44841db4e33412e805da46ba4262fe7 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1484496353 Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationNegativeTest-1484496353-project Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 500cba6141d34c4081d38a7b1be1775e Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationNegativeTest-1484496353-project Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ca7be734-48df-44ab-9b11-275028d8f513 could not be found. Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ca7be734-48df-44ab-9b11-275028d8f513'} Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ca7be734-48df-44ab-9b11-275028d8f513 could not be found. Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ca7be734-48df-44ab-9b11-275028d8f513 could not be found. Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:50.083348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ca7be734-48df-44ab-9b11-275028d8f513 could not be found. Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Returning 404 to user: Instance ca7be734-48df-44ab-9b11-275028d8f513 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:50.087940 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13828132-6d36-4b14-ad85-e3edefea1f62 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513" status: 404 len: 111 microversion: 2.1 time: 0.060803 Oct 15 20:58:50.089695 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1446/2560] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:50 2022] GET /compute/v2.1/servers/ca7be734-48df-44ab-9b11-275028d8f513 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:50.095178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:50.096090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:50.102749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.104765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.105499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.106517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.107391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.107983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.650915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:50.662469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-85c33eb3-cd55-4c47-9192-708b6da0f254 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.571657 Oct 15 20:58:50.665734 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1447/2561] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:50 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:50.670873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:50.671678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:50.687242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.687923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.689181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.690065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.690750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.691599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.828048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:50.839455 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82bfb987-3606-4cd4-9400-6fbca131aa8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 200 len: 1260 microversion: 2.1 time: 0.172396 Oct 15 20:58:50.840355 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1448/2562] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:50 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 1260 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:50.849596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3f7266c7-86b8-41f9-a37e-81b3012fec2e req-4e90cf50-a0fc-41bb-a9df-96064de85345 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "87c2e279-0350-4f47-b7f3-68c0054c0100", "tag": "731071c0-c8ed-432e-a1d7-02c713022ccb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:50.879967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3f7266c7-86b8-41f9-a37e-81b3012fec2e req-4e90cf50-a0fc-41bb-a9df-96064de85345 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:50.880563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3f7266c7-86b8-41f9-a37e-81b3012fec2e req-4e90cf50-a0fc-41bb-a9df-96064de85345 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:50.881962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3f7266c7-86b8-41f9-a37e-81b3012fec2e req-4e90cf50-a0fc-41bb-a9df-96064de85345 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:50.927018 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3f7266c7-86b8-41f9-a37e-81b3012fec2e req-4e90cf50-a0fc-41bb-a9df-96064de85345 service nova] Creating event network-changed:731071c0-c8ed-432e-a1d7-02c713022ccb for instance 87c2e279-0350-4f47-b7f3-68c0054c0100 on nch2-master-1 Oct 15 20:58:50.941618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3f7266c7-86b8-41f9-a37e-81b3012fec2e req-4e90cf50-a0fc-41bb-a9df-96064de85345 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092262 Oct 15 20:58:50.965856 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1449/2563] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:50.983189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:50.987305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:51.014640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:51.015232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:51.018017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:51.018539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:51.022810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:51.023277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:51.089215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:58:51.215788 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63b50ade-e38c-45b9-a6d9-6f972aed91e9 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 204 len: 0 microversion: 2.1 time: 0.246991 Oct 15 20:58:51.219639 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1450/2564] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:58:50 2022] DELETE /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:58:51.225202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:51.226153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:51.245743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:51.248316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:51.250375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:51.251436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:51.253185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:51.258308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:52.108415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:52.126446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97c2c875-2624-4180-983a-0144ab04a55d tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1308 microversion: 2.1 time: 0.908173 Oct 15 20:58:52.132971 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1451/2565] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:51 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1308 bytes in 915 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:52.145008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:52.146310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:52.160496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:52.161362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:52.161970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:52.162872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:52.163491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:52.164902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:52.786593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:52.802199 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-037bd689-c5b9-4deb-a4ef-c9c3ced51658 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1474 microversion: 2.1 time: 0.661828 Oct 15 20:58:52.803225 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1452/2566] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:52 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1474 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:52.817273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:52.818205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:52.839628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:52.840305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:52.841090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:52.842316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:52.843518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:52.848089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:54.002606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:54.030040 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0474c33-d4bb-4362-82fd-9151610a05b1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 1.223949 Oct 15 20:58:54.032326 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1453/2567] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:52 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 1228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:54.051106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:54.057937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:54.066835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:54.069578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:54.070301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:54.073427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:54.074709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:54.078490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:55.262365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:55.277701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57a9e541-2f9e-4cfd-872d-cd7a4394895e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 200 len: 1266 microversion: 2.1 time: 1.242372 Oct 15 20:58:55.278302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1454/2568] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:54 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 1266 bytes in 1245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:55.283457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-98419490-ae6f-4f7e-9684-36644803de9c req-9fa77328-61d8-4fc9-b2f8-b05411befdff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e14662ce-cafb-4dcf-add1-d49193e3ed0f", "name": "network-vif-deleted", "tag": "15c191de-11bf-42b3-b218-d8913de071bc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:55.315628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-98419490-ae6f-4f7e-9684-36644803de9c req-9fa77328-61d8-4fc9-b2f8-b05411befdff service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:55.317550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-98419490-ae6f-4f7e-9684-36644803de9c req-9fa77328-61d8-4fc9-b2f8-b05411befdff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:55.319835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-98419490-ae6f-4f7e-9684-36644803de9c req-9fa77328-61d8-4fc9-b2f8-b05411befdff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:55.348791 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-98419490-ae6f-4f7e-9684-36644803de9c req-9fa77328-61d8-4fc9-b2f8-b05411befdff service nova] Creating event network-vif-deleted:15c191de-11bf-42b3-b218-d8913de071bc for instance e14662ce-cafb-4dcf-add1-d49193e3ed0f on nch2-master-1 Oct 15 20:58:55.363792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-98419490-ae6f-4f7e-9684-36644803de9c req-9fa77328-61d8-4fc9-b2f8-b05411befdff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079619 Oct 15 20:58:55.368209 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1455/2569] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:55.380350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:55.382384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:55.391609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:55.393441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:55.394272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:55.395658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:55.396978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:55.398256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:56.166524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:56.182557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97f4a99e-ea97-4e3b-8ce8-44188e6ce6d1 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1308 microversion: 2.1 time: 0.808519 Oct 15 20:58:56.183441 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1456/2570] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:55 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1308 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:56.206360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:56.207345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:56.216723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:56.218223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:56.219787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:56.221003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:56.222066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:56.222955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:56.389794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '500cba6141d34c4081d38a7b1be1775e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c44841db4e33412e805da46ba4262fe7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:56.394039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:58:56.406609 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82a6d215-ba0b-42a5-bc4b-87d735198a3f tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 200 len: 1432 microversion: 2.1 time: 0.221853 Oct 15 20:58:56.407200 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1457/2571] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:56 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 1432 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:56.415788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:56.416405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:56.435568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:56.436169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:56.436619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:56.437373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:56.437834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:56.438260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:57.022762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:57.032806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0aec4855-e955-43a5-a79c-76800ec16750 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.624539 Oct 15 20:58:57.051812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1458/2572] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:56 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:57.332907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerTestJSON-server-1042653605", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:57.427357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:58:58.045832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:58:58.046133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:58:58.046617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:58:58.047027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:58:58.047770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:58:58.069666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:58.070163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:58.070603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:58.071851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:58.071851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:58.072206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:58.110268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Getting quotas for project 57ac50a5098942e58c8ab72ff09d5159. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:58:58.132057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Getting quotas for user 83d85ef4ebdb452a93f24682e2bbd46c and project 57ac50a5098942e58c8ab72ff09d5159. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:58:58.163420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:58:58.166088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:58:58.257565 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e2e1db0-33ef-4824-ba23-e96fc3b3179d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.204071 Oct 15 20:58:58.266678 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1459/2573] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 1209 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:58:58.275114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bc9ea48c-7657-4569-837d-80d371c01532 req-603a5006-c2df-472c-9f5e-cebcb7973ba5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "87c2e279-0350-4f47-b7f3-68c0054c0100", "name": "network-vif-plugged", "status": "completed", "tag": "731071c0-c8ed-432e-a1d7-02c713022ccb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:58.305006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bc9ea48c-7657-4569-837d-80d371c01532 req-603a5006-c2df-472c-9f5e-cebcb7973ba5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:58.309321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bc9ea48c-7657-4569-837d-80d371c01532 req-603a5006-c2df-472c-9f5e-cebcb7973ba5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:58.309862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bc9ea48c-7657-4569-837d-80d371c01532 req-603a5006-c2df-472c-9f5e-cebcb7973ba5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:58.344093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-bc9ea48c-7657-4569-837d-80d371c01532 req-603a5006-c2df-472c-9f5e-cebcb7973ba5 service nova] Creating event network-vif-plugged:731071c0-c8ed-432e-a1d7-02c713022ccb for instance 87c2e279-0350-4f47-b7f3-68c0054c0100 on nch2-master-1 Oct 15 20:58:58.350098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bc9ea48c-7657-4569-837d-80d371c01532 req-603a5006-c2df-472c-9f5e-cebcb7973ba5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084421 Oct 15 20:58:58.350925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1460/2574] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:58.356231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:58.360660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:58.383502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:58.385316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:58.388841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:58.398179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:58.407535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:58.408248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:59.008225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:59.019143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ba6e864-69bf-4d33-b9da-86fd254fea66 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 200 len: 1308 microversion: 2.1 time: 0.664689 Oct 15 20:58:59.021346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1461/2575] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:58 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 1308 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:59.029028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:59.029996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:59.042208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:59.042979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:59.043733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:59.045325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:59.046031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:59.046663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:59.378123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:58:59.387468 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38e68f18-93f0-4c40-8540-22363f68913f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1308 microversion: 2.1 time: 0.364656 Oct 15 20:58:59.391355 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1462/2576] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:59 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1308 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:58:59.412018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:58:59.412635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] [instance: a708d875-0764-45ee-9b65-d5c576abd402] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:58:59.432870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:59.433455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:59.434113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:59.436766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:59.457737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:59.458492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:59.480209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] HTTP exception thrown: Instance a708d875-0764-45ee-9b65-d5c576abd402 could not be found. Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Request method failure captured: Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 HTTP/1.1 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx6033dUYqOQD15Tawz1vYlA-F5RsXBYArI13IHh55ZlBQNokveysQHkdSLx5Gr6LsGxWNOY9321QVus20adlsSoeryThGv079LQPIHAQJR0tlOu_6JJ0FmSPbt96vVEr4sgPISvRdyHyv_1xvjMEHemZ9yByK1M-Xg6nQfxP5x4Do0 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c44841db4e33412e805da46ba4262fe7 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-LiveMigrationNegativeTest-1484496353 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c44841db4e33412e805da46ba4262fe7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c44841db4e33412e805da46ba4262fe7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c44841db4e33412e805da46ba4262fe7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c44841db4e33412e805da46ba4262fe7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-LiveMigrationNegativeTest-1484496353 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c44841db4e33412e805da46ba4262fe7 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1484496353 Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-LiveMigrationNegativeTest-1484496353-project Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 500cba6141d34c4081d38a7b1be1775e Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-LiveMigrationNegativeTest-1484496353-project Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance a708d875-0764-45ee-9b65-d5c576abd402 could not be found. Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'a708d875-0764-45ee-9b65-d5c576abd402'} Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a708d875-0764-45ee-9b65-d5c576abd402 could not be found. Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a708d875-0764-45ee-9b65-d5c576abd402 could not be found. Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:58:59.482407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a708d875-0764-45ee-9b65-d5c576abd402 could not be found. Oct 15 20:58:59.486990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:58:59.489095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] Returning 404 to user: Instance a708d875-0764-45ee-9b65-d5c576abd402 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:58:59.491925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f2845fe-0679-4bf6-96fc-335893ed1190 tempest-LiveMigrationNegativeTest-1484496353 tempest-LiveMigrationNegativeTest-1484496353-project] 10.222.0.16 "GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402" status: 404 len: 111 microversion: 2.1 time: 0.083838 Oct 15 20:58:59.492683 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1463/2577] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:58:59 2022] GET /compute/v2.1/servers/a708d875-0764-45ee-9b65-d5c576abd402 => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:58:59.504602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-aaed5f17-737c-4364-b7bc-3b0fec08089d req-d03f0f6c-8381-4160-96b4-f05ed1d76eb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a708d875-0764-45ee-9b65-d5c576abd402", "name": "network-vif-deleted", "tag": "5570eac9-7f3b-4f14-8252-089d27fe947b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:59.534891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aaed5f17-737c-4364-b7bc-3b0fec08089d req-d03f0f6c-8381-4160-96b4-f05ed1d76eb2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:58:59.535556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aaed5f17-737c-4364-b7bc-3b0fec08089d req-d03f0f6c-8381-4160-96b4-f05ed1d76eb2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:58:59.536215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aaed5f17-737c-4364-b7bc-3b0fec08089d req-d03f0f6c-8381-4160-96b4-f05ed1d76eb2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:58:59.558875 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-aaed5f17-737c-4364-b7bc-3b0fec08089d req-d03f0f6c-8381-4160-96b4-f05ed1d76eb2 service nova] Creating event network-vif-deleted:5570eac9-7f3b-4f14-8252-089d27fe947b for instance a708d875-0764-45ee-9b65-d5c576abd402 on nch2-master-1 Oct 15 20:58:59.568402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-aaed5f17-737c-4364-b7bc-3b0fec08089d req-d03f0f6c-8381-4160-96b4-f05ed1d76eb2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072301 Oct 15 20:58:59.578152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1464/2578] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:58:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:58:59.752646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServersNegativeTestJSON-server-835215565", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 3, "networks": [{"uuid": "9f9c8d6c-98f9-403d-8d3f-b8765ccc792f"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:58:59.875890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] validate_networks() for [('9f9c8d6c-98f9-403d-8d3f-b8765ccc792f', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:59:00.679596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:00.680768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:00.681855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:00.683833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:00.685332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:00.726431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:00.729542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:00.731608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:00.738158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:00.739852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:00.742713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:00.780933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Getting quotas for project 8d37935df706485388d6137dff1faba5. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:00.787135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Getting quotas for user 6c8c93cfed5140b183cc0e3a559b1ce7 and project 8d37935df706485388d6137dff1faba5. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:00.808609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Going to run 3 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:59:00.846372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:00.912585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=183834bb-a5a3-43c7-a9c4-20c50652331d,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:01.039040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:01.224270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ebf635c-b9e8-4d40-9744-dbfcadddd09e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.644464 Oct 15 20:59:01.232372 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1465/2579] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:58:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 1653 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:59:01.238814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:01.239679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:01.248146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:01.248910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:01.249555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:01.250448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:01.251068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:01.251970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:01.513594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:01.539542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33e25b18-b80d-43f2-8b93-921e6c315845 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.304840 Oct 15 20:59:01.557772 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1466/2580] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:01 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:01.571661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:01.572290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:01.585563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:01.587066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:01.587600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:01.588653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:01.589837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:01.590667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:01.771428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:01.781780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d3b56a9-ea9d-4ad9-828f-f00871a27993 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1275 microversion: 2.1 time: 0.222524 Oct 15 20:59:01.810247 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1467/2581] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:01 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1275 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:01.826208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:01.826960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:01.835164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:01.835667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:01.836113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:01.836826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:01.840518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:01.842633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:02.202283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:02.213840 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44981aeb-6764-48fa-be2b-5175e8a9f150 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 200 len: 1308 microversion: 2.1 time: 0.402340 Oct 15 20:59:02.214486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1468/2582] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:01 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 1308 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:02.219265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:02.224541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:02.237364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:02.240182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:02.242414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:02.244227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:02.245816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:02.246543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:02.679664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:02.689434 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4cfba593-cbef-424a-8c18-3cc7fa7477c0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1490 microversion: 2.1 time: 0.473932 Oct 15 20:59:02.693720 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1469/2583] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:02 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1490 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:03.261256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8bc4072-854c-4fbe-8ab2-1171ccd839f8 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Calling method '.version_select at 0x7fe74ee42820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:03.413397 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8bc4072-854c-4fbe-8ab2-1171ccd839f8 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips" status: 200 len: 20 microversion: 2.1 time: 0.718308 Oct 15 20:59:03.413984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1470/2584] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 20:59:02 2022] GET /compute/v2.1/os-floating-ips => generated 20 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:59:03.419833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:03.424833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:03.436281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:03.449053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:03.449667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:03.450143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:03.458115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:03.458115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:03.458115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:03.518658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:03.583171 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a380926-386e-4cac-ada5-021948bf697e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1015 microversion: 2.1 time: 0.167836 Oct 15 20:59:03.591339 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1471/2585] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:59:03 2022] GET /compute/v2.1/servers => generated 1015 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:03.595464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-15c05217-564e-442b-8d15-5cf9f20a4498 req-740eb2f7-feb9-4135-a487-a1c0d57fca3d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "359b81c9-e8fc-491e-95d7-b2887dfb032f", "tag": "4ae4edec-e1c4-4b33-a749-85c9de8a0ed8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:03.609492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:INSERT=15,nova_api:SELECT=84,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:03.614602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-15c05217-564e-442b-8d15-5cf9f20a4498 req-740eb2f7-feb9-4135-a487-a1c0d57fca3d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:03.616249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-15c05217-564e-442b-8d15-5cf9f20a4498 req-740eb2f7-feb9-4135-a487-a1c0d57fca3d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:03.616713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-15c05217-564e-442b-8d15-5cf9f20a4498 req-740eb2f7-feb9-4135-a487-a1c0d57fca3d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:03.644132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-15c05217-564e-442b-8d15-5cf9f20a4498 req-740eb2f7-feb9-4135-a487-a1c0d57fca3d service nova] Creating event network-changed:4ae4edec-e1c4-4b33-a749-85c9de8a0ed8 for instance 359b81c9-e8fc-491e-95d7-b2887dfb032f on nch1-master-1 Oct 15 20:59:03.657923 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-15c05217-564e-442b-8d15-5cf9f20a4498 req-740eb2f7-feb9-4135-a487-a1c0d57fca3d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072597 Oct 15 20:59:03.663575 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1472/2586] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:03.684331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:03.685235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:03.704944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:03.705548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:03.706006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:03.706746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:03.712062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:03.712575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:05.169794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:05.208103 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25e65ee5-a492-4df7-bbfc-eaa3e30d3353 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 1.541909 Oct 15 20:59:05.208899 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1473/2587] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:03 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 1544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:05.228959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:05.229887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:05.238488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:05.239489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:05.239932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:05.240801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:05.241606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:05.242618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:06.185769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:06.213066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b60739d2-c70f-4825-91ca-5ecb4008874f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1317 microversion: 2.1 time: 0.990042 Oct 15 20:59:06.214240 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1474/2588] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:05 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1317 bytes in 1005 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:06.221942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-673450032", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:06.291932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:59:07.832159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:07.832726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:07.832979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:07.833802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:07.834229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:07.848130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:07.848992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:07.849660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:07.850611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:07.851380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:07.852034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:07.862726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=13 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:07.879597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:07.887319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:07.905690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:59:07.908332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:07.976158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e9c591e-65fc-4134-8814-bfcbc2041f00 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.760164 Oct 15 20:59:07.976811 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1475/2589] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:59:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 1762 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:59:07.999186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:07.999817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:08.015770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.016558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.017201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.018095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.018716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.019379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.315280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:08.325937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57ea20ce-db3c-4742-b6b4-8a3b0b158aa0 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 200 len: 1308 microversion: 2.1 time: 0.335303 Oct 15 20:59:08.343646 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1476/2590] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:07 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 1308 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:08.351055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dee07ce-5760-4f3e-9a8d-297905e12827 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Action: 'create', calling method: .version_select at 0x7fe74eeceaf0>, body: {"pool": "non_exist_pool"} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:08.422975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5dee07ce-5760-4f3e-9a8d-297905e12827 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] HTTP exception thrown: Floating IP pool not found. Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dee07ce-5760-4f3e-9a8d-297905e12827 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Request method failure captured: Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-floating-ips HTTP/1.1 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 26 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8UefTdbQbvutxy5mlrfim46MqADLhHrg8Lwe-bDojCjPg_u1QHpimW-alG7PWzv54niqSHszWZUGa90CyoVV3CgopgDZnM1ytywww0YZ_U31IK0e50Y7ZfIYPFsUgvqYEXEt9RcCx-Hy-vsgjAUABOfZz3dCivWMwHFzBRqXs1qAM Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 884bc7642fc14986a1f105a44322716c Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsNegativeTestJSON-1423742052 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/884bc7642fc14986a1f105a44322716c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_884bc7642fc14986a1f105a44322716c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/884bc7642fc14986a1f105a44322716c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/884bc7642fc14986a1f105a44322716c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsNegativeTestJSON-1423742052 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 884bc7642fc14986a1f105a44322716c Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsNegativeTestJSON-1423742052 Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsNegativeTestJSON-1423742052-project Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a9d5bc114a3347f58daee9ea6a440dea Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsNegativeTestJSON-1423742052-project Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: {"pool": "non_exist_pool"} Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP pool not found. Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'pool': 'non_exist_pool'}} Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 127, in create Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi address = self.network_api.allocate_floating_ip(context, pool) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2972, in allocate_floating_ip Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi pool_id = self._get_floating_ip_pool_id_by_name_or_id(client, pool) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2962, in _get_floating_ip_pool_id_by_name_or_id Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpPoolNotFound() Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpPoolNotFound: Floating IP pool not found. Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 142, in create Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP pool not found. Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:08.424671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:08.428948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP pool not found. Oct 15 20:59:08.428948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:08.428948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dee07ce-5760-4f3e-9a8d-297905e12827 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Returning 404 to user: Floating IP pool not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:08.428948 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5dee07ce-5760-4f3e-9a8d-297905e12827 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 404 len: 73 microversion: 2.1 time: 0.079509 Oct 15 20:59:08.428948 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1477/2591] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 20:59:08 2022] POST /compute/v2.1/os-floating-ips => generated 73 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:08.432967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:08.440459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:08.447564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.451727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.452453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.454624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.456513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.457229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.567789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:08.577573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0b2c164-da21-4cb9-bb04-475c8fe7d325 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1280 microversion: 2.1 time: 0.149737 Oct 15 20:59:08.578157 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1478/2592] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:08 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1280 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:08.720065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdmin275Test-server-1510122076", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:08.774410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:08.774832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:08.775309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:08.775717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:08.776128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:08.785908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.786395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.786830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.787557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.788016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.788449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.806301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Getting quotas for project 9ea11fd1a5a64f4a9641d7ed07f8e94f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:08.811649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Getting quotas for user 42276cf2f2894712ac91f87e4ce1f6ca and project 9ea11fd1a5a64f4a9641d7ed07f8e94f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:08.825750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:59:08.827934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:08.876440 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e1bcdb9-3e68-4bd4-90fb-cd3419461dea tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.75 time: 0.291807 Oct 15 20:59:08.884429 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1479/2593] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:59:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 294 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:59:08.884429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-74d03109-f8da-41f8-96b7-7309f46376e6 req-ec80af61-4a78-42af-8338-db9ea84ba2bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:08.894456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-74d03109-f8da-41f8-96b7-7309f46376e6 req-ec80af61-4a78-42af-8338-db9ea84ba2bf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.896336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-74d03109-f8da-41f8-96b7-7309f46376e6 req-ec80af61-4a78-42af-8338-db9ea84ba2bf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.896336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-74d03109-f8da-41f8-96b7-7309f46376e6 req-ec80af61-4a78-42af-8338-db9ea84ba2bf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.906807 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-74d03109-f8da-41f8-96b7-7309f46376e6 req-ec80af61-4a78-42af-8338-db9ea84ba2bf service nova] Creating event network-changed:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 20:59:08.919556 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-74d03109-f8da-41f8-96b7-7309f46376e6 req-ec80af61-4a78-42af-8338-db9ea84ba2bf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039825 Oct 15 20:59:08.920187 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1480/2594] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:08.925812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:08.926515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:08.934289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.934839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.935374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:08.936120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:08.936560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:08.936990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:09.331349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:09.342587 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2cf2270e-f9c2-4c10-9fa9-2a3cec95b55f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1512 microversion: 2.1 time: 0.421047 Oct 15 20:59:09.343643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1481/2595] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:08 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1512 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:09.351331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-70fde37c-953f-429b-9133-55100fa1854c req-8b6ce835-7a14-4245-b162-b40bef317189 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "359b81c9-e8fc-491e-95d7-b2887dfb032f", "name": "network-vif-plugged", "status": "completed", "tag": "4ae4edec-e1c4-4b33-a749-85c9de8a0ed8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:09.363350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-70fde37c-953f-429b-9133-55100fa1854c req-8b6ce835-7a14-4245-b162-b40bef317189 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:09.364128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-70fde37c-953f-429b-9133-55100fa1854c req-8b6ce835-7a14-4245-b162-b40bef317189 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:09.364950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-70fde37c-953f-429b-9133-55100fa1854c req-8b6ce835-7a14-4245-b162-b40bef317189 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:09.370397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:09.381346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-70fde37c-953f-429b-9133-55100fa1854c req-8b6ce835-7a14-4245-b162-b40bef317189 service nova] Creating event network-vif-plugged:4ae4edec-e1c4-4b33-a749-85c9de8a0ed8 for instance 359b81c9-e8fc-491e-95d7-b2887dfb032f on nch1-master-1 Oct 15 20:59:09.386411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-70fde37c-953f-429b-9133-55100fa1854c req-8b6ce835-7a14-4245-b162-b40bef317189 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041009 Oct 15 20:59:09.387014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1482/2596] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:09.394255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:09.399332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:09.400066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Searching by: {'deleted': False, 'project_id': '7e45e841218949e688b3deee07fd25ed'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:09.406792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:09.407538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:09.408217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:09.409180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:09.409764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:09.410573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:09.449371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:09.483860 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-973d27d5-d3b7-4895-b6e1-1bbe2805d1e5 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 334 microversion: 2.1 time: 0.094787 Oct 15 20:59:09.484451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1483/2597] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:59:09 2022] GET /compute/v2.1/servers => generated 334 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:09.492151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:09.492151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:09.499257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:09.499744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:09.500181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:09.500885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:09.501367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:09.501791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:10.052056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:10.061568 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8335d86-fc88-4af3-baa9-58bdf972116c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1317 microversion: 2.1 time: 0.576030 Oct 15 20:59:10.062256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1484/2598] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:09 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1317 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:10.079278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:10.085674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:10.092915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:10.093998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:10.094101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:10.094847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:10.095311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:10.095928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:10.261131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:10.269732 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51f32d7e-c063-4833-8067-20e0c5e75b05 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1260 microversion: 2.1 time: 0.206130 Oct 15 20:59:10.270313 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1485/2599] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:10 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1260 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:10.276070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3fd163c-ffd1-4ec3-b519-01bee63a10b0 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Calling method '.version_select at 0x7fe74eda7430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:10.707947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-e3fd163c-ffd1-4ec3-b519-01bee63a10b0 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP e0611dce-f9f3-4a44-9c78-ed92e0f094ab could not be found", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:59:10.708860 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e3fd163c-ffd1-4ec3-b519-01bee63a10b0 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] HTTP exception thrown: Floating IP not found for ID e0611dce-f9f3-4a44-9c78-ed92e0f094ab Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3fd163c-ffd1-4ec3-b519-01bee63a10b0 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Request method failure captured: Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-floating-ips/e0611dce-f9f3-4a44-9c78-ed92e0f094ab HTTP/1.1 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8UefTdbQbvutxy5mlrfim46MqADLhHrg8Lwe-bDojCjPg_u1QHpimW-alG7PWzv54niqSHszWZUGa90CyoVV3CgopgDZnM1ytywww0YZ_U31IK0e50Y7ZfIYPFsUgvqYEXEt9RcCx-Hy-vsgjAUABOfZz3dCivWMwHFzBRqXs1qAM Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 884bc7642fc14986a1f105a44322716c Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsNegativeTestJSON-1423742052 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/884bc7642fc14986a1f105a44322716c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_884bc7642fc14986a1f105a44322716c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/884bc7642fc14986a1f105a44322716c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/884bc7642fc14986a1f105a44322716c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsNegativeTestJSON-1423742052 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 884bc7642fc14986a1f105a44322716c Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsNegativeTestJSON-1423742052 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsNegativeTestJSON-1423742052-project Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a9d5bc114a3347f58daee9ea6a440dea Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsNegativeTestJSON-1423742052-project Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP not found for ID e0611dce-f9f3-4a44-9c78-ed92e0f094ab Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e0611dce-f9f3-4a44-9c78-ed92e0f094ab'} Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP e0611dce-f9f3-4a44-9c78-ed92e0f094ab could not be found Oct 15 20:59:10.710925 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d9e66659-bfc6-47fc-8365-5f4adaf283b5'] Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 158, in delete Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID e0611dce-f9f3-4a44-9c78-ed92e0f094ab. Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 161, in delete Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID e0611dce-f9f3-4a44-9c78-ed92e0f094ab Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID e0611dce-f9f3-4a44-9c78-ed92e0f094ab Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3fd163c-ffd1-4ec3-b519-01bee63a10b0 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] Returning 404 to user: Floating IP not found for ID e0611dce-f9f3-4a44-9c78-ed92e0f094ab {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e3fd163c-ffd1-4ec3-b519-01bee63a10b0 tempest-FloatingIPsNegativeTestJSON-1423742052 tempest-FloatingIPsNegativeTestJSON-1423742052-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/e0611dce-f9f3-4a44-9c78-ed92e0f094ab" status: 404 len: 111 microversion: 2.1 time: 0.441118 Oct 15 20:59:10.715260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1486/2600] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 20:59:10 2022] DELETE /compute/v2.1/os-floating-ips/e0611dce-f9f3-4a44-9c78-ed92e0f094ab => generated 111 bytes in 442 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:10.721529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Action: 'update', calling method: >, body: {"server": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:10.722669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:10.732066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:10.732663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:10.733150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:10.734068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:10.734707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:10.735139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:11.011800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:11.024945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:11.026835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:11.032909 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd2cb916-8970-4bca-a95d-dc94c75d6bd6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "PUT /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1411 microversion: 2.75 time: 0.318902 Oct 15 20:59:11.049471 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1487/2601] 10.222.0.16 () {62 vars in 1297 bytes} [Sat Oct 15 20:59:10 2022] PUT /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1411 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:11.061514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:11.062958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:11.069945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:11.070800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:11.071510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:11.072518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:11.073250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:11.073799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:11.810450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:11.821450 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b7281381-6223-47a0-986a-eb8c02a84917 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 200 len: 1488 microversion: 2.1 time: 0.769699 Oct 15 20:59:11.822657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1488/2602] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:11 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 1488 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:11.828192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:11.834286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:11.835184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Searching by: {'deleted': False, 'project_id': '7e45e841218949e688b3deee07fd25ed'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:11.842950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:11.844474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:11.845220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:11.846264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:11.847070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:11.847724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:11.887242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:11.958779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:11.967438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:11.968537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:11.969255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:12.829524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-202fdbf6-e96e-478b-a738-9ca95d0bc075 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 1696 microversion: 2.1 time: 1.005629 Oct 15 20:59:12.831241 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1489/2603] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:59:11 2022] GET /compute/v2.1/servers/detail => generated 1696 bytes in 1008 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:12.849393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:12.850272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:12.869168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:12.870268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:12.870936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:12.871661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:12.872379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:12.873081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:13.553145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:13.564746 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8270729-aff3-4348-b52b-5e9867c8425b tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1322 microversion: 2.1 time: 0.732208 Oct 15 20:59:13.565707 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1490/2604] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:12 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1322 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:13.578755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:13.579548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:13.595824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:13.596492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:13.597078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:13.597846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:13.598307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:13.598735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:14.312995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:14.325994 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ef2de72-6e9f-46ab-a3e6-be371e52f0da tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1489 microversion: 2.1 time: 0.758984 Oct 15 20:59:14.326603 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1491/2605] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:13 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1489 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:14.349284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:14.350145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:14.359844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:14.360286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:14.360736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:14.361797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:14.362280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:14.362720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:14.554176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:14.571255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:14.574238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:14.581268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1c00e0c-cf1a-4283-b101-9e198b93f162 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1417 microversion: 2.75 time: 0.249882 Oct 15 20:59:14.582886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1492/2606] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:14 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1417 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:14.589651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:14.591601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:14.602195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:14.605570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:14.606268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:14.607337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:14.607968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:14.608411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:14.965399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:14.980192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d2e06291-47bc-4689-adad-e8139bfa3e36 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1317 microversion: 2.1 time: 0.395654 Oct 15 20:59:14.985960 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1493/2607] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:14 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1317 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:14.998788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:14.999858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:15.008046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:15.008680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:15.009551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:15.010519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:15.011111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:15.011693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:15.650063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:15.661773 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61c1b94b-7616-40b0-bd7c-99db2dd92cbc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.674646 Oct 15 20:59:15.662365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1494/2608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:14 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:15.667364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7672bd44-c006-4315-b031-2a7c7f9e1d24 req-c81a092e-d032-4cd2-9989-e500f5cdb09a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "183834bb-a5a3-43c7-a9c4-20c50652331d", "tag": "5fddc112-0579-4080-a39f-1b786f647006"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:15.680168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7672bd44-c006-4315-b031-2a7c7f9e1d24 req-c81a092e-d032-4cd2-9989-e500f5cdb09a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:15.680663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7672bd44-c006-4315-b031-2a7c7f9e1d24 req-c81a092e-d032-4cd2-9989-e500f5cdb09a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:15.681169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7672bd44-c006-4315-b031-2a7c7f9e1d24 req-c81a092e-d032-4cd2-9989-e500f5cdb09a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:15.713584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7672bd44-c006-4315-b031-2a7c7f9e1d24 req-c81a092e-d032-4cd2-9989-e500f5cdb09a service nova] Creating event network-changed:5fddc112-0579-4080-a39f-1b786f647006 for instance 183834bb-a5a3-43c7-a9c4-20c50652331d on nch1-master-1 Oct 15 20:59:15.727219 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7672bd44-c006-4315-b031-2a7c7f9e1d24 req-c81a092e-d032-4cd2-9989-e500f5cdb09a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063417 Oct 15 20:59:15.728082 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1495/2609] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:15.737313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-909010618"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:15.740293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:15.748259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:15.748763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:15.749234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:15.750185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:15.750997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:15.751596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:15.977044 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-424dadf3-da46-4ca5-8973-ed75a87c82ae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f/action" status: 202 len: 0 microversion: 2.1 time: 0.247313 Oct 15 20:59:15.977750 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1496/2610] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:59:15 2022] POST /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f/action => generated 0 bytes in 249 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:59:15.983309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f836524-c1e4-4a7a-b50c-6475109b9993 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:15.991495 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f836524-c1e4-4a7a-b50c-6475109b9993 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.012608 Oct 15 20:59:15.992051 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1497/2611] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:59:15 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:15.999965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f67ff5f8-56e0-4813-83b5-3efe5e18964e req-5ab4edd5-2229-44ff-82c5-458360483355 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "name": "network-vif-plugged", "status": "completed", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:16.011415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f67ff5f8-56e0-4813-83b5-3efe5e18964e req-5ab4edd5-2229-44ff-82c5-458360483355 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.011906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f67ff5f8-56e0-4813-83b5-3efe5e18964e req-5ab4edd5-2229-44ff-82c5-458360483355 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.012342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f67ff5f8-56e0-4813-83b5-3efe5e18964e req-5ab4edd5-2229-44ff-82c5-458360483355 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.023901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f67ff5f8-56e0-4813-83b5-3efe5e18964e req-5ab4edd5-2229-44ff-82c5-458360483355 service nova] Creating event network-vif-plugged:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 20:59:16.029009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f67ff5f8-56e0-4813-83b5-3efe5e18964e req-5ab4edd5-2229-44ff-82c5-458360483355 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035685 Oct 15 20:59:16.029818 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1498/2612] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:16.034783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.035458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:16.041822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.042364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.042815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.043528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.044024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.044462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.072638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:16.102565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:16.152314 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f06ce018-cf57-4633-b144-9a259886151c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 204 len: 0 microversion: 2.1 time: 0.121253 Oct 15 20:59:16.152977 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1499/2613] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:16 2022] DELETE /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:16.158555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.159304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:16.165307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.165736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.166184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.166884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.167350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.167784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.347529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:16.355360 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da3924ec-f2db-480f-aeb7-f747f30bb1a8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1322 microversion: 2.1 time: 0.200722 Oct 15 20:59:16.355921 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1500/2614] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:16 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1322 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:16.361259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.362313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:16.368161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.368824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.369874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.371106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.371716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.372937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.441256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:16.449482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:16.451459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:16.456338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51dcc8bf-22ce-4432-9007-a9af5ea568fc tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1438 microversion: 2.75 time: 0.098909 Oct 15 20:59:16.456924 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1501/2615] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:16 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1438 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:16.579966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d78a9241-94b3-4ae3-a6a0-51c92590c508 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Calling method '.version_select at 0x7fe74eda7e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.601186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d78a9241-94b3-4ae3-a6a0-51c92590c508 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] HTTP exception thrown: Image not found. Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d78a9241-94b3-4ae3-a6a0-51c92590c508 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Request method failure captured: Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/1e662120-28f9-42de-9799-278b600ebf00/metadata/os_distro HTTP/1.1 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8jABUuH0AJLlt-zI4Yl3TfwjMGN6RJBU4p5Bi6BTvpEPmCwOxHl8-g4RsgxBBK1m-sdLgtvSrQQQXD_Bm1Qb5KyhgGtQfYSDZGwN7ZKB2gJRQUNJZwrK9EThXxAZjt8CFQlmDXrDVsx001W7sJEwhi1YgpBjaSTImEElOM-Vm4vy4 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d07d1b1c31b04feb91f36fbf7fdc1b24 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'image_id': '1e662120-28f9-42de-9799-278b600ebf00', 'id': 'os_distro'} Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 1e662120-28f9-42de-9799-278b600ebf00 Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:16.609449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 1e662120-28f9-42de-9799-278b600ebf00 could not be found. Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 125, in delete Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d78a9241-94b3-4ae3-a6a0-51c92590c508 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d78a9241-94b3-4ae3-a6a0-51c92590c508 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] 10.222.0.16 "DELETE /compute/v2.1/images/1e662120-28f9-42de-9799-278b600ebf00/metadata/os_distro" status: 404 len: 62 microversion: 2.1 time: 0.152991 Oct 15 20:59:16.614366 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1502/2616] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 20:59:16 2022] DELETE /compute/v2.1/images/1e662120-28f9-42de-9799-278b600ebf00/metadata/os_distro => generated 62 bytes in 154 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:16.617718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a971ba0-c424-461c-8a5f-cf3ed8522ee7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ee265e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.662538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a971ba0-c424-461c-8a5f-cf3ed8522ee7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.049893 Oct 15 20:59:16.671011 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1503/2617] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:16 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:16.671011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.671011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:16.674734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.675231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.675678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.676390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.676858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.677318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.812729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:16.820428 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4fcc98f-2b17-444a-8295-a41bc5c1f081 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1317 microversion: 2.1 time: 0.156224 Oct 15 20:59:16.820988 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1504/2618] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:16 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1317 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:16.826021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.826630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:16.828367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:16.833999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.834496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.834935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.835640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.836099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.836527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.970005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:16.977754 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe2dc970-3e8e-487c-8482-8df6148a65e0 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1495 microversion: 2.1 time: 0.155804 Oct 15 20:59:16.978320 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1505/2619] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:16 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1495 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:16.982910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:16.983405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:16.989317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.989803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.990239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:16.990929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:16.991385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:16.991810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.018355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:17.094794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e74698e6-7b05-455a-bdd7-f47ec6ea5239 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 204 len: 0 microversion: 2.1 time: 0.115263 Oct 15 20:59:17.095374 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1506/2620] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:16 2022] DELETE /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:17.100530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:17.103409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:17.109134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.109627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.110067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.110764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.111225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.111913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.199757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:17.200379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:17.200857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:17.201296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:17.201712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:17.260777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:17.266804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.267303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.267752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.268520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.268933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.269393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.342407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:17.351378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:17.352832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:17.359131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-761c5dd6-f9b8-4581-afe6-a3520880ec99 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "POST /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe/action" status: 202 len: 1495 microversion: 2.75 time: 0.262636 Oct 15 20:59:17.369240 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1507/2621] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:59:17 2022] POST /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe/action => generated 1495 bytes in 264 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Oct 15 20:59:17.369240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb8e6aa7-6474-4d72-89fa-da2bf88927e4 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Calling method '.version_select at 0x7fe74ef80310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:17.396167 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-eb8e6aa7-6474-4d72-89fa-da2bf88927e4 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] HTTP exception thrown: Image not found. Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb8e6aa7-6474-4d72-89fa-da2bf88927e4 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Request method failure captured: Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/images/cff12e7c-7bf2-4a2c-ade3-9712a241c38b/metadata/os_version HTTP/1.1 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8jABUuH0AJLlt-zI4Yl3TfwjMGN6RJBU4p5Bi6BTvpEPmCwOxHl8-g4RsgxBBK1m-sdLgtvSrQQQXD_Bm1Qb5KyhgGtQfYSDZGwN7ZKB2gJRQUNJZwrK9EThXxAZjt8CFQlmDXrDVsx001W7sJEwhi1YgpBjaSTImEElOM-Vm4vy4 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d07d1b1c31b04feb91f36fbf7fdc1b24 Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'image_id': 'cff12e7c-7bf2-4a2c-ade3-9712a241c38b', 'id': 'os_version'} Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID cff12e7c-7bf2-4a2c-ade3-9712a241c38b Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:17.398359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image cff12e7c-7bf2-4a2c-ade3-9712a241c38b could not be found. Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 58, in show Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi metadata = self._get_image(context, image_id)['properties'] Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb8e6aa7-6474-4d72-89fa-da2bf88927e4 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb8e6aa7-6474-4d72-89fa-da2bf88927e4 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] 10.222.0.16 "GET /compute/v2.1/images/cff12e7c-7bf2-4a2c-ade3-9712a241c38b/metadata/os_version" status: 404 len: 62 microversion: 2.1 time: 0.039043 Oct 15 20:59:17.402934 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1508/2622] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 20:59:17 2022] GET /compute/v2.1/images/cff12e7c-7bf2-4a2c-ade3-9712a241c38b/metadata/os_version => generated 62 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:17.410879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:17.410879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:17.423124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.423680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.424271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.425431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.425974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.427992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.622642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:17.630530 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a08e548-9336-47da-b529-e3964c0ef50b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.224689 Oct 15 20:59:17.631091 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1509/2623] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:17 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:17.635433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:17.636037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:17.642121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.642605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.643042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.643730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.644186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.644612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.779402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:17.787277 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2171f44c-d33e-4c24-aa7f-46d764149a42 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1699 microversion: 2.1 time: 0.155239 Oct 15 20:59:17.787839 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1510/2624] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:17 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1699 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:17.792291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:17.792895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:17.798880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.799366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.799809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.800505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.800959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.801412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.931686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:17.940425 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-afcdfcbf-60ec-4643-aec1-172342f5e6b8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1322 microversion: 2.1 time: 0.151623 Oct 15 20:59:17.940997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1511/2625] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:17 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1322 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:17.945473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:17.946083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:17.952081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.952577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.953024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:17.953745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:17.954224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:17.954645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:18.022992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:18.031657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:18.033150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:18.038043 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a057ce0e-977f-46bf-973e-022cd0482b54 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.096070 Oct 15 20:59:18.038610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1512/2626] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:17 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:18.043252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33d5088c-3950-48da-a8e7-307687b7898c tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Calling method '.version_select at 0x7fe74ef80280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:18.065951 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-33d5088c-3950-48da-a8e7-307687b7898c tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] HTTP exception thrown: Image not found. Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33d5088c-3950-48da-a8e7-307687b7898c tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Request method failure captured: Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/images/38f344ab-2be0-486a-9380-3bced1c7a35f/metadata HTTP/1.1 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8jABUuH0AJLlt-zI4Yl3TfwjMGN6RJBU4p5Bi6BTvpEPmCwOxHl8-g4RsgxBBK1m-sdLgtvSrQQQXD_Bm1Qb5KyhgGtQfYSDZGwN7ZKB2gJRQUNJZwrK9EThXxAZjt8CFQlmDXrDVsx001W7sJEwhi1YgpBjaSTImEElOM-Vm4vy4 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d07d1b1c31b04feb91f36fbf7fdc1b24 Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'image_id': '38f344ab-2be0-486a-9380-3bced1c7a35f'} Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 38f344ab-2be0-486a-9380-3bced1c7a35f Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:18.068049 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 38f344ab-2be0-486a-9380-3bced1c7a35f could not be found. Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 51, in index Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi metadata = self._get_image(context, image_id)['properties'] Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33d5088c-3950-48da-a8e7-307687b7898c tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33d5088c-3950-48da-a8e7-307687b7898c tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] 10.222.0.16 "GET /compute/v2.1/images/38f344ab-2be0-486a-9380-3bced1c7a35f/metadata" status: 404 len: 62 microversion: 2.1 time: 0.030006 Oct 15 20:59:18.072763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1513/2627] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 20:59:18 2022] GET /compute/v2.1/images/38f344ab-2be0-486a-9380-3bced1c7a35f/metadata => generated 62 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:18.077372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-007fbab3-05e3-4074-993d-80ef4d00210a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ee42c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:18.136290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-007fbab3-05e3-4074-993d-80ef4d00210a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.064470 Oct 15 20:59:18.137017 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1514/2628] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:18 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:18.141748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "55d566f6-a7eb-497e-8f4e-43c60433fc35", "tag": "a601a90d-756d-4ac7-97b5-50063c98b756"}, {"name": "network-changed", "server_uuid": "cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b", "tag": "dc6638f8-3a22-4290-9aae-3189cabf87b0"}, {"name": "network-changed", "server_uuid": "1910f53a-ef2a-4f63-836d-9850556d1762", "tag": "5f4c041d-5d6e-4a7f-a157-e88a7ad86f9e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:18.153312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:18.153806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:18.154242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:18.168539 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Creating event network-changed:a601a90d-756d-4ac7-97b5-50063c98b756 for instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 on nch1-master-1 Oct 15 20:59:18.169603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Creating event network-changed:dc6638f8-3a22-4290-9aae-3189cabf87b0 for instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b on nch2-master-1 Oct 15 20:59:18.170115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] Creating event network-changed:5f4c041d-5d6e-4a7f-a157-e88a7ad86f9e for instance 1910f53a-ef2a-4f63-836d-9850556d1762 on nch2-master-1 Oct 15 20:59:18.179651 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e062b0d0-84a0-445e-88fd-313cc4a3d48a req-e6c71265-2de1-4112-9b23-56111e932db1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.041610 Oct 15 20:59:18.180299 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1515/2629] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:18 2022] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:18.186012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:18.186646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:18.198247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:18.198759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:18.199190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:18.199905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:18.200648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:18.201307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:18.518432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:18.528629 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c36d244a-f1ce-42c4-9f5c-a18bf2e897b5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1317 microversion: 2.1 time: 0.346616 Oct 15 20:59:18.530162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1516/2630] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:18 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1317 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:18.535681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:18.536412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:18.550188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:18.550833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:18.551281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:18.551988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:18.552448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:18.553003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:18.557555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:SELECT=94,nova_cell1:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:19.098087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:19.107442 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d90266e-d998-43a7-880b-75347dc519c9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1495 microversion: 2.1 time: 0.575658 Oct 15 20:59:19.117592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1517/2631] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:18 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1495 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:19.126276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-582a02f8-209d-4aa1-b157-4bcf3eaea170 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Action: 'update_all', calling method: .version_select at 0x7fe74eddc550>, body: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:19.152898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-582a02f8-209d-4aa1-b157-4bcf3eaea170 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] HTTP exception thrown: Image not found. Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-582a02f8-209d-4aa1-b157-4bcf3eaea170 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Request method failure captured: Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/images/bf458d99-7b44-4e33-83e2-f3ccb443163b/metadata HTTP/1.1 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 57 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8jABUuH0AJLlt-zI4Yl3TfwjMGN6RJBU4p5Bi6BTvpEPmCwOxHl8-g4RsgxBBK1m-sdLgtvSrQQQXD_Bm1Qb5KyhgGtQfYSDZGwN7ZKB2gJRQUNJZwrK9EThXxAZjt8CFQlmDXrDVsx001W7sJEwhi1YgpBjaSTImEElOM-Vm4vy4 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d07d1b1c31b04feb91f36fbf7fdc1b24 Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'image_id': 'bf458d99-7b44-4e33-83e2-f3ccb443163b', 'body': {'metadata': {'os_distro': 'alt1', 'os_version': 'alt2'}}} Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID bf458d99-7b44-4e33-83e2-f3ccb443163b Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:19.155235 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image bf458d99-7b44-4e33-83e2-f3ccb443163b could not be found. Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 109, in update_all Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-582a02f8-209d-4aa1-b157-4bcf3eaea170 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:19.160077 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-582a02f8-209d-4aa1-b157-4bcf3eaea170 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] 10.222.0.16 "PUT /compute/v2.1/images/bf458d99-7b44-4e33-83e2-f3ccb443163b/metadata" status: 404 len: 62 microversion: 2.1 time: 0.036882 Oct 15 20:59:19.166293 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1518/2632] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:59:19 2022] PUT /compute/v2.1/images/bf458d99-7b44-4e33-83e2-f3ccb443163b/metadata => generated 62 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:19.176736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5364ed84-3d8c-4513-bcba-2fe041d1e388 req-3db080cd-2bd5-4728-995b-c38c3462ab3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "183834bb-a5a3-43c7-a9c4-20c50652331d", "name": "network-vif-plugged", "status": "completed", "tag": "5fddc112-0579-4080-a39f-1b786f647006"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:19.187437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5364ed84-3d8c-4513-bcba-2fe041d1e388 req-3db080cd-2bd5-4728-995b-c38c3462ab3a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:19.187928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5364ed84-3d8c-4513-bcba-2fe041d1e388 req-3db080cd-2bd5-4728-995b-c38c3462ab3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:19.188363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5364ed84-3d8c-4513-bcba-2fe041d1e388 req-3db080cd-2bd5-4728-995b-c38c3462ab3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:19.210334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5364ed84-3d8c-4513-bcba-2fe041d1e388 req-3db080cd-2bd5-4728-995b-c38c3462ab3a service nova] Creating event network-vif-plugged:5fddc112-0579-4080-a39f-1b786f647006 for instance 183834bb-a5a3-43c7-a9c4-20c50652331d on nch1-master-1 Oct 15 20:59:19.215291 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5364ed84-3d8c-4513-bcba-2fe041d1e388 req-3db080cd-2bd5-4728-995b-c38c3462ab3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047791 Oct 15 20:59:19.219540 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1519/2633] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:19.224991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:19.225712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:19.232872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:19.233393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:19.233827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:19.234725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:19.235367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:19.235947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:20.076723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:20.088633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7485fa45-a846-441a-822e-7d902535661f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.867573 Oct 15 20:59:20.089843 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1520/2634] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:19 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 869 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:20.096114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:20.096953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:20.111925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:20.112713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:20.113123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:20.114887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:20.115433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:20.116254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:20.277527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d437aff8aa486ca43edb2605a57b4e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e45e841218949e688b3deee07fd25ed', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:20.291510 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4200b08d-9b95-404d-9ec5-f343f6f09944 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 200 len: 1354 microversion: 2.1 time: 0.199361 Oct 15 20:59:20.307723 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1521/2635] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:20 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 1354 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:20.381158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:20.383397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:20.402564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:20.406452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:20.407074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:20.407864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:20.408346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:20.411240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:20.940005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:20.949177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30f8cf1d-ee0c-4f7d-ba57-8c14f27b1c96 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1322 microversion: 2.1 time: 0.601171 Oct 15 20:59:20.950120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1522/2636] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:20 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1322 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:20.955464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:20.956285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:20.962713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:20.963201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:20.963638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:20.964539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:20.964996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:20.965850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.036334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:21.059555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:21.061072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:21.070222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce159f3c-22dc-4b5f-81b4-e2a0f6675f85 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.118448 Oct 15 20:59:21.081914 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1523/2637] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:20 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:21.081914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5578a7bb-5269-4fd0-be62-396c9c5c65e8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74eda7700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:21.135170 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5578a7bb-5269-4fd0-be62-396c9c5c65e8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.062665 Oct 15 20:59:21.144810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1524/2638] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:21 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:21.150899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fd610d2-9e09-46f4-9e09-eb6ba967b392 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Action: 'update', calling method: .version_select at 0x7fe74ee42c10>, body: {"meta": {"os_distro": "alt"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:21.186980 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9fd610d2-9e09-46f4-9e09-eb6ba967b392 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] HTTP exception thrown: Image not found. Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fd610d2-9e09-46f4-9e09-eb6ba967b392 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Request method failure captured: Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/images/a6454680-70c3-4c2d-8b0b-c397bfeced0a/metadata/os_distro HTTP/1.1 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 30 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8jABUuH0AJLlt-zI4Yl3TfwjMGN6RJBU4p5Bi6BTvpEPmCwOxHl8-g4RsgxBBK1m-sdLgtvSrQQQXD_Bm1Qb5KyhgGtQfYSDZGwN7ZKB2gJRQUNJZwrK9EThXxAZjt8CFQlmDXrDVsx001W7sJEwhi1YgpBjaSTImEElOM-Vm4vy4 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d07d1b1c31b04feb91f36fbf7fdc1b24 Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: {"meta": {"os_distro": "alt"}} Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'image_id': 'a6454680-70c3-4c2d-8b0b-c397bfeced0a', 'id': 'os_distro', 'body': {'meta': {'os_distro': 'alt'}}} Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID a6454680-70c3-4c2d-8b0b-c397bfeced0a Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:21.189812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image a6454680-70c3-4c2d-8b0b-c397bfeced0a could not be found. Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 93, in update Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fd610d2-9e09-46f4-9e09-eb6ba967b392 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fd610d2-9e09-46f4-9e09-eb6ba967b392 tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] 10.222.0.16 "PUT /compute/v2.1/images/a6454680-70c3-4c2d-8b0b-c397bfeced0a/metadata/os_distro" status: 404 len: 62 microversion: 2.1 time: 0.045267 Oct 15 20:59:21.195410 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1525/2639] 10.222.0.16 () {60 vars in 1310 bytes} [Sat Oct 15 20:59:21 2022] PUT /compute/v2.1/images/a6454680-70c3-4c2d-8b0b-c397bfeced0a/metadata/os_distro => generated 62 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:21.205855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:21.205855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:21.206766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.208401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.208401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.208401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.208999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.209416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.368544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:21.377087 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f122d34f-06a3-4314-9f23-e04632728e71 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1467 microversion: 2.1 time: 0.183816 Oct 15 20:59:21.377745 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1526/2640] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:21 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1467 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:21.382167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4772a13e-4d51-47f6-8461-ad2d1be695f3 req-52ee0029-7442-41c8-8f76-a11a6b3405db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e970fbbc-4044-415f-833d-21f6bd4d33a2", "name": "network-vif-deleted", "tag": "dba907e7-a00a-4b1c-813e-5237b472bb44"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:21.391565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4772a13e-4d51-47f6-8461-ad2d1be695f3 req-52ee0029-7442-41c8-8f76-a11a6b3405db service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.392070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4772a13e-4d51-47f6-8461-ad2d1be695f3 req-52ee0029-7442-41c8-8f76-a11a6b3405db service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.392499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4772a13e-4d51-47f6-8461-ad2d1be695f3 req-52ee0029-7442-41c8-8f76-a11a6b3405db service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.404607 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4772a13e-4d51-47f6-8461-ad2d1be695f3 req-52ee0029-7442-41c8-8f76-a11a6b3405db service nova] Creating event network-vif-deleted:dba907e7-a00a-4b1c-813e-5237b472bb44 for instance e970fbbc-4044-415f-833d-21f6bd4d33a2 on nch1-master-1 Oct 15 20:59:21.409786 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4772a13e-4d51-47f6-8461-ad2d1be695f3 req-52ee0029-7442-41c8-8f76-a11a6b3405db service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030818 Oct 15 20:59:21.410369 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1527/2641] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:21.415296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:21.415996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:21.422717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.423201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.423639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.424333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.424787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.425244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.455416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.455900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.456340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.456793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lazy-loading 'fault' on Instance uuid 15bdf694-ed96-4052-b284-784636eefd0a {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:59:21.503844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:21.511949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c572a25-895f-4095-8607-05e899e7ed29 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 200 len: 1305 microversion: 2.1 time: 0.100465 Oct 15 20:59:21.512610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1528/2642] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:21 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 1305 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:21.517093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-35dfb1ab-6bc8-4438-a441-7ac22b308301 req-ce2b072a-e3e9-4fc2-833b-cbeb1f7cd671 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15bdf694-ed96-4052-b284-784636eefd0a", "name": "network-vif-deleted", "tag": "929e53c0-85c1-4e31-a351-d67f070c1486"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:21.526793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-35dfb1ab-6bc8-4438-a441-7ac22b308301 req-ce2b072a-e3e9-4fc2-833b-cbeb1f7cd671 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.527348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-35dfb1ab-6bc8-4438-a441-7ac22b308301 req-ce2b072a-e3e9-4fc2-833b-cbeb1f7cd671 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.527845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-35dfb1ab-6bc8-4438-a441-7ac22b308301 req-ce2b072a-e3e9-4fc2-833b-cbeb1f7cd671 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.540517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-35dfb1ab-6bc8-4438-a441-7ac22b308301 req-ce2b072a-e3e9-4fc2-833b-cbeb1f7cd671 service nova] Creating event network-vif-deleted:929e53c0-85c1-4e31-a351-d67f070c1486 for instance 15bdf694-ed96-4052-b284-784636eefd0a on nch1-master-1 Oct 15 20:59:21.546094 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-35dfb1ab-6bc8-4438-a441-7ac22b308301 req-ce2b072a-e3e9-4fc2-833b-cbeb1f7cd671 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032274 Oct 15 20:59:21.546703 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1529/2643] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:21.551832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:21.552543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:21.558602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.559087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.559525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.560220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.560678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.561128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.700919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:21.708990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8249d374-07be-4283-b5c3-40a7c4263372 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.161159 Oct 15 20:59:21.713098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1530/2644] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:21 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:21.714273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83125dd2-825d-4534-afb0-2634d063368b tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Action: 'create', calling method: .version_select at 0x7fe74fa460d0>, body: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:21.741503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-83125dd2-825d-4534-afb0-2634d063368b tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] HTTP exception thrown: Image not found. Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83125dd2-825d-4534-afb0-2634d063368b tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Request method failure captured: Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/images/37dcfd3b-47ec-40c3-89c9-18f13ae715a7/metadata HTTP/1.1 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 57 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8jABUuH0AJLlt-zI4Yl3TfwjMGN6RJBU4p5Bi6BTvpEPmCwOxHl8-g4RsgxBBK1m-sdLgtvSrQQQXD_Bm1Qb5KyhgGtQfYSDZGwN7ZKB2gJRQUNJZwrK9EThXxAZjt8CFQlmDXrDVsx001W7sJEwhi1YgpBjaSTImEElOM-Vm4vy4 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ac52ed6c42444cecbd7212dae59bee9f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ac52ed6c42444cecbd7212dae59bee9f Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d07d1b1c31b04feb91f36fbf7fdc1b24 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-1479279892-project Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'image_id': '37dcfd3b-47ec-40c3-89c9-18f13ae715a7', 'body': {'metadata': {'os_distro': 'alt1', 'os_version': 'alt2'}}} Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 37dcfd3b-47ec-40c3-89c9-18f13ae715a7 Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:21.743932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 37dcfd3b-47ec-40c3-89c9-18f13ae715a7 could not be found. Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 69, in create Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83125dd2-825d-4534-afb0-2634d063368b tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83125dd2-825d-4534-afb0-2634d063368b tempest-ImagesMetadataNegativeTestJSON-1479279892 tempest-ImagesMetadataNegativeTestJSON-1479279892-project] 10.222.0.16 "POST /compute/v2.1/images/37dcfd3b-47ec-40c3-89c9-18f13ae715a7/metadata" status: 404 len: 62 microversion: 2.1 time: 0.034795 Oct 15 20:59:21.753127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1531/2645] 10.222.0.16 () {60 vars in 1281 bytes} [Sat Oct 15 20:59:21 2022] POST /compute/v2.1/images/37dcfd3b-47ec-40c3-89c9-18f13ae715a7/metadata => generated 62 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:21.757093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d8b9e331-821f-4a5a-93d4-1503ff2e5e22 req-137d6319-4326-48f0-86ac-7fa94a6563a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "55d566f6-a7eb-497e-8f4e-43c60433fc35", "name": "network-vif-plugged", "status": "completed", "tag": "a601a90d-756d-4ac7-97b5-50063c98b756"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:21.764289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d8b9e331-821f-4a5a-93d4-1503ff2e5e22 req-137d6319-4326-48f0-86ac-7fa94a6563a0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.764783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d8b9e331-821f-4a5a-93d4-1503ff2e5e22 req-137d6319-4326-48f0-86ac-7fa94a6563a0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.765253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d8b9e331-821f-4a5a-93d4-1503ff2e5e22 req-137d6319-4326-48f0-86ac-7fa94a6563a0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.776402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d8b9e331-821f-4a5a-93d4-1503ff2e5e22 req-137d6319-4326-48f0-86ac-7fa94a6563a0 service nova] Creating event network-vif-plugged:a601a90d-756d-4ac7-97b5-50063c98b756 for instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 on nch1-master-1 Oct 15 20:59:21.782206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d8b9e331-821f-4a5a-93d4-1503ff2e5e22 req-137d6319-4326-48f0-86ac-7fa94a6563a0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034877 Oct 15 20:59:21.783005 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1532/2646] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:21.788376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:21.789316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] [instance: e970fbbc-4044-415f-833d-21f6bd4d33a2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:21.796199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.796696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.797185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.797894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.798352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.798986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.815478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] HTTP exception thrown: Instance e970fbbc-4044-415f-833d-21f6bd4d33a2 could not be found. Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Request method failure captured: Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 HTTP/1.1 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx7aB2zpSUbT_E17CNHapOmgyZ4fTM7-79ooHackEWjZ9hpz90Bp2yN1w_4L45q7U2l3cPcsu0XezJsfpWHkbvwj3NiqykGrXGgD_Uipwx1xDet2rLETZyaO-hA5rbOLxAxE_-GGCdyasbacOFvOrCmchNXdYgqMTtGOmrA2lx_HikY Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 7e45e841218949e688b3deee07fd25ed Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1638100613 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7e45e841218949e688b3deee07fd25ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7e45e841218949e688b3deee07fd25ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7e45e841218949e688b3deee07fd25ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7e45e841218949e688b3deee07fd25ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1638100613 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 7e45e841218949e688b3deee07fd25ed Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1638100613 Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1638100613-project Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8d437aff8aa486ca43edb2605a57b4e Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1638100613-project Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e970fbbc-4044-415f-833d-21f6bd4d33a2 could not be found. Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e970fbbc-4044-415f-833d-21f6bd4d33a2'} Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e970fbbc-4044-415f-833d-21f6bd4d33a2 could not be found. Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e970fbbc-4044-415f-833d-21f6bd4d33a2 could not be found. Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:21.817232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e970fbbc-4044-415f-833d-21f6bd4d33a2 could not be found. Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Returning 404 to user: Instance e970fbbc-4044-415f-833d-21f6bd4d33a2 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-826819e4-becc-41d7-a274-5c9da8e0bf15 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2" status: 404 len: 111 microversion: 2.1 time: 0.034560 Oct 15 20:59:21.821891 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1533/2647] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:21 2022] GET /compute/v2.1/servers/e970fbbc-4044-415f-833d-21f6bd4d33a2 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:21.827624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-image-129444419", "metadata": {"image_type": "test"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:21.830082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:21.836233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.836737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.837218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.838346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:21.838813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:21.839283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:21.874188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:21.902681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Getting quotas for project 57ac50a5098942e58c8ab72ff09d5159. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:21.908373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Getting quotas for user 83d85ef4ebdb452a93f24682e2bbd46c and project 57ac50a5098942e58c8ab72ff09d5159. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:21.946790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:22.137130 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d41ef8f5-7777-4e1d-831e-7c21c5282e62 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "POST /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031/action" status: 202 len: 0 microversion: 2.1 time: 0.313297 Oct 15 20:59:22.138493 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1534/2648] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:59:21 2022] POST /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031/action => generated 0 bytes in 319 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:59:22.143617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02dc693a-2988-409b-9f60-b19f7bce1df3 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] Calling method '.version_select at 0x7fe74ed4bc10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.165181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02dc693a-2988-409b-9f60-b19f7bce1df3 tempest-ServersTestJSON-1638100613 tempest-ServersTestJSON-1638100613-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-763033871" status: 202 len: 0 microversion: 2.1 time: 0.025426 Oct 15 20:59:22.165786 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1535/2649] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 20:59:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-763033871 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:59:22.171444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.172187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:22.179209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.179703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.180172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.180863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.181350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.181784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.338888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:22.348455 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-62b1ecc6-5b2f-4c8d-8593-060d712f74e4 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1322 microversion: 2.1 time: 0.181562 Oct 15 20:59:22.350001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1536/2650] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:22 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1322 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:22.355128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.355912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:22.362830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.363527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.364151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.365187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.365804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.366381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.442418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:22.452529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:22.454194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:22.460369 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b92a9b51-8fff-4824-8853-81ec0c10901a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.109339 Oct 15 20:59:22.460942 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1537/2651] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:22 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:22.468236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-103f9e71-7c39-46f2-857b-23df96d39008 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74fa46b80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.520247 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-103f9e71-7c39-46f2-857b-23df96d39008 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.057982 Oct 15 20:59:22.521318 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1538/2652] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:22 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:22.531415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ae05fd3-1b60-466b-a624-3606906811ba tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ee42dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.584079 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ae05fd3-1b60-466b-a624-3606906811ba tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.061504 Oct 15 20:59:22.584656 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1539/2653] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:22 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:22.589720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.590574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:22.597234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.597736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.598180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.598888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.599520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.599958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.615520 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a HTTP/1.1 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '15bdf694-ed96-4052-b284-784636eefd0a'} Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:22.617249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:22.621671 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddf38c0f-d489-4a75-a716-e96703fba777 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 404 len: 111 microversion: 2.1 time: 0.035426 Oct 15 20:59:22.625198 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1540/2654] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:22 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:22.733579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:22.734489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:22.741844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.742705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.743482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:22.744389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:22.745013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:22.745620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:23.009813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:23.024765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13142510-974f-4e3f-b6f2-6303c589d727 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.295284 Oct 15 20:59:23.025425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1541/2655] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:22 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:23.044723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-467270387", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:23.155882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] validate_networks() for [('a4bd3ce2-4f0b-4de3-afc0-c2866f12f81d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:59:24.152499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:24.153304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:24.153943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:24.155309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:24.155901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:24.174708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:24.175530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:24.177181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:24.178714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:24.179741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:24.181841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:24.238420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:24.269195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Getting quotas for user 2bba6c9b3f674959ad79d4a394ed9f2a and project ca9af3805eba4f69b474a8b2205f0920. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:24.305199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:59:24.307791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:24.343963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:24.388645 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3746971c-e646-4c49-8ada-8ceabd837d3c tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.361334 Oct 15 20:59:24.393096 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1542/2656] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:59:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 1367 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:59:24.400866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:24.401817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:24.427968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:24.427968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:24.427968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:24.427968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:24.427968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:24.427968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:24.429243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:24.724734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:24.737769 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cbbbd355-bbf3-4ece-a74c-0d740b678397 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1515 microversion: 2.1 time: 0.343474 Oct 15 20:59:24.749314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1543/2657] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:24 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1515 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:24.749314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:24.749314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:24.755823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:24.756115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:24.756654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:24.757594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:24.758244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:24.758879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:24.871445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:24.883138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:24.884835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:24.892139 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-62dfe3fb-cb46-40f6-aeb6-5d17bc3f0124 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.149434 Oct 15 20:59:24.894873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1544/2658] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:24 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:24.900543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6904abc3-470b-44b4-8dcc-7927c5a8d41d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74fec63a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:24.964777 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6904abc3-470b-44b4-8dcc-7927c5a8d41d tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.067900 Oct 15 20:59:24.970492 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1545/2659] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:24 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:24.980080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82d39a53-5aea-4b9a-b1b0-6dca6c4a15fa tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe754418dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:25.072401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82d39a53-5aea-4b9a-b1b0-6dca6c4a15fa tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.096472 Oct 15 20:59:25.073350 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1546/2660] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:24 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:25.095366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-44d292d4-f25e-4209-a344-a83da9b2469e req-f98a50a8-82b1-43db-93aa-737598d35a85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b", "name": "network-vif-plugged", "status": "completed", "tag": "dc6638f8-3a22-4290-9aae-3189cabf87b0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:25.111204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44d292d4-f25e-4209-a344-a83da9b2469e req-f98a50a8-82b1-43db-93aa-737598d35a85 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:25.112572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44d292d4-f25e-4209-a344-a83da9b2469e req-f98a50a8-82b1-43db-93aa-737598d35a85 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:25.114240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44d292d4-f25e-4209-a344-a83da9b2469e req-f98a50a8-82b1-43db-93aa-737598d35a85 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:25.129920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-44d292d4-f25e-4209-a344-a83da9b2469e req-f98a50a8-82b1-43db-93aa-737598d35a85 service nova] Creating event network-vif-plugged:dc6638f8-3a22-4290-9aae-3189cabf87b0 for instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b on nch2-master-1 Oct 15 20:59:25.140627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-44d292d4-f25e-4209-a344-a83da9b2469e req-f98a50a8-82b1-43db-93aa-737598d35a85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058710 Oct 15 20:59:25.146152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1547/2661] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:25.155156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:25.156042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:25.166992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:25.167839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:25.169676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:25.170851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:25.171835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:25.172385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:25.670612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:25.680140 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-236a950d-c71f-4e98-af52-290a1a2404f5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.532286 Oct 15 20:59:25.681901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1548/2662] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:25 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:25.706629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:25.707413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:25.715326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:25.716879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:25.717597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:25.718680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:25.719523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:25.720200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:25.839188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:25.849001 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa80d173-59dc-4dfd-ba7c-40e438946b63 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1266 microversion: 2.1 time: 0.165143 Oct 15 20:59:25.850694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1549/2663] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:25 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:26.033660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07ed61a0-111d-425f-ba38-1413229dcd44 tempest-ListImageFiltersNegativeTestJSON-1001977303 tempest-ListImageFiltersNegativeTestJSON-1001977303-project] Calling method '.version_select at 0x7fe74eecef70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:26.064171 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-07ed61a0-111d-425f-ba38-1413229dcd44 tempest-ListImageFiltersNegativeTestJSON-1001977303 tempest-ListImageFiltersNegativeTestJSON-1001977303-project] HTTP exception thrown: Image not found. Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07ed61a0-111d-425f-ba38-1413229dcd44 tempest-ListImageFiltersNegativeTestJSON-1001977303 tempest-ListImageFiltersNegativeTestJSON-1001977303-project] Request method failure captured: Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/images/280419d5-ade1-4908-8e9f-4c0645f6e5b0 HTTP/1.1 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8s8lxgopYT02pAeN-s0ooXm0eVy-Uu2CWpCLemYxXIAIe9FwNw4v9p37DGiIr5i0cDphjzSIHFdGdpxWnOy4e9VOLmEwSopjMydqQROpiUAOPRjWe-7kOWiUSiU2tS27IWqEeo4VVNhF_DwprpTsDN-MpMEuuvGZmw0oTySJTzIXg Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c8800104fcdd4b7b8eed6332dd249c80 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListImageFiltersNegativeTestJSON-1001977303 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c8800104fcdd4b7b8eed6332dd249c80"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c8800104fcdd4b7b8eed6332dd249c80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c8800104fcdd4b7b8eed6332dd249c80"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c8800104fcdd4b7b8eed6332dd249c80"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListImageFiltersNegativeTestJSON-1001977303 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c8800104fcdd4b7b8eed6332dd249c80 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListImageFiltersNegativeTestJSON-1001977303 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListImageFiltersNegativeTestJSON-1001977303-project Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 37534369e7d644d78af7e3a93ab25462 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListImageFiltersNegativeTestJSON-1001977303-project Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '280419d5-ade1-4908-8e9f-4c0645f6e5b0'} Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 280419d5-ade1-4908-8e9f-4c0645f6e5b0 Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:26.066667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 280419d5-ade1-4908-8e9f-4c0645f6e5b0 could not be found. Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:26.070798 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:26.074686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07ed61a0-111d-425f-ba38-1413229dcd44 tempest-ListImageFiltersNegativeTestJSON-1001977303 tempest-ListImageFiltersNegativeTestJSON-1001977303-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:26.076257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07ed61a0-111d-425f-ba38-1413229dcd44 tempest-ListImageFiltersNegativeTestJSON-1001977303 tempest-ListImageFiltersNegativeTestJSON-1001977303-project] 10.222.0.16 "GET /compute/v2.1/images/280419d5-ade1-4908-8e9f-4c0645f6e5b0" status: 404 len: 62 microversion: 2.1 time: 0.224273 Oct 15 20:59:26.076998 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1550/2664] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:25 2022] GET /compute/v2.1/images/280419d5-ade1-4908-8e9f-4c0645f6e5b0 => generated 62 bytes in 226 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:26.082847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:26.098183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:26.128141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:26.129024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:26.132157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:26.134918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:26.136781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:26.137980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:27.301392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:27.336788 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73d27c9f-0b01-4d8b-bc58-23780cdd64d6 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d" status: 200 len: 1516 microversion: 2.1 time: 1.256926 Oct 15 20:59:27.339312 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1551/2665] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:26 2022] GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d => generated 1516 bytes in 1260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:27.349320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:27.351958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:27.373846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:27.383028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:27.389772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:27.397601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:27.398743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:27.399433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:27.618270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:27.629542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:27.635402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:27.648419 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-49936e48-5ca9-4973-8a24-a06eb7aaa0c6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.305274 Oct 15 20:59:27.651840 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1552/2666] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:27 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:27.662034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba1d3247-bc16-4e79-8db7-8f62fda76691 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74ed5d5e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:27.755945 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba1d3247-bc16-4e79-8db7-8f62fda76691 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.102652 Oct 15 20:59:27.760474 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1553/2667] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:27 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:27.781784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc7ad666-7f5f-4fd8-9aad-93954600cc33 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ee42dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:27.870698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fc7ad666-7f5f-4fd8-9aad-93954600cc33 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.102798 Oct 15 20:59:27.872499 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1554/2668] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:27 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:27.877205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:27.877941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:27.895914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:27.897432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:27.899570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:27.901090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:27.902445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:27.903851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:28.424926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:28.487102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6309eb07-e652-426d-bd6a-a199ab8a4256 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.613270 Oct 15 20:59:28.490985 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1555/2669] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:27 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:28.504630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:28.505297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:28.532245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:28.532752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:28.533432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:28.534353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:28.541791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:28.542643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:28.678517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:28.689363 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3cab23e-6115-454a-a7f9-ee1da07238c2 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1272 microversion: 2.1 time: 0.195093 Oct 15 20:59:28.696731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1556/2670] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:28 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1272 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:28.702092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1e66d582-f421-462f-ac21-016b224ee65e req-95f99996-3418-4c84-bdd3-fcf823983fd1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "359b81c9-e8fc-491e-95d7-b2887dfb032f", "name": "network-vif-plugged", "status": "completed", "tag": "4ae4edec-e1c4-4b33-a749-85c9de8a0ed8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:28.725099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1e66d582-f421-462f-ac21-016b224ee65e req-95f99996-3418-4c84-bdd3-fcf823983fd1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:28.730901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1e66d582-f421-462f-ac21-016b224ee65e req-95f99996-3418-4c84-bdd3-fcf823983fd1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:28.732182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1e66d582-f421-462f-ac21-016b224ee65e req-95f99996-3418-4c84-bdd3-fcf823983fd1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:28.755105 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1e66d582-f421-462f-ac21-016b224ee65e req-95f99996-3418-4c84-bdd3-fcf823983fd1 service nova] Creating event network-vif-plugged:4ae4edec-e1c4-4b33-a749-85c9de8a0ed8 for instance 359b81c9-e8fc-491e-95d7-b2887dfb032f on nch1-master-1 Oct 15 20:59:28.770608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1e66d582-f421-462f-ac21-016b224ee65e req-95f99996-3418-4c84-bdd3-fcf823983fd1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069999 Oct 15 20:59:28.771630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1557/2671] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:28.783620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:28.784758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:28.794198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:28.794863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:28.795455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:28.796383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:28.797582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:28.798166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:29.470825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:29.480109 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69317467-749e-4af2-a230-50a50e84f69a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b" status: 200 len: 1515 microversion: 2.1 time: 0.706111 Oct 15 20:59:29.484218 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1558/2672] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:28 2022] GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b => generated 1515 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:29.502873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:29.504143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:29.534285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:29.540005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:29.540893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:29.541901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:29.542645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:29.545373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:29.687147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:29.716812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:29.724711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:29.747876 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be14e0f3-6e73-4408-ad54-d591953bf586 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.262269 Oct 15 20:59:29.769751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1559/2673] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:29 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:29.803188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69fe1c6e-3138-44a2-aaff-c69ae425a132 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74f9eb430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:29.930344 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69fe1c6e-3138-44a2-aaff-c69ae425a132 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.158786 Oct 15 20:59:29.931246 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1560/2674] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:29 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:29.945858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4030d17-51cf-4f40-b0b2-5b2dd661b1b6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f8b1dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:30.061157 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4030d17-51cf-4f40-b0b2-5b2dd661b1b6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1413 microversion: 2.1 time: 0.128503 Oct 15 20:59:30.070963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1561/2675] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:29 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1413 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:30.101341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:30.106736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:30.116938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:30.118229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:30.119038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:30.120098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:30.120724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:30.121376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:30.176240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:30.505404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3636d645-83a6-4e03-92a1-f2cbedd2c895 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "DELETE /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d" status: 204 len: 0 microversion: 2.1 time: 0.431073 Oct 15 20:59:30.507293 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1562/2676] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:30 2022] DELETE /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:30.514859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:30.518294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:30.536382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:30.537205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:30.539211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:30.542804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:30.543497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:30.545916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:30.853650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:30.864387 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d3f1d96-77f2-4411-b024-92c22855b803 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1308 microversion: 2.1 time: 0.355612 Oct 15 20:59:30.865997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1563/2677] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:30 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1308 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:30.889964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:30.891533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:30.911509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:30.912600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:30.913293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:30.914362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:30.916228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:30.916833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:31.235636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:31.250961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-195098b7-62cf-46bc-ab8a-5109c682e7b8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1314 microversion: 2.1 time: 0.382524 Oct 15 20:59:31.251804 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1564/2678] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:30 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1314 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:31.284848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:31.287468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:31.304181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:31.304709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:31.307606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:31.311630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:31.312158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:31.312623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:31.709637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:31.724216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-846507a4-4deb-47db-af53-d69baed6a301 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d" status: 200 len: 1522 microversion: 2.1 time: 0.450769 Oct 15 20:59:31.725173 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1565/2679] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:31 2022] GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d => generated 1522 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:31.744036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:31.746444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:31.765316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:31.766126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:31.767384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:31.770136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:31.771478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:31.772077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:31.828579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:31.975853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:31.998330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:32.004069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:32.022503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-004ff65b-9675-4bd1-88d7-2a034090ccd8 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.292621 Oct 15 20:59:32.023448 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1566/2680] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:31 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:32.067117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0a49c28f-64bc-4616-af22-13b4eea10838 req-85ea6abf-88f7-4948-b5ab-ffea2dceab4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1910f53a-ef2a-4f63-836d-9850556d1762", "name": "network-vif-plugged", "status": "completed", "tag": "5f4c041d-5d6e-4a7f-a157-e88a7ad86f9e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:32.090062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a49c28f-64bc-4616-af22-13b4eea10838 req-85ea6abf-88f7-4948-b5ab-ffea2dceab4b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:32.094780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a49c28f-64bc-4616-af22-13b4eea10838 req-85ea6abf-88f7-4948-b5ab-ffea2dceab4b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:32.098795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a49c28f-64bc-4616-af22-13b4eea10838 req-85ea6abf-88f7-4948-b5ab-ffea2dceab4b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:32.135038 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0a49c28f-64bc-4616-af22-13b4eea10838 req-85ea6abf-88f7-4948-b5ab-ffea2dceab4b service nova] Creating event network-vif-plugged:5f4c041d-5d6e-4a7f-a157-e88a7ad86f9e for instance 1910f53a-ef2a-4f63-836d-9850556d1762 on nch2-master-1 Oct 15 20:59:32.147201 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0a49c28f-64bc-4616-af22-13b4eea10838 req-85ea6abf-88f7-4948-b5ab-ffea2dceab4b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105783 Oct 15 20:59:32.148090 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1567/2681] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:32.163171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06392fea-800d-4907-8728-54566380320e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74f881670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:34.206969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:34.292960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:41.082956 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-06392fea-800d-4907-8728-54566380320e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 8.930903 Oct 15 20:59:41.096901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1568/2682] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:32 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 8947 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:41.102342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0951f4ea-7935-42d0-8f77-ff75063c7491 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f8b13a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:41.170124 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0951f4ea-7935-42d0-8f77-ff75063c7491 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1422 microversion: 2.1 time: 0.071681 Oct 15 20:59:41.171498 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1569/2683] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:41 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1422 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:41.177151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5b2d0c25-c7af-4a30-bed7-dd2ca6caa98a req-f58b7678-eead-43f0-a29c-8472d4e38887 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6be11d80-e61b-4556-9907-c828a8d94fed", "tag": "8f3bed3a-535d-49a7-a40e-0f0e438f875d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:41.191583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5b2d0c25-c7af-4a30-bed7-dd2ca6caa98a req-f58b7678-eead-43f0-a29c-8472d4e38887 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.192578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5b2d0c25-c7af-4a30-bed7-dd2ca6caa98a req-f58b7678-eead-43f0-a29c-8472d4e38887 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.193525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5b2d0c25-c7af-4a30-bed7-dd2ca6caa98a req-f58b7678-eead-43f0-a29c-8472d4e38887 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.213823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5b2d0c25-c7af-4a30-bed7-dd2ca6caa98a req-f58b7678-eead-43f0-a29c-8472d4e38887 service nova] Creating event network-changed:8f3bed3a-535d-49a7-a40e-0f0e438f875d for instance 6be11d80-e61b-4556-9907-c828a8d94fed on nch1-master-1 Oct 15 20:59:41.221717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5b2d0c25-c7af-4a30-bed7-dd2ca6caa98a req-f58b7678-eead-43f0-a29c-8472d4e38887 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048706 Oct 15 20:59:41.223368 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1570/2684] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:41.230550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:41.231884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:41.239821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.240622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.241358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.242336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.242909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.243669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.531717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:41.556535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e43608a-7d75-45ca-be88-794d2e5dacac tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1489 microversion: 2.1 time: 0.331470 Oct 15 20:59:41.573940 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1571/2685] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:41 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1489 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:41.590196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:41.590862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:41.597694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.598203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.598651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.599384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.599854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.605184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.897995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:41.913432 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c917c4f0-5cec-4ba0-98fd-195e5f71bcf1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1314 microversion: 2.1 time: 0.333146 Oct 15 20:59:41.914294 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1572/2686] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:41 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1314 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:41.919638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:41.920377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:41.943566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.944077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.944529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.945320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:41.954430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:41.955164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:41.992612 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] HTTP exception thrown: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d HTTP/1.1 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '183834bb-a5a3-43c7-a9c4-20c50652331d'} Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:41.994532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 404 to user: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:41.999931 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9cca748-8de4-4422-a6b9-5b9fe0caf35c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d" status: 404 len: 111 microversion: 2.1 time: 0.080737 Oct 15 20:59:42.006343 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1573/2687] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:41 2022] GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:42.013280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:42.013894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:42.038141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.038558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.039544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:42.040459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.041555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.042152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:42.203349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:42.214119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:42.217136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:42.229651 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fe2f4f0-19b8-49ef-a9bd-d7e2a0b5779a tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.220778 Oct 15 20:59:42.245260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1574/2688] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:42 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:42.258233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-53483996-9af2-459b-b644-11fb1287ce27 req-a75a3c01-d6b8-4c19-aa14-496c84c8fe8b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "name": "network-vif-plugged", "status": "completed", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:42.269343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-53483996-9af2-459b-b644-11fb1287ce27 req-a75a3c01-d6b8-4c19-aa14-496c84c8fe8b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.270039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-53483996-9af2-459b-b644-11fb1287ce27 req-a75a3c01-d6b8-4c19-aa14-496c84c8fe8b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.270630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-53483996-9af2-459b-b644-11fb1287ce27 req-a75a3c01-d6b8-4c19-aa14-496c84c8fe8b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:42.285162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-53483996-9af2-459b-b644-11fb1287ce27 req-a75a3c01-d6b8-4c19-aa14-496c84c8fe8b service nova] Creating event network-vif-plugged:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 20:59:42.292180 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-53483996-9af2-459b-b644-11fb1287ce27 req-a75a3c01-d6b8-4c19-aa14-496c84c8fe8b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045255 Oct 15 20:59:42.293398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1575/2689] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:42.452613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAaction247Test-server-489787828", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:42.525754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:42.526351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:42.527057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:42.527722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:42.528223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:42.538348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.538839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.540386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:42.541354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.541827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.542319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:42.561415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Getting quotas for project 6b955a7e7fb04b569773c0453624911a. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:42.567390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Getting quotas for user 6925000aee9d499a85c6dc1282d09993 and project 6b955a7e7fb04b569773c0453624911a. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:42.582490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:59:42.584939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:42.655444 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5be93a8-85ca-43ac-8c10-9801d193476c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.360379 Oct 15 20:59:42.656279 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1576/2690] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:59:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 362 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 20:59:42.776772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Calling method '.version_select at 0x7fe74fb07790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:42.810916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:59:42.811527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Neutron security group 25076b3a-67d7-419c-9be0-e4e26297d65d not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:59:42.812064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] HTTP exception thrown: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.812064 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Request method failure captured: Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-security-groups/25076b3a-67d7-419c-9be0-e4e26297d65d HTTP/1.1 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx87jY74Uf9_EqaQHmoAjZvIy6zECfyiUgSkRbaFkfx9093E5sWbOBolBh3odKKii7GWsJA4AOl-OXmrbsBRL-HfEVH2D7REmo3Vr-GtUiJXQRbWXkAf2s8p-HO8-Bw3lUNSXjE9VOKgqKTaFUdhG-U-yF9HuyHPiNAQR8ZF7gJYUmg Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 598073dd466a4417a1a7e31ba2dc95d2 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '25076b3a-67d7-419c-9be0-e4e26297d65d'} Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:59:42.816273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Returning 404 to user: Security group 25076b3a-67d7-419c-9be0-e4e26297d65d does not exist Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-b62daa5a-7a34-4f49-b639-1f8d635dfa92'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:42.821064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-49c18378-54b1-48ea-a06a-bbfc9abaa167 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/25076b3a-67d7-419c-9be0-e4e26297d65d" status: 404 len: 194 microversion: 2.1 time: 0.160423 Oct 15 20:59:42.826409 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1577/2691] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:59:42 2022] DELETE /compute/v2.1/os-security-groups/25076b3a-67d7-419c-9be0-e4e26297d65d => generated 194 bytes in 167 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:42.828651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e554d3b-fc0d-4a8b-a2df-ea92a2b20e88 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74eece430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:42.886931 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e554d3b-fc0d-4a8b-a2df-ea92a2b20e88 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 200 len: 1422 microversion: 2.1 time: 0.062044 Oct 15 20:59:42.887490 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1578/2692] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:42 2022] GET /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 1422 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:42.907627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:42.908267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:42.915896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.916277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.916721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:42.917503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:42.917969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:42.919085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.180569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:43.197322 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5268143b-5500-4746-ac47-ae8d974bde52 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1490 microversion: 2.1 time: 0.293520 Oct 15 20:59:43.201612 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1579/2693] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:42 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1490 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:43.221626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:43.229448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:43.235957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'changes-since': datetime.datetime(2051, 1, 1, 12, 34, tzinfo=datetime.timezone.utc), 'vm_state': ['active'], 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:43.243506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:43.244427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:43.245129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.246143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:43.246842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:43.247488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.357764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d74950dd-cb59-4692-9a99-3e4ce6d38d8a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?changes-since=2051-01-01T12%3A34%3A00Z&status=ACTIVE" status: 200 len: 15 microversion: 2.1 time: 0.151784 Oct 15 20:59:43.358886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1580/2694] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Oct 15 20:59:43 2022] GET /compute/v2.1/servers?changes-since=2051-01-01T12%3A34%3A00Z&status=ACTIVE => generated 15 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:59:43.368581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fce46a79-aac6-4094-97a9-b39eb4ea774e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74ee00af0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:43.423451 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fce46a79-aac6-4094-97a9-b39eb4ea774e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.063222 Oct 15 20:59:43.424223 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1581/2695] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:43 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:43.432664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:43.433681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:43.445672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:43.446344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:43.446785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.447498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:43.447988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:43.448496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.613292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:43.625963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:43.629102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:43.631942 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60afbae3-ce6d-48dd-89b2-427b909d51a0 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1276 microversion: 2.47 time: 0.206814 Oct 15 20:59:43.645106 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1582/2696] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:43 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1276 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:43.654710 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7485a0a2-cb2a-4de7-be00-981f2967d7c1 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/" status: 404 len: 112 microversion: - time: 0.012177 Oct 15 20:59:43.654710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1583/2697] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Oct 15 20:59:43 2022] DELETE /compute/v2.1/os-security-groups/ => generated 112 bytes in 14 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 15 20:59:43.662529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:43.662529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:43.664924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:43.665464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:43.665911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.666637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:43.667098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:43.667538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:43.896110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:43.908290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f80ce8e5-2410-408e-a424-1be85808c652 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1314 microversion: 2.1 time: 0.256690 Oct 15 20:59:43.909414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1584/2698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:43 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:43.915640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-1880484966", "imageRef": "b7725f7d-013b-4e75-8927-10802f221d8d", "flavorRef": "452", "networks": [{"uuid": "c130cb51-e60d-43b7-af80-10e77fbe9595"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:43.995560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] validate_networks() for [('c130cb51-e60d-43b7-af80-10e77fbe9595', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 20:59:44.560451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:44.561606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:44.563534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:44.564190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:44.564833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:44.574449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:44.575245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:44.575912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:44.576868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:44.577425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:44.578085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:44.605331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 20:59:44.618076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for user 5d93a680570f4d23b1f6df534bc082b2 and project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 20:59:44.635714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=95,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:44.643063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 20:59:44.646151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b7725f7d-013b-4e75-8927-10802f221d8d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 20:59:44.719845 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-210cda74-0eb2-4830-a119-810d97c4e4b9 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.808951 Oct 15 20:59:44.721722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1585/2699] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 20:59:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 811 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 20:59:44.733453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:44.734462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:44.743331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:44.743721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:44.744171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:44.744971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:44.745425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:44.747398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:44.920951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:44.929150 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb3af1e3-a41d-4b9c-b76f-a0c81259c84c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1489 microversion: 2.1 time: 0.206298 Oct 15 20:59:44.929691 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1586/2700] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:44 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1489 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:44.952686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9256c311-3df0-4583-9609-ea0016a501dc req-14017cac-16bb-419d-982e-35c14670c52f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "183834bb-a5a3-43c7-a9c4-20c50652331d", "name": "network-vif-deleted", "tag": "5fddc112-0579-4080-a39f-1b786f647006"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:44.964605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9256c311-3df0-4583-9609-ea0016a501dc req-14017cac-16bb-419d-982e-35c14670c52f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:44.965707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9256c311-3df0-4583-9609-ea0016a501dc req-14017cac-16bb-419d-982e-35c14670c52f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:44.966574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9256c311-3df0-4583-9609-ea0016a501dc req-14017cac-16bb-419d-982e-35c14670c52f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:44.983842 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9256c311-3df0-4583-9609-ea0016a501dc req-14017cac-16bb-419d-982e-35c14670c52f service nova] Creating event network-vif-deleted:5fddc112-0579-4080-a39f-1b786f647006 for instance 183834bb-a5a3-43c7-a9c4-20c50652331d on nch1-master-1 Oct 15 20:59:44.989548 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9256c311-3df0-4583-9609-ea0016a501dc req-14017cac-16bb-419d-982e-35c14670c52f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046022 Oct 15 20:59:44.990494 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1587/2701] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:44.999988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:45.000921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:45.008388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.008945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.009421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:45.010133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.010603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.011132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:45.126383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:45.144449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:45.147293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:45.165689 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fbf08ea-6e2c-431b-b9d5-6eb15da54886 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.173671 Oct 15 20:59:45.167863 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1588/2702] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:44 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:45.182754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44c1af08-5f9c-41a8-90a9-6fc1b1618d4a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44c1af08-5f9c-41a8-90a9-6fc1b1618d4a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers?changes-since=2011%2F01%2F01 HTTP/1.1 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: action_args: {} Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '2011/01/01' is not a 'date-time' Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['changes-since']['items']: Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'date-time', 'type': 'string'} Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance[0]: Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '2011/01/01' Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Oct 15 20:59:45.191012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44c1af08-5f9c-41a8-90a9-6fc1b1618d4a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 400 to user: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:45.197221 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44c1af08-5f9c-41a8-90a9-6fc1b1618d4a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?changes-since=2011%2F01%2F01" status: 400 len: 146 microversion: 2.1 time: 0.015940 Oct 15 20:59:45.209027 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1589/2703] 10.222.0.16 () {58 vars in 1182 bytes} [Sat Oct 15 20:59:45 2022] GET /compute/v2.1/servers?changes-since=2011%2F01%2F01 => generated 146 bytes in 40 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:59:45.214052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d295464-1c68-4f26-887b-6976a3173572 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Calling method '.version_select at 0x7fe74f9d80d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:45.425480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d295464-1c68-4f26-887b-6976a3173572 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups" status: 200 len: 707 microversion: 2.1 time: 0.214945 Oct 15 20:59:45.426258 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1590/2704] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 20:59:45 2022] GET /compute/v2.1/os-security-groups => generated 707 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:45.431953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-204fa1f4-0707-4281-8a81-a019829ff403 req-856d0130-d185-4caa-bb0c-5ad52291ce56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be11d80-e61b-4556-9907-c828a8d94fed", "name": "network-vif-plugged", "status": "completed", "tag": "8f3bed3a-535d-49a7-a40e-0f0e438f875d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:45.443141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-204fa1f4-0707-4281-8a81-a019829ff403 req-856d0130-d185-4caa-bb0c-5ad52291ce56 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.443624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-204fa1f4-0707-4281-8a81-a019829ff403 req-856d0130-d185-4caa-bb0c-5ad52291ce56 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.444056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-204fa1f4-0707-4281-8a81-a019829ff403 req-856d0130-d185-4caa-bb0c-5ad52291ce56 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:45.456612 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-204fa1f4-0707-4281-8a81-a019829ff403 req-856d0130-d185-4caa-bb0c-5ad52291ce56 service nova] Creating event network-vif-plugged:8f3bed3a-535d-49a7-a40e-0f0e438f875d for instance 6be11d80-e61b-4556-9907-c828a8d94fed on nch1-master-1 Oct 15 20:59:45.466399 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-204fa1f4-0707-4281-8a81-a019829ff403 req-856d0130-d185-4caa-bb0c-5ad52291ce56 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038810 Oct 15 20:59:45.467007 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1591/2705] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:45.472100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e701f8a6-5533-4187-b82e-4d092207aae0 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe754706d30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:45.555437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e701f8a6-5533-4187-b82e-4d092207aae0 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.087204 Oct 15 20:59:45.564239 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1592/2706] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:45 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:45.574132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:45.576207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:45.584157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.584657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.585177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:45.586452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.586925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.587367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:45.679920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:45.689985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:45.691615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:45.693180 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd9c1ca7-f189-49ee-8fdc-7ddf666c27e7 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1338 microversion: 2.47 time: 0.127770 Oct 15 20:59:45.697554 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1593/2707] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:45 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1338 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:45.708258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:45.709736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:45.719164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.719835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.731234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:45.732219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:45.733106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:45.733579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.000634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:46.025097 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ad9878c-549e-4c7d-b257-45d44bbe6d04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1212 microversion: 2.1 time: 0.322926 Oct 15 20:59:46.025940 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1594/2708] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:45 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1212 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:46.041555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:46.046264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:46.064035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.064654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.065132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.083095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.087649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.088684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.373650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:46.386182 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09b7ea56-fb02-4190-bc5e-94deb468d06a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1314 microversion: 2.1 time: 0.359063 Oct 15 20:59:46.402267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1595/2709] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:46 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1314 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:46.409591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:46.410519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:46.418732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.419319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.419846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.420843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.421716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.422441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.453649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:46.523826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:46.580384 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d20ec545-24e8-411c-99f4-60a9de35f9d3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 204 len: 0 microversion: 2.1 time: 0.176237 Oct 15 20:59:46.581356 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1596/2710] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:46 2022] DELETE /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:46.586740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:46.592869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:46.593704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:46.600463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.601248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.601713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.602470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.602941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.603369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.640558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:46.685204 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-464c3cbb-c11e-4ba2-b492-dadc979d2897 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 681 microversion: 2.1 time: 0.101022 Oct 15 20:59:46.685774 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1597/2711] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:59:46 2022] GET /compute/v2.1/servers => generated 681 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:46.691345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fcfac6e8-d49a-44c1-beae-6a8703ca3b0c tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Calling method '.version_select at 0x7fe74fc13820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:46.831714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-fcfac6e8-d49a-44c1-beae-6a8703ca3b0c tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Error message: {"NeutronError": {"type": "SecurityGroupCannotRemoveDefault", "message": "Insufficient rights for removing default security group.", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:59:46.832488 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-fcfac6e8-d49a-44c1-beae-6a8703ca3b0c tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] HTTP exception thrown: Insufficient rights for removing default security group. Oct 15 20:59:46.832488 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fcfac6e8-d49a-44c1-beae-6a8703ca3b0c tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Request method failure captured: Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-security-groups/cb4f9c33-a66e-4311-9aac-f1851766c7f8 HTTP/1.1 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx87jY74Uf9_EqaQHmoAjZvIy6zECfyiUgSkRbaFkfx9093E5sWbOBolBh3odKKii7GWsJA4AOl-OXmrbsBRL-HfEVH2D7REmo3Vr-GtUiJXQRbWXkAf2s8p-HO8-Bw3lUNSXjE9VOKgqKTaFUdhG-U-yF9HuyHPiNAQR8ZF7gJYUmg Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 598073dd466a4417a1a7e31ba2dc95d2 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: exception: Insufficient rights for removing default security group. Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'cb4f9c33-a66e-4311-9aac-f1851766c7f8'} Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1073, in delete_security_group Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:59:46.834760 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Insufficient rights for removing default security group. Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Insufficient rights for removing default security group. Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Insufficient rights for removing default security group. Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Insufficient rights for removing default security group. Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fcfac6e8-d49a-44c1-beae-6a8703ca3b0c tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Returning 400 to user: Insufficient rights for removing default security group. Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e4290028-e3f7-4136-9311-a58375e1178c'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fcfac6e8-d49a-44c1-beae-6a8703ca3b0c tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/cb4f9c33-a66e-4311-9aac-f1851766c7f8" status: 400 len: 182 microversion: 2.1 time: 0.150908 Oct 15 20:59:46.839900 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1598/2712] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 20:59:46 2022] DELETE /compute/v2.1/os-security-groups/cb4f9c33-a66e-4311-9aac-f1851766c7f8 => generated 182 bytes in 152 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:59:46.851391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:46.852262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:46.859788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.860397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.860918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.861724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:46.862504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:46.863007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:46.933101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:46.942888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:46.944293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:46.951153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-545ae6a0-1b38-42aa-9933-fb88631548dd tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.110266 Oct 15 20:59:46.951756 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1599/2713] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:46 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:46.956480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f72b5d59-7d39-4178-b947-6bf586afc5e9 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74faa0280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.005175 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f72b5d59-7d39-4178-b947-6bf586afc5e9 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1423 microversion: 2.1 time: 0.052329 Oct 15 20:59:47.005767 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1600/2714] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:46 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1423 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:47.010353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.010957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:47.018509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.019005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.019447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.020156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.020625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.021071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.160572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.169164 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3216285b-c2f6-4f01-828b-3e8d60d71a16 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1495 microversion: 2.1 time: 0.161753 Oct 15 20:59:47.169739 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1601/2715] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1495 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:47.174237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.179077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.179548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.servers [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Removing options 'limit' from query {{(pid=138688) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Oct 15 20:59:47.180211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:47.185316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.185804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.186252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.187031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.187504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.187941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.214266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Listed batch of 2 results from cell out of 102 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:47.243241 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c2e40ae-86e7-46fa-b64e-4462423174f8 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=102" status: 200 len: 681 microversion: 2.1 time: 0.072465 Oct 15 20:59:47.243834 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1602/2716] 10.222.0.16 () {58 vars in 1144 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers?limit=102 => generated 681 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:47.248339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.248940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:47.255051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.255538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.255973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.256669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.257152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.257576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.325600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.333710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.335158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.336285 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ef3cac1-233d-4f01-ab50-bd67814417aa tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1338 microversion: 2.47 time: 0.091533 Oct 15 20:59:47.336834 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1603/2717] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1338 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:47.341733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Calling method '.version_select at 0x7fe74f9eb670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.363756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:59:47.364346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Neutron security group ea08ee66-278e-4664-a977-9dc94019f426 not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:59:47.364868 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] HTTP exception thrown: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.364868 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Request method failure captured: Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-security-groups/ea08ee66-278e-4664-a977-9dc94019f426 HTTP/1.1 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx87jY74Uf9_EqaQHmoAjZvIy6zECfyiUgSkRbaFkfx9093E5sWbOBolBh3odKKii7GWsJA4AOl-OXmrbsBRL-HfEVH2D7REmo3Vr-GtUiJXQRbWXkAf2s8p-HO8-Bw3lUNSXjE9VOKgqKTaFUdhG-U-yF9HuyHPiNAQR8ZF7gJYUmg Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 598073dd466a4417a1a7e31ba2dc95d2 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ea08ee66-278e-4664-a977-9dc94019f426'} Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:59:47.366780 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Returning 404 to user: Security group ea08ee66-278e-4664-a977-9dc94019f426 does not exist Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-ac44c009-1779-4a3f-b3c6-dc5f83a06c20'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e73eb1ec-acf1-4976-a080-38499572eee2 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/ea08ee66-278e-4664-a977-9dc94019f426" status: 404 len: 194 microversion: 2.1 time: 0.030401 Oct 15 20:59:47.371098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1604/2718] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/os-security-groups/ea08ee66-278e-4664-a977-9dc94019f426 => generated 194 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:47.374983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.374983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:47.381971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.382470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.382933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.383635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.384105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.384539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.450718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.458630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5362026-34f7-4022-a6da-1473009df2e4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1204 microversion: 2.1 time: 0.088919 Oct 15 20:59:47.459133 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1605/2719] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1204 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:47.463458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d0f3220-fd3e-4ec2-b443-5e652bdd6a08 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d0f3220-fd3e-4ec2-b443-5e652bdd6a08 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers?limit=-1 HTTP/1.1 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: action_args: {} Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '-1' does not match '^[0-9]*$' Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['limit']['items']: Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minimum': 0, Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance[0]: Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '-1' Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Oct 15 20:59:47.467473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d0f3220-fd3e-4ec2-b443-5e652bdd6a08 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 400 to user: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d0f3220-fd3e-4ec2-b443-5e652bdd6a08 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=-1" status: 400 len: 127 microversion: 2.1 time: 0.008761 Oct 15 20:59:47.471418 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1606/2720] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers?limit=-1 => generated 127 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:59:47.473650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Action: 'update', calling method: .version_select at 0x7fe74f8b1d30>, body: {"security_group": {"name": "tempest-sg-429010892", "description": "tempest-description-2057954064"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:47.497609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 20:59:47.498230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Neutron security group df2f0c9d-18da-467d-8026-c350e75fa628 not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 20:59:47.498750 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] HTTP exception thrown: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.498750 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Request method failure captured: Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/os-security-groups/df2f0c9d-18da-467d-8026-c350e75fa628 HTTP/1.1 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 101 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx87jY74Uf9_EqaQHmoAjZvIy6zECfyiUgSkRbaFkfx9093E5sWbOBolBh3odKKii7GWsJA4AOl-OXmrbsBRL-HfEVH2D7REmo3Vr-GtUiJXQRbWXkAf2s8p-HO8-Bw3lUNSXjE9VOKgqKTaFUdhG-U-yF9HuyHPiNAQR8ZF7gJYUmg Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/39d4d95d057544d2a6aa821f38be4e9a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 39d4d95d057544d2a6aa821f38be4e9a Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 598073dd466a4417a1a7e31ba2dc95d2 Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-2058075023-project Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: {"security_group": {"name": "tempest-sg-429010892", "description": "tempest-description-2057954064"}} Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'df2f0c9d-18da-467d-8026-c350e75fa628', 'body': {'security_group': {'name': 'tempest-sg-429010892', 'description': 'tempest-description-2057954064'}}} Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 20:59:47.503665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 246, in update Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 248, in update Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] Returning 404 to user: Security group df2f0c9d-18da-467d-8026-c350e75fa628 does not exist Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e31c10a5-6e8d-4ae7-bc6d-6749a56c506d'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4365eade-1da9-497a-863e-3e4a43e87659 tempest-SecurityGroupsNegativeTestJSON-2058075023 tempest-SecurityGroupsNegativeTestJSON-2058075023-project] 10.222.0.16 "PUT /compute/v2.1/os-security-groups/df2f0c9d-18da-467d-8026-c350e75fa628" status: 404 len: 194 microversion: 2.1 time: 0.034930 Oct 15 20:59:47.507300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1607/2721] 10.222.0.16 () {60 vars in 1290 bytes} [Sat Oct 15 20:59:47 2022] PUT /compute/v2.1/os-security-groups/df2f0c9d-18da-467d-8026-c350e75fa628 => generated 194 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:47.515083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.515083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:47.521977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.522474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.522920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.523631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.524094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.524523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.676979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.685797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d65b294-df17-4844-aa0b-1a09b7bb6b70 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1314 microversion: 2.1 time: 0.179096 Oct 15 20:59:47.686567 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1608/2722] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1314 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:47.706387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f868240-bc9d-465e-ae1b-41af676b44ae tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f868240-bc9d-465e-ae1b-41af676b44ae tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers?limit=testing HTTP/1.1 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: action_args: {} Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'testing' does not match '^[0-9]*$' Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['limit']['items']: Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minimum': 0, Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance[0]: Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'testing' Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Oct 15 20:59:47.710424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f868240-bc9d-465e-ae1b-41af676b44ae tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 400 to user: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f868240-bc9d-465e-ae1b-41af676b44ae tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=testing" status: 400 len: 137 microversion: 2.1 time: 0.009266 Oct 15 20:59:47.714667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1609/2723] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers?limit=testing => generated 137 bytes in 25 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 20:59:47.720865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c4c7aff6-756f-4d93-9e29-64edffc9a787 req-c54054df-fedb-40d1-a59c-d1830257e6e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "359b81c9-e8fc-491e-95d7-b2887dfb032f", "name": "network-vif-unplugged", "status": "completed", "tag": "4ae4edec-e1c4-4b33-a749-85c9de8a0ed8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:47.732247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c4c7aff6-756f-4d93-9e29-64edffc9a787 req-c54054df-fedb-40d1-a59c-d1830257e6e7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.732748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c4c7aff6-756f-4d93-9e29-64edffc9a787 req-c54054df-fedb-40d1-a59c-d1830257e6e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.733225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c4c7aff6-756f-4d93-9e29-64edffc9a787 req-c54054df-fedb-40d1-a59c-d1830257e6e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.744527 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c4c7aff6-756f-4d93-9e29-64edffc9a787 req-c54054df-fedb-40d1-a59c-d1830257e6e7 service nova] Creating event network-vif-unplugged:4ae4edec-e1c4-4b33-a749-85c9de8a0ed8 for instance 359b81c9-e8fc-491e-95d7-b2887dfb032f on nch1-master-1 Oct 15 20:59:47.751307 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c4c7aff6-756f-4d93-9e29-64edffc9a787 req-c54054df-fedb-40d1-a59c-d1830257e6e7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037477 Oct 15 20:59:47.752863 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1610/2724] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:47.757508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74667249-1aa3-4b7c-a017-17b8d78f5f21 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.763501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-74667249-1aa3-4b7c-a017-17b8d78f5f21 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.764295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-74667249-1aa3-4b7c-a017-17b8d78f5f21 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'flavor': 'non_existing_flavor', 'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:47.773098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.servers [None req-74667249-1aa3-4b7c-a017-17b8d78f5f21 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Flavor 'non_existing_flavor' could not be found {{(pid=138688) _get_servers /opt/stack/nova/nova/api/openstack/compute/servers.py:336}} Oct 15 20:59:47.773098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-74667249-1aa3-4b7c-a017-17b8d78f5f21 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?flavor=non_existing_flavor" status: 200 len: 15 microversion: 2.1 time: 0.018690 Oct 15 20:59:47.773098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1611/2725] 10.222.0.16 () {58 vars in 1178 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers?flavor=non_existing_flavor => generated 15 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:59:47.785947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.791881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.792830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'image': 'non_existing_image', 'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:47.804167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.805470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.806020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.808299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.808952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.810176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.913572 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77cb3e42-c2a1-4ad2-a4ce-9f4869299070 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?image=non_existing_image" status: 200 len: 15 microversion: 2.1 time: 0.131457 Oct 15 20:59:47.914353 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1612/2726] 10.222.0.16 () {58 vars in 1174 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers?image=non_existing_image => generated 15 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:59:47.927236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:47.933018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:47.934142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'name': 'non_existing_server_name', 'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:47.939706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.940200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.940770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:47.941416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:47.942068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:47.942509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.049480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e88fd30f-6bb9-4977-b82e-b566b328637c tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?name=non_existing_server_name" status: 200 len: 15 microversion: 2.1 time: 0.126319 Oct 15 20:59:48.051287 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1613/2727] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Oct 15 20:59:47 2022] GET /compute/v2.1/servers?name=non_existing_server_name => generated 15 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 20:59:48.055941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:48.058119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:48.064722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.065249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.065697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.066413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.068030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.069154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.161928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.172314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.173831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.178966 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b7ba4f7-e047-4e42-8b78-ec104fb4439c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1447 microversion: 2.75 time: 0.126692 Oct 15 20:59:48.179543 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1614/2728] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:48 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1447 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:48.184846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-990365a9-8d7e-4890-99b8-bbf784fc485f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74f8b1a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:48.254193 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-990365a9-8d7e-4890-99b8-bbf784fc485f tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1432 microversion: 2.1 time: 0.073213 Oct 15 20:59:48.254791 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1615/2729] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:48 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1432 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:48.259254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:48.265640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.266415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:48.272169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.272687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.273149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.273878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.274337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.274764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.305107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:48.346018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.352640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.353159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.353602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.483852 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3d6ba5a-b9bd-4c20-b887-55549a072c47 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 3023 microversion: 2.1 time: 0.227928 Oct 15 20:59:48.499500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1616/2730] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 20:59:48 2022] GET /compute/v2.1/servers/detail => generated 3023 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:48.509003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:48.509627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:48.521343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.521831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.522264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.522970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.523427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.523849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.721192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.731504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d8bfc9d-4766-4b13-9848-8b8514c58ab4 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1495 microversion: 2.1 time: 0.229419 Oct 15 20:59:48.748751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1617/2731] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:48 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1495 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:48.753943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72fa2283-e285-4fa6-80a6-4a5c7d2416c4 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75536ef70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:48.817304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72fa2283-e285-4fa6-80a6-4a5c7d2416c4 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 200 len: 1432 microversion: 2.1 time: 0.067244 Oct 15 20:59:48.827729 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1618/2732] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:48 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 1432 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:48.834176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:48.834812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:48.846800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.847306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.847754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.850720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:48.855539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:48.856090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:48.981890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.991697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.994223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:48.995619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb71f802-f331-4189-9b75-b02920bfd821 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1359 microversion: 2.47 time: 0.166597 Oct 15 20:59:49.003192 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1619/2733] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:48 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1359 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:49.009629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:49.021053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:49.028039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:49.028546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:49.029000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:49.029763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:49.030300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:49.030726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:49.143667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:49.154412 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5645dffa-918f-4efe-8b52-bb690b897cae tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1266 microversion: 2.1 time: 0.148654 Oct 15 20:59:49.157893 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1620/2734] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:49 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:49.166768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae343ecb-464e-4da1-a314-a55703dd8b09 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:49.183306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae343ecb-464e-4da1-a314-a55703dd8b09 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:49.185601 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae343ecb-464e-4da1-a314-a55703dd8b09 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers?status=non_existing_status" status: 200 len: 15 microversion: 2.1 time: 0.026694 Oct 15 20:59:49.186166 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1621/2735] 10.222.0.16 () {58 vars in 1178 bytes} [Sat Oct 15 20:59:49 2022] GET /compute/v2.1/servers?status=non_existing_status => generated 15 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 20:59:49.192215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:49.192829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:49.213152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:49.214091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:49.215650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:49.216902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:49.217746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:49.221435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:50.339629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:50.360361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be76dbfe-cf9d-410c-b610-3760528ff7e3 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1502 microversion: 2.1 time: 1.171845 Oct 15 20:59:50.361416 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1622/2736] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:49 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1502 bytes in 1173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:50.366508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:50.375257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:50.392808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:50.393866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:50.394718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:50.395730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:50.397437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:50.399990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:50.962914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:50.967853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:50.974092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecc56dd2-343e-4521-b676-e447383f371a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1467 microversion: 2.1 time: 0.611397 Oct 15 20:59:50.975226 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1623/2737] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:50 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1467 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:50.980363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-1-1776648356"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:50.983279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:50.993054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:50.994263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:50.995446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:50.996843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:50.997562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:50.998910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.462426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ea2c392-50d0-482a-bc80-32605b35d984 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "POST /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d/action" status: 202 len: 52 microversion: 2.47 time: 0.483796 Oct 15 20:59:51.475212 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1624/2738] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:59:50 2022] POST /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d/action => generated 52 bytes in 498 msecs (HTTP/1.1 202) 9 headers in 364 bytes (1 switches on core 0) Oct 15 20:59:51.484529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:51.490205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:51.491130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Searching by: {'deleted': False, 'project_id': '8d37935df706485388d6137dff1faba5'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 20:59:51.498394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.499611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.500349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.501654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.502518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.503205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.539536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 20:59:51.584876 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-357e8908-845a-4197-9857-06dbffd0db0a tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 681 microversion: 2.1 time: 0.106735 Oct 15 20:59:51.595071 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1625/2739] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 20:59:51 2022] GET /compute/v2.1/servers => generated 681 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:51.595071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:51.595071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:51.600103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.600914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.601697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.602676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.603389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.604077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.708369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:51.723445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:51.724928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:51.739685 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e42aedd4-ec40-4aed-af16-fadd80bce8fe tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1453 microversion: 2.75 time: 0.152274 Oct 15 20:59:51.751553 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1626/2740] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:51 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1453 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:51.751553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:51.751553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:51.765695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.766225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.766659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.767386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.767855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.768287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.930385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:51.939263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d341abf0-b803-4161-8729-6e5517861439 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1453 microversion: 2.1 time: 0.194265 Oct 15 20:59:51.945680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1627/2741] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:51 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1453 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:51.951246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:51.952071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:51.958633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.959333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.959936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:51.960806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:51.961480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:51.962095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.371289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:52.384253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec169448-edb5-48dc-8df1-82c59572fe94 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.437167 Oct 15 20:59:52.385413 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1628/2742] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:51 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:52.391233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:52.391777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:52.399505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.400144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.400596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.401497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.401961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.402905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.459617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lazy-loading 'flavor' on Instance uuid 6be11d80-e61b-4556-9907-c828a8d94fed {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:59:52.579614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a78ad01-f7c1-43ac-96dc-1e4549dcc83f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "POST /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed/action" status: 202 len: 0 microversion: 2.1 time: 0.192447 Oct 15 20:59:52.585062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1629/2743] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:59:52 2022] POST /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed/action => generated 0 bytes in 199 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 20:59:52.592591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66eea438-7a76-4d1e-908e-f5e65a7813df tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75439b310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:52.669537 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66eea438-7a76-4d1e-908e-f5e65a7813df tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/77903f56-0555-4c00-82ae-bbf42e11a2a9" status: 200 len: 784 microversion: 2.1 time: 0.080775 Oct 15 20:59:52.679377 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1630/2744] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:52 2022] GET /compute/v2.1/images/77903f56-0555-4c00-82ae-bbf42e11a2a9 => generated 784 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:52.688458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:52.689731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:52.707403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.708139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.708744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.710204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.711009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.711601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.760381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:52.924000 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e05c073-6c83-41a6-824e-f25d2fecf85c tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 204 len: 0 microversion: 2.47 time: 0.243056 Oct 15 20:59:52.924960 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1631/2745] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 20:59:52 2022] DELETE /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 20:59:52.930487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-30a97906-edaa-4295-8361-8ffc9c69f11a req-59b0e118-4c98-4ba1-97c7-337e47d99045 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1910f53a-ef2a-4f63-836d-9850556d1762", "name": "network-vif-deleted", "tag": "5f4c041d-5d6e-4a7f-a157-e88a7ad86f9e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:52.940337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30a97906-edaa-4295-8361-8ffc9c69f11a req-59b0e118-4c98-4ba1-97c7-337e47d99045 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.940921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30a97906-edaa-4295-8361-8ffc9c69f11a req-59b0e118-4c98-4ba1-97c7-337e47d99045 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.941382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30a97906-edaa-4295-8361-8ffc9c69f11a req-59b0e118-4c98-4ba1-97c7-337e47d99045 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.954654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-30a97906-edaa-4295-8361-8ffc9c69f11a req-59b0e118-4c98-4ba1-97c7-337e47d99045 service nova] Creating event network-vif-deleted:5f4c041d-5d6e-4a7f-a157-e88a7ad86f9e for instance 1910f53a-ef2a-4f63-836d-9850556d1762 on nch2-master-1 Oct 15 20:59:52.961083 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-30a97906-edaa-4295-8361-8ffc9c69f11a req-59b0e118-4c98-4ba1-97c7-337e47d99045 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033620 Oct 15 20:59:52.961997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1632/2746] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:52.968494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:52.969111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:52.977094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.979436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.979436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:52.979436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:52.979436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:52.979797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:53.027436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:53.174425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:53.195091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-578f2744-4464-4db2-8924-bf036bb6c695 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b" status: 204 len: 0 microversion: 2.1 time: 0.231595 Oct 15 20:59:53.201362 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1633/2747] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:52 2022] DELETE /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:53.214594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:53.219481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:53.226556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:53.227664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:53.228116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:53.228840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:53.229330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:53.229759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:53.654126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:53.663206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d34aaa4-a0ea-4db6-a588-9b7acad0befe tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1510 microversion: 2.1 time: 0.461947 Oct 15 20:59:53.664013 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1634/2748] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:53 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1510 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:53.677406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26036e39-c39a-4dff-b0e0-f0556bfe6879 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:53.692465 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26036e39-c39a-4dff-b0e0-f0556bfe6879 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.024497 Oct 15 20:59:53.693577 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1635/2749] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 20:59:53 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:53.710321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:53.712141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:53.729027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:53.729515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:53.729964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:53.731275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:53.735514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:53.736206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:53.841514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=5,nova_cell1:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:53.954638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:53.969076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:53.973279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:53.986574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f9bd109-914b-414f-8c1b-f3834600f34b tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1453 microversion: 2.75 time: 0.291548 Oct 15 20:59:54.002150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1636/2750] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:53 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1453 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:54.016164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:54.016885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:54.024289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.025352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.025869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.026777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.027315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.028485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.119560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:54.129001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:54.130485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:54.131680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8326c84b-f693-4f9a-9eeb-bd6e072142a8 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1365 microversion: 2.47 time: 0.128337 Oct 15 20:59:54.141148 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1637/2751] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:54 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1365 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:54.141148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:54.141148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:54.154957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.154957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.154957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.154957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.154957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.154957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.313535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:54.329418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7679b683-2f47-4849-ab60-05944497dce2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1293 microversion: 2.1 time: 0.195615 Oct 15 20:59:54.330667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1638/2752] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:54 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1293 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:54.336255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:54.336780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:54.345578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.345952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.346576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.347289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.348081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.348520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.372361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] HTTP exception thrown: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d HTTP/1.1 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '183834bb-a5a3-43c7-a9c4-20c50652331d'} Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:59:54.378520 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 404 to user: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc454415-bb65-4007-ab12-9529cd2c721e tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "DELETE /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d" status: 404 len: 111 microversion: 2.1 time: 0.045049 Oct 15 20:59:54.382308 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1639/2753] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:54 2022] DELETE /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:54.384410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-834ca6c2-4914-4a52-a03e-589c8dfd7940 req-2cc8bef6-c45d-40cf-8520-47d5454e2896 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "name": "network-vif-unplugged", "status": "completed", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:54.400621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-834ca6c2-4914-4a52-a03e-589c8dfd7940 req-2cc8bef6-c45d-40cf-8520-47d5454e2896 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.401271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-834ca6c2-4914-4a52-a03e-589c8dfd7940 req-2cc8bef6-c45d-40cf-8520-47d5454e2896 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.401721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-834ca6c2-4914-4a52-a03e-589c8dfd7940 req-2cc8bef6-c45d-40cf-8520-47d5454e2896 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.433946 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-834ca6c2-4914-4a52-a03e-589c8dfd7940 req-2cc8bef6-c45d-40cf-8520-47d5454e2896 service nova] Creating event network-vif-unplugged:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 20:59:54.446181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-834ca6c2-4914-4a52-a03e-589c8dfd7940 req-2cc8bef6-c45d-40cf-8520-47d5454e2896 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.067184 Oct 15 20:59:54.447103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1640/2754] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:54.452606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:54.453353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:54.460121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.460854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.461389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.462111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:54.462699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:54.462988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:54.662230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:54.670548 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-909b20f4-9bcd-486e-87c7-4e4cdf8f2e93 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.222059 Oct 15 20:59:54.671396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1641/2755] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:54 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:54.677472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9069524-09c9-487b-8971-1c0290d08755 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74f865d30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:55.841741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9069524-09c9-487b-8971-1c0290d08755 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "DELETE /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 204 len: 0 microversion: 2.1 time: 1.169485 Oct 15 20:59:55.842500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1642/2756] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:54 2022] DELETE /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 0 bytes in 1171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:55.850040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:55.859062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:55.887416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:55.888108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:55.888991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:55.889919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:55.890590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:55.891191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:55.953604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 20:59:56.062996 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7338b89e-ec4c-4484-9d0b-e7149c750464 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "DELETE /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 204 len: 0 microversion: 2.1 time: 0.216069 Oct 15 20:59:56.064072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1643/2757] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 20:59:55 2022] DELETE /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 20:59:56.079013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:56.080445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:56.087180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.087766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.088277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.089076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.089733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.091118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.342459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.353578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e53f02b2-9592-40dd-82a6-9f64857c2b92 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1510 microversion: 2.1 time: 0.287812 Oct 15 20:59:56.354737 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1644/2758] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:56 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1510 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:56.361704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:56.362985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:56.372459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.373361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.373963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.374870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.375675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.376417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.500270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.514969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.516664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.523598 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b3d8d5f-7e2e-4948-a9b2-76060155b56c tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1453 microversion: 2.75 time: 0.166679 Oct 15 20:59:56.535873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1645/2759] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:56 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1453 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:56.540931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:56.548160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:56.579123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.580094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.580723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.581784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.582429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.583046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.733381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.743609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.748077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.749617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8e4812b-846e-43c3-aa2e-9b2cb1744e80 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1365 microversion: 2.47 time: 0.212312 Oct 15 20:59:56.750303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1646/2760] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:56 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1365 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:56.758333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:56.759510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:56.769956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.770818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.771635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.773415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.774374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.774858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.811036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.811625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.812284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.812812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lazy-loading 'fault' on Instance uuid 1910f53a-ef2a-4f63-836d-9850556d1762 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:59:56.868743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:56.879921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63e989bf-be61-4dd6-91ea-10d9e7c0cbcd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 200 len: 1298 microversion: 2.1 time: 0.128016 Oct 15 20:59:56.881923 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1647/2761] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:56 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 1298 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:56.887316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:56.888470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:56.913360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.914062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.914600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:56.915432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:56.916143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:56.916672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:57.557318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:57.566624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-777c1548-ce21-41ca-a8f3-5be88925c48c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.683400 Oct 15 20:59:57.580987 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1648/2762] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:56 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 685 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:57.580987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ab64374-e8d4-403d-933a-c4de20356af8 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74f865e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:57.597557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3ab64374-e8d4-403d-933a-c4de20356af8 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] HTTP exception thrown: Image not found. Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ab64374-e8d4-403d-933a-c4de20356af8 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Request method failure captured: Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 HTTP/1.1 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8PX0-49snCz46mp-NhhQmPmVLgOYnXRDBm5JZolI1wgX3QUCaT9-lCECUUcQru_YYXFhNc9VKPOfClBYhxcvHWxiBB0zyTeNnuzf3YvcHosWw4P2pG2U_a8aeLFvXKayL7urDAFU5iSPj4NlGYI6PUdshLtdi42vyh8Je7UAoRxGE Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 57ac50a5098942e58c8ab72ff09d5159 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 57ac50a5098942e58c8ab72ff09d5159 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerTestJSON-1585030782-project Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 83d85ef4ebdb452a93f24682e2bbd46c Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerTestJSON-1585030782-project Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f3ca929c-bd96-4eca-a596-b2ee1ac252e8'} Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID f3ca929c-bd96-4eca-a596-b2ee1ac252e8 Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 20:59:57.600449 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image f3ca929c-bd96-4eca-a596-b2ee1ac252e8 could not be found. Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ab64374-e8d4-403d-933a-c4de20356af8 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ab64374-e8d4-403d-933a-c4de20356af8 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 404 len: 62 microversion: 2.1 time: 0.033881 Oct 15 20:59:57.604439 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1649/2763] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 20:59:57 2022] GET /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 62 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 20:59:57.617367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:57.617367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:57.621298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:57.621887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:57.622434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:57.623271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:57.623804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:57.624420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:57.886449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:57.906547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e7e48abb-1830-4171-b870-abac42e602cb tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b" status: 200 len: 1521 microversion: 2.1 time: 0.299309 Oct 15 20:59:57.925972 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1650/2764] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:57 2022] GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b => generated 1521 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:57.940887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:57.943251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:57.960891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:57.966141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:57.966944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:57.969236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:57.969988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:57.972429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.222009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.231665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-721bedb0-e197-4be3-9435-0a88be2c9597 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1510 microversion: 2.1 time: 0.304461 Oct 15 20:59:58.233326 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1651/2765] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:57 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1510 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 20:59:58.245177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:58.250091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:58.261138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.269143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.269938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.270876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.271660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.272995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.343437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=14 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:58.362954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.375600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.377649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.384070 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ace5e8f5-3930-407d-97a4-c08753d449b3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1438 microversion: 2.75 time: 0.149471 Oct 15 20:59:58.400252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1652/2766] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:58 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1438 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:58.410441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-\u20a1-313283578"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:58.413302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:58.421010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.421779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.422925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.423934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.424549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.425171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.719251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 20:59:58.788994 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-616a5638-0d75-4532-89cd-9926f6a7e51a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "POST /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031/action" status: 202 len: 0 microversion: 2.1 time: 0.387273 Oct 15 20:59:58.792427 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1653/2767] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 20:59:58 2022] POST /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031/action => generated 0 bytes in 392 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 20:59:58.811572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:58.812516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:58.820932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.821741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.822639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.823546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.824230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.825376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.917281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.930897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.932701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 20:59:58.934828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72519606-2e55-4594-9f14-e13d4fb37a8a tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1365 microversion: 2.47 time: 0.140586 Oct 15 20:59:58.938693 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1654/2768] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 20:59:58 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1365 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 20:59:58.957990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cf017529-601d-4a38-9797-7af9fe7b7870 req-443ea579-ae33-47f5-922d-1b5ab7ee03f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b", "name": "network-vif-deleted", "tag": "dc6638f8-3a22-4290-9aae-3189cabf87b0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:58.971580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cf017529-601d-4a38-9797-7af9fe7b7870 req-443ea579-ae33-47f5-922d-1b5ab7ee03f9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:58.973357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cf017529-601d-4a38-9797-7af9fe7b7870 req-443ea579-ae33-47f5-922d-1b5ab7ee03f9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:58.973966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cf017529-601d-4a38-9797-7af9fe7b7870 req-443ea579-ae33-47f5-922d-1b5ab7ee03f9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:58.989663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cf017529-601d-4a38-9797-7af9fe7b7870 req-443ea579-ae33-47f5-922d-1b5ab7ee03f9 service nova] Creating event network-vif-deleted:dc6638f8-3a22-4290-9aae-3189cabf87b0 for instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b on nch2-master-1 Oct 15 20:59:58.997949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cf017529-601d-4a38-9797-7af9fe7b7870 req-443ea579-ae33-47f5-922d-1b5ab7ee03f9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056939 Oct 15 20:59:58.999272 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1655/2769] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 20:59:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 20:59:59.005857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:59.006768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:59.018828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.022458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.024886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.031878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.032803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.034388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.056123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 HTTP/1.1 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1910f53a-ef2a-4f63-836d-9850556d1762'} Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 20:59:59.059357 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0ce2c755-bb0d-4791-b343-fd68da8fbb61 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 404 len: 111 microversion: 2.1 time: 0.060309 Oct 15 20:59:59.064075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1656/2770] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:58 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 20:59:59.317877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 20:59:59.320637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:59.329487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.330125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.330794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.332394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.332981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.334070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.466240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 20:59:59.466668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 20:59:59.467278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 20:59:59.467703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 20:59:59.468217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 20:59:59.535591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:59.541841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.542651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.542923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.544312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.544828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.545411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.669359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 20:59:59.701505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb835f6c-9dac-4145-9891-8e670eb6fd00 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe/action" status: 202 len: 1942 microversion: 2.75 time: 0.634892 Oct 15 20:59:59.705246 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1657/2771] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 20:59:59 2022] POST /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe/action => generated 1942 bytes in 643 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Oct 15 20:59:59.710192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 20:59:59.711011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 20:59:59.718511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.719248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.719988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 20:59:59.720894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 20:59:59.721530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 20:59:59.722290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:00.415503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:00.429085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba153415-026c-49a0-bde6-73e3d4c26005 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.722518 Oct 15 21:00:00.439806 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1658/2772] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 20:59:59 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:00.460054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e548d23-b99c-4336-bf0e-cf295285129e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74fc17d30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:00.522883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e548d23-b99c-4336-bf0e-cf295285129e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.077811 Oct 15 21:00:00.523735 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1659/2773] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:00 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:00.536873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:00.537573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:00.550413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:00.550965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:00.551427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:00.552192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:00.552668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:00.553156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:00.585194 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] HTTP exception thrown: Instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b could not be found. Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b HTTP/1.1 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b could not be found. Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b'} Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b could not be found. Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b could not be found. Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:00.586765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b could not be found. Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 404 to user: Instance cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-013b5e78-da4f-49d8-b109-d220a38af4ff tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b" status: 404 len: 111 microversion: 2.1 time: 0.063358 Oct 15 21:00:00.591885 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1660/2774] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:00 2022] GET /compute/v2.1/servers/cf80b22d-3225-4ef0-9b18-5ec3e6eb5d4b => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:00.594808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:00.595260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:00.605224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:00.606531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:00.606975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:00.607852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:00.608147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:00.609177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:00.648276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:00:00.852544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3cc96e1d-bb01-4d1d-8c62-f833c0040f85 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 204 len: 0 microversion: 2.1 time: 0.262343 Oct 15 21:00:00.853342 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1661/2775] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:00 2022] DELETE /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:01.026861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:01.027672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:01.034203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.034713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.035156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.035860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.036314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.036934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.238565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:01.247182 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5239bc3-d6db-4226-88c1-e9717f9fd003 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1493 microversion: 2.1 time: 0.388700 Oct 15 21:00:01.247986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1662/2776] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:00 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1493 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:01.257585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:01.259544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:01.268706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.269227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.269666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.270381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.270983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.271415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.514449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=18 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:01.519908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:01.587486 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9f3e466-fd0e-4f15-88a0-14c7af057290 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.338252 Oct 15 21:00:01.591949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1663/2777] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:01 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:01.597362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:01.604828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:01.635241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.639723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.640493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.641431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.642179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.643131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.914481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:01.943868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:01.948345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:01.953458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5af0a6d6-6f40-44c9-849f-8bfef6f60082 tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1365 microversion: 2.47 time: 0.360144 Oct 15 21:00:01.954344 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1664/2778] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:01 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1365 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:01.964214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:01.967969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 183834bb-a5a3-43c7-a9c4-20c50652331d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:01.985631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.988783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.989743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:01.991469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:01.992139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:01.992932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.013278 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] HTTP exception thrown: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d HTTP/1.1 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '183834bb-a5a3-43c7-a9c4-20c50652331d'} Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:02.019148 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. Oct 15 21:00:02.023488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:02.025903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 404 to user: Instance 183834bb-a5a3-43c7-a9c4-20c50652331d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:02.027013 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b3dc1b1-0ef3-4f50-b786-60a07e225f48 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d" status: 404 len: 111 microversion: 2.1 time: 0.071387 Oct 15 21:00:02.028158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1665/2779] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:01 2022] GET /compute/v2.1/servers/183834bb-a5a3-43c7-a9c4-20c50652331d => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:02.033628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:02.034531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:02.042469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.043374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.044230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.045556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.046285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.047612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.233125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:02.243272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd639f84-f01e-4848-a7b3-c7f3f4f2085c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1496 microversion: 2.1 time: 0.213723 Oct 15 21:00:02.244497 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1666/2780] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:02 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1496 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:02.262812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ef23e4e0-7f8c-40a0-907b-feaca03e44aa req-8dd72c82-cc87-4963-afd6-5714624df0c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3fba876c-4a9f-4ad3-b241-45b22a885319", "tag": "9d314f4a-83b9-4ee2-8914-0a7b2c42e245"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:02.279156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef23e4e0-7f8c-40a0-907b-feaca03e44aa req-8dd72c82-cc87-4963-afd6-5714624df0c1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.279156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef23e4e0-7f8c-40a0-907b-feaca03e44aa req-8dd72c82-cc87-4963-afd6-5714624df0c1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.279526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef23e4e0-7f8c-40a0-907b-feaca03e44aa req-8dd72c82-cc87-4963-afd6-5714624df0c1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.303621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ef23e4e0-7f8c-40a0-907b-feaca03e44aa req-8dd72c82-cc87-4963-afd6-5714624df0c1 service nova] Creating event network-changed:9d314f4a-83b9-4ee2-8914-0a7b2c42e245 for instance 3fba876c-4a9f-4ad3-b241-45b22a885319 on nch1-master-1 Oct 15 21:00:02.310559 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ef23e4e0-7f8c-40a0-907b-feaca03e44aa req-8dd72c82-cc87-4963-afd6-5714624df0c1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064642 Oct 15 21:00:02.311391 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1667/2781] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:02.320987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:02.321639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:02.328261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.328892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.329283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.329992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.330446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.330870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.385643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:00:02.452976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:SELECT=94,nova_cell1:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:02.519706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:02.526198 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca603125-ba81-4814-909f-3ef24e5279d8 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 204 len: 0 microversion: 2.1 time: 0.209043 Oct 15 21:00:02.530481 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1668/2782] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:02 2022] DELETE /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 0 bytes in 211 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:02.536648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:02.537453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:02.544041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.544558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.545171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.545902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.546371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.546809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.827473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:02.836770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb87550f-a83b-40d4-a10a-34225f2227cd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.307910 Oct 15 21:00:02.850835 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1669/2783] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:02 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:02.876746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f934c43c-4410-4f1f-a74b-2198fe04af9e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75850fdc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:02.940104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f934c43c-4410-4f1f-a74b-2198fe04af9e tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.087862 Oct 15 21:00:02.950043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1670/2784] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:02 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:02.955679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:02.956347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:02.966613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.967340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.967979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:02.968884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:02.969738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:02.970347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.013086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.013597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.014048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.014518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lazy-loading 'fault' on Instance uuid 55d566f6-a7eb-497e-8f4e-43c60433fc35 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:03.076112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c8c93cfed5140b183cc0e3a559b1ce7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d37935df706485388d6137dff1faba5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:03.099259 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-178c05ef-94f6-4a23-8fb6-9d6971b7bcb5 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 200 len: 1312 microversion: 2.1 time: 0.146757 Oct 15 21:00:03.100526 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1671/2785] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:02 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 1312 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:03.128501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:03.129845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:03.150004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.150575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.151031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.151764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.152621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.153235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.382166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:03.392437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c200f336-7642-48a9-bc74-1f6f5fb10c2f tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1499 microversion: 2.1 time: 0.290158 Oct 15 21:00:03.393424 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1672/2786] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:03 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1499 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:03.398579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:03.399308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:03.407040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.407642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.408171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.408959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.409554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.410070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.528807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:03.551110 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fab09ca4-75bc-41da-92bc-a806c26a7cff tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.156319 Oct 15 21:00:03.562339 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1673/2787] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:03 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:03.569346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:03.570547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:03.579220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.579925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.580368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.581189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.581916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.582523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.704833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:03.716796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:03.721964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6925000aee9d499a85c6dc1282d09993', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b955a7e7fb04b569773c0453624911a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:03.723868 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55905706-0e14-4d28-9052-e2ff7280134e tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 200 len: 1365 microversion: 2.47 time: 0.158448 Oct 15 21:00:03.724983 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1674/2788] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:03 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 1365 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:03.729832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:03.730732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:03.737239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.745304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.746514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.747412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.748820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.749303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.809404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.810117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.810625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.811073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lazy-loading 'fault' on Instance uuid 87c2e279-0350-4f47-b7f3-68c0054c0100 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:03.879937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:03.892117 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab87223a-afa1-485c-afe5-9cae91044831 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 200 len: 1298 microversion: 2.1 time: 0.165829 Oct 15 21:00:03.893121 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1675/2789] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:03 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 1298 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:03.898321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:03.899187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:03.924307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.924981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.925538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:03.926911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:03.927712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:03.928327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:04.318318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:04.338038 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c6e2b0e-a7c6-479e-a0eb-4a3e36f0621c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.443642 Oct 15 21:00:04.339224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1676/2790] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:03 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:04.344418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fec72ec2-5e18-42ca-a8d2-2046b71d6c5b tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74ff37940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:04.472251 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fec72ec2-5e18-42ca-a8d2-2046b71d6c5b tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.131556 Oct 15 21:00:04.472971 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1677/2791] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:04 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:04.491775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:04.497017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] [instance: 55d566f6-a7eb-497e-8f4e-43c60433fc35] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:04.508615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:04.509549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:04.511000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:04.513142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:04.513973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:04.521486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:04.542658 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] HTTP exception thrown: Instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 could not be found. Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Request method failure captured: Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 HTTP/1.1 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8Rzc1uWN6BTD8b8b_aXXOF21kok7Be7ofWg0Y5KmNAS5mD-s0NNcNls1mO4CNMIOwP6AEshyQ9R4kb3PZFCs4e1AwF0JO5vdQlzu3mP-lAtVofzqWtb9fADnQW4AYWntqUPNGcdWXxbDRiulszxr7olIRLj-tp8WIJAYbk8dXGt0U Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8d37935df706485388d6137dff1faba5 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d37935df706485388d6137dff1faba5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d37935df706485388d6137dff1faba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d37935df706485388d6137dff1faba5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8d37935df706485388d6137dff1faba5 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-1262259006 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6c8c93cfed5140b183cc0e3a559b1ce7 Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServersNegativeTestJSON-1262259006-project Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 could not be found. Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '55d566f6-a7eb-497e-8f4e-43c60433fc35'} Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 could not be found. Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 could not be found. Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:04.546462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 could not be found. Oct 15 21:00:04.550314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:04.552734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] Returning 404 to user: Instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:04.553948 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c7d4c8b-8ae6-499b-a391-423810455008 tempest-ListServersNegativeTestJSON-1262259006 tempest-ListServersNegativeTestJSON-1262259006-project] 10.222.0.16 "GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35" status: 404 len: 111 microversion: 2.1 time: 0.070586 Oct 15 21:00:04.555094 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1678/2792] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:04 2022] GET /compute/v2.1/servers/55d566f6-a7eb-497e-8f4e-43c60433fc35 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:04.570638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "55d566f6-a7eb-497e-8f4e-43c60433fc35", "name": "network-vif-deleted", "tag": "a601a90d-756d-4ac7-97b5-50063c98b756"}, {"server_uuid": "87c2e279-0350-4f47-b7f3-68c0054c0100", "name": "network-vif-deleted", "tag": "731071c0-c8ed-432e-a1d7-02c713022ccb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:04.598668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:04.599170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:04.599669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:04.615335 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] Creating event network-vif-deleted:a601a90d-756d-4ac7-97b5-50063c98b756 for instance 55d566f6-a7eb-497e-8f4e-43c60433fc35 on nch1-master-1 Oct 15 21:00:04.616143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] Creating event network-vif-deleted:731071c0-c8ed-432e-a1d7-02c713022ccb for instance 87c2e279-0350-4f47-b7f3-68c0054c0100 on nch2-master-1 Oct 15 21:00:04.627334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e17059d4-7b8d-4f07-8501-574a6bc0b1a5 req-c7551e32-5864-473d-bc85-933904068e79 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.070502 Oct 15 21:00:04.628734 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1679/2793] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:04 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:04.635863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:04.637468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:04.644531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:04.645111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:04.645715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:04.646944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:04.647074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:04.647495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.406142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2bba6c9b3f674959ad79d4a394ed9f2a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ca9af3805eba4f69b474a8b2205f0920', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:05.422680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-afd9f4b0-0d76-4e8d-8388-9b11c91c7c07 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 200 len: 1499 microversion: 2.1 time: 0.784065 Oct 15 21:00:05.427310 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1680/2794] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:04 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 1499 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:05.432172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:05.432807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:05.448718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.449825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.450275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.451000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.451464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.452165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.618100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:05.638654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e347d2ca-becd-4183-b05a-009aea760de4 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.210148 Oct 15 21:00:05.640176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1681/2795] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:05 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:05.645269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:05.646112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] [instance: b87c05b0-44fa-48e4-8743-0e8994af095d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:05.653576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.654334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.654947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.655854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.656480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.657112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.674269 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] HTTP exception thrown: Instance b87c05b0-44fa-48e4-8743-0e8994af095d could not be found. Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Request method failure captured: Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d HTTP/1.1 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx83fJhqJ2qArC_TGlxZacAnwI0zY4CzrYkEG-0VW9TALP_VZT3ToR2HQWaKBt0wNZXOv6zQAt0CM1SMEZ83_eBhyxr5XzBNFN4EGSI14um2sshL06pq1CfbHryu0a9jopy5ziO2KGM70dlVgc5HGv70fg3iot__UMJQC0Zz0pi1j0U Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.47 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6b955a7e7fb04b569773c0453624911a Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAaction247Test-1106887624 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b955a7e7fb04b569773c0453624911a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b955a7e7fb04b569773c0453624911a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b955a7e7fb04b569773c0453624911a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b955a7e7fb04b569773c0453624911a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAaction247Test-1106887624 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6b955a7e7fb04b569773c0453624911a Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAaction247Test-1106887624 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAaction247Test-1106887624-project Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6925000aee9d499a85c6dc1282d09993 Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAaction247Test-1106887624-project Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b87c05b0-44fa-48e4-8743-0e8994af095d could not be found. Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b87c05b0-44fa-48e4-8743-0e8994af095d'} Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b87c05b0-44fa-48e4-8743-0e8994af095d could not be found. Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b87c05b0-44fa-48e4-8743-0e8994af095d could not be found. Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.676230 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b87c05b0-44fa-48e4-8743-0e8994af095d could not be found. Oct 15 21:00:05.680875 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.682924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] Returning 404 to user: Instance b87c05b0-44fa-48e4-8743-0e8994af095d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:05.684946 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d5c2cdeb-53f2-4a1a-a02a-a27b92e0918f tempest-ServersAaction247Test-1106887624 tempest-ServersAaction247Test-1106887624-project] 10.222.0.16 "GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d" status: 404 len: 111 microversion: 2.47 time: 0.043260 Oct 15 21:00:05.686426 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1682/2796] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:05 2022] GET /compute/v2.1/servers/b87c05b0-44fa-48e4-8743-0e8994af095d => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:00:05.693736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:05.694603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:05.701822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.702882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.703542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.704517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.705244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.705905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.723387 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 HTTP/1.1 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '87c2e279-0350-4f47-b7f3-68c0054c0100'} Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:05.725355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:00:05.729137 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:05.731062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:05.732186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9bac712b-2e7b-423d-8b9b-0ac2100ccf64 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 404 len: 111 microversion: 2.1 time: 0.044533 Oct 15 21:00:05.732993 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1683/2797] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:05 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:05.739460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9e8df35c-22f5-4350-950f-26c98298573e req-b56657d0-8e2d-4022-b849-f953e6f829f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "name": "network-vif-plugged", "status": "completed", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:05.755626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9e8df35c-22f5-4350-950f-26c98298573e req-b56657d0-8e2d-4022-b849-f953e6f829f9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.756392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9e8df35c-22f5-4350-950f-26c98298573e req-b56657d0-8e2d-4022-b849-f953e6f829f9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.760787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9e8df35c-22f5-4350-950f-26c98298573e req-b56657d0-8e2d-4022-b849-f953e6f829f9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.798036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9e8df35c-22f5-4350-950f-26c98298573e req-b56657d0-8e2d-4022-b849-f953e6f829f9 service nova] Creating event network-vif-plugged:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 21:00:05.808936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9e8df35c-22f5-4350-950f-26c98298573e req-b56657d0-8e2d-4022-b849-f953e6f829f9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073412 Oct 15 21:00:05.811165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1684/2798] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:05.820457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:05.826180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:05.842053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.846828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.847515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:05.848669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:05.849565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:05.850380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:06.478655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:06.492857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a464f905-3f87-45a0-a814-205990405aad tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.676399 Oct 15 21:00:06.493619 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1685/2799] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:05 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:06.513511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73063ee6-fd2b-4284-8ae1-7edef4a55a87 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75857f1f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:06.601481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73063ee6-fd2b-4284-8ae1-7edef4a55a87 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.101747 Oct 15 21:00:06.609249 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1686/2800] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:06 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:06.616064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1495466617", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:06.680223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:00:07.985809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:00:07.987050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:00:07.988027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:00:07.988685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:00:07.989558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:00:08.006516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:08.010543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:08.014576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:08.015553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:08.017165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:08.018838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:08.092091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:08.119782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:08.161338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:00:08.166218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:00:08.313368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e7b1ce4b-76e1-4ac5-b252-d276c3fa7c31 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.709527 Oct 15 21:00:08.320278 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1687/2801] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:00:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 1711 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:00:08.380313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:08.384614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:08.406780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:08.409422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:08.411225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:08.413346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:08.414548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:08.415381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:08.459684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed HTTP/1.1 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6be11d80-e61b-4556-9907-c828a8d94fed'} Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:08.463937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:08.469768 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:08.472141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:08.475304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-74f19ebb-1b36-4777-80ff-b63d663555fb tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 404 len: 111 microversion: 2.1 time: 0.157869 Oct 15 21:00:08.482752 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1688/2802] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:08 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 111 bytes in 167 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:08.495735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:08.500455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:08.519460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:08.524732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:08.529103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:08.531557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:08.532054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:08.532485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:09.911341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:09.962250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55644da9-2490-4c3b-b732-b09fd139b1ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 1.477713 Oct 15 21:00:09.965111 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1689/2803] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:08 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 1482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:10.003914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f52bde91-5f80-4eb3-95f7-e7b3127afbcb req-c946ffc3-6163-4e6f-9a5e-83e3213b0f79 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6be11d80-e61b-4556-9907-c828a8d94fed", "name": "network-vif-deleted", "tag": "8f3bed3a-535d-49a7-a40e-0f0e438f875d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:10.040893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f52bde91-5f80-4eb3-95f7-e7b3127afbcb req-c946ffc3-6163-4e6f-9a5e-83e3213b0f79 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:10.045632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f52bde91-5f80-4eb3-95f7-e7b3127afbcb req-c946ffc3-6163-4e6f-9a5e-83e3213b0f79 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:10.049310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f52bde91-5f80-4eb3-95f7-e7b3127afbcb req-c946ffc3-6163-4e6f-9a5e-83e3213b0f79 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:10.139620 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f52bde91-5f80-4eb3-95f7-e7b3127afbcb req-c946ffc3-6163-4e6f-9a5e-83e3213b0f79 service nova] Creating event network-vif-deleted:8f3bed3a-535d-49a7-a40e-0f0e438f875d for instance 6be11d80-e61b-4556-9907-c828a8d94fed on nch1-master-1 Oct 15 21:00:10.172642 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f52bde91-5f80-4eb3-95f7-e7b3127afbcb req-c946ffc3-6163-4e6f-9a5e-83e3213b0f79 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.182524 Oct 15 21:00:10.179738 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1690/2804] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:10.812459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:10.813280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:10.830539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:10.833921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:10.834841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:10.836352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:10.838989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:10.841246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:10.847657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:11.533840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:11.543690 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d2abd88-1d8a-4133-9530-e93d726f5d27 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 1.362417 Oct 15 21:00:11.566306 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1691/2805] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:10 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 1386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:11.577193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-074d98cb-2a66-478e-b554-a25b86de8c92 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75416cb80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:11.671995 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-074d98cb-2a66-478e-b554-a25b86de8c92 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.104290 Oct 15 21:00:11.677613 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1692/2806] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:11 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:11.690021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-34aa7897-b481-4200-80bf-595e700822a4 req-7f69f9ac-a6bc-490a-a865-388580dc09d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3fba876c-4a9f-4ad3-b241-45b22a885319", "name": "network-vif-plugged", "status": "completed", "tag": "9d314f4a-83b9-4ee2-8914-0a7b2c42e245"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:11.729805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-34aa7897-b481-4200-80bf-595e700822a4 req-7f69f9ac-a6bc-490a-a865-388580dc09d5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:11.732350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-34aa7897-b481-4200-80bf-595e700822a4 req-7f69f9ac-a6bc-490a-a865-388580dc09d5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:11.734323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-34aa7897-b481-4200-80bf-595e700822a4 req-7f69f9ac-a6bc-490a-a865-388580dc09d5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:11.773573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-34aa7897-b481-4200-80bf-595e700822a4 req-7f69f9ac-a6bc-490a-a865-388580dc09d5 service nova] Creating event network-vif-plugged:9d314f4a-83b9-4ee2-8914-0a7b2c42e245 for instance 3fba876c-4a9f-4ad3-b241-45b22a885319 on nch1-master-1 Oct 15 21:00:11.783531 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-34aa7897-b481-4200-80bf-595e700822a4 req-7f69f9ac-a6bc-490a-a865-388580dc09d5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103617 Oct 15 21:00:11.787852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1693/2807] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:11.803001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:11.803729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:11.812858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:11.813392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:11.813836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:11.814622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:11.815026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:11.815460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.102846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:12.119377 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51ffe25e-5780-4814-b28a-9c50bfdbc13e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1267 microversion: 2.1 time: 0.328611 Oct 15 21:00:12.121952 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1694/2808] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:11 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1267 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:12.145977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:12.147013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:12.154218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.155059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.155813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.157729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.159082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.162049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.189828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed HTTP/1.1 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6be11d80-e61b-4556-9907-c828a8d94fed'} Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:12.192880 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.200451 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.203003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:12.204067 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-388bdb5e-062c-42f3-adcf-e6ec1eb49302 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 404 len: 111 microversion: 2.1 time: 0.078167 Oct 15 21:00:12.210610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1695/2809] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:12 2022] DELETE /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:12.218723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:12.219938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:12.241815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.242735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.244187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.245320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.246057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.246803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.439827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:12.495582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0ff6ec8-4041-46aa-849f-3cc495f07459 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.283602 Oct 15 21:00:12.515211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1696/2810] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:12 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:12.541722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:12.543473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6be11d80-e61b-4556-9907-c828a8d94fed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:12.560373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.561185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.563957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.564916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.565577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.566936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.593772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed HTTP/1.1 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6be11d80-e61b-4556-9907-c828a8d94fed'} Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:12.595339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 6be11d80-e61b-4556-9907-c828a8d94fed could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:12.599957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66424b94-d190-4d64-8fa9-9ebd4e6b2739 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed" status: 404 len: 111 microversion: 2.1 time: 0.080357 Oct 15 21:00:12.611725 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1697/2811] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:12 2022] GET /compute/v2.1/servers/6be11d80-e61b-4556-9907-c828a8d94fed => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:12.625939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:12.628503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:12.637286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.647476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.648298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:12.649296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:12.649787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:12.650235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.016676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:13.036803 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52ac1572-9507-4103-a2bf-234b350e5078 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1308 microversion: 2.1 time: 0.421763 Oct 15 21:00:13.040585 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1698/2812] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:12 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1308 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:13.078746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:13.079265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:13.099654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:13.104836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:13.106243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.109226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:13.109924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:13.111619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.148541 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a HTTP/1.1 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '15bdf694-ed96-4052-b284-784636eefd0a'} Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:13.155079 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.159618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.161863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:13.162933 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-135973af-eaa0-4555-a7da-e1ce315dfe3a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 404 len: 111 microversion: 2.1 time: 0.094799 Oct 15 21:00:13.170124 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1699/2813] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:13 2022] DELETE /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 111 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:13.176328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95d0673c-9603-46ca-bd27-510a151292d0 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe758525dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:13.276359 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95d0673c-9603-46ca-bd27-510a151292d0 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.104091 Oct 15 21:00:13.284893 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1700/2814] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:13 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:13.290148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:13.298729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:13.338683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:13.342294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:13.342829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.343707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:13.344281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:13.346664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.866782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:13.880126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a37bcd2-767f-42e9-9494-5f7515856a67 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.591449 Oct 15 21:00:13.881811 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1701/2815] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:13 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:13.888162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:13.889107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 15bdf694-ed96-4052-b284-784636eefd0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:13.910297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:13.911147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:13.914712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.915728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:13.916563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:13.919090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:13.957273 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a HTTP/1.1 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '15bdf694-ed96-4052-b284-784636eefd0a'} Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:13.960916 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. Oct 15 21:00:13.966015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:13.969893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 15bdf694-ed96-4052-b284-784636eefd0a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:13.971446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a5710e5-1a63-48fd-82c3-490c7ad936d6 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a" status: 404 len: 111 microversion: 2.1 time: 0.088344 Oct 15 21:00:13.975393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1702/2816] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:13 2022] GET /compute/v2.1/servers/15bdf694-ed96-4052-b284-784636eefd0a => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:13.989370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:13.992634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:14.031297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:14.033807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:14.036089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:14.037862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:14.041942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:14.045691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:14.337911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:14.374582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bbc93cd2-9f6d-413a-a8b0-2c13abc7521f tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.397562 Oct 15 21:00:14.375856 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1703/2817] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:13 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:14.395526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:14.396345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:14.409356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:14.411196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:14.416820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:14.419318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:14.419983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:14.420796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:14.444526 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 HTTP/1.1 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6e486821-32b0-426e-aa16-aefb86e003a0'} Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:14.450837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:14.455288 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:14.458878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:14.461143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffcdf80c-dbaf-44b2-aee1-7ad03793a399 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 404 len: 111 microversion: 2.1 time: 0.083361 Oct 15 21:00:14.467695 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1704/2818] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:14 2022] DELETE /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:14.473685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:14.481691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:14.501020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:14.501835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:14.504047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:14.504986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:14.505745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:14.509068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:14.901555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:14.913351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73c91663-cc76-4c0b-916d-156c6f238758 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1488 microversion: 2.1 time: 0.443350 Oct 15 21:00:14.922640 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1705/2819] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:14 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1488 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:14.927760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44901876-e742-4424-9ac4-141358185091 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe758578ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:15.008344 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44901876-e742-4424-9ac4-141358185091 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.084401 Oct 15 21:00:15.009672 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1706/2820] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:14 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:15.020000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:15.022886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 6e486821-32b0-426e-aa16-aefb86e003a0] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:15.063915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:15.065679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:15.076509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:15.077976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:15.078948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:15.079570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:15.127454 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 HTTP/1.1 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6e486821-32b0-426e-aa16-aefb86e003a0'} Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:15.132610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. Oct 15 21:00:15.137217 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:15.140994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 6e486821-32b0-426e-aa16-aefb86e003a0 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:15.143155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be602916-6e4c-4076-83cb-451465c047d9 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0" status: 404 len: 111 microversion: 2.1 time: 0.131325 Oct 15 21:00:15.148954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1707/2821] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:15 2022] GET /compute/v2.1/servers/6e486821-32b0-426e-aa16-aefb86e003a0 => generated 111 bytes in 138 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:15.161005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:15.164334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:15.182040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:15.183985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:15.184505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:15.186553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:15.187466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:15.189351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:15.740448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:15.770886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c594f43-20e4-467f-85aa-2bda929ddc44 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.619655 Oct 15 21:00:15.771774 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1708/2822] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:15 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:15.782105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:15.782748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:15.795843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:15.796435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:15.796971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:15.798382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:15.798925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:15.799480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:15.861386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:00:16.063984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:16.127142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07fc8f2b-9ac9-4944-aed9-e783904ebd91 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 204 len: 0 microversion: 2.1 time: 0.353415 Oct 15 21:00:16.127876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1709/2823] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:15 2022] DELETE /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:16.137561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:16.138498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:16.156910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.159365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.160017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:16.160877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.164988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.165601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:16.198365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 HTTP/1.1 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '170b74a2-0481-4eab-acfd-00157c8cb2c5'} Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:16.206554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e399c64-bd01-46f2-8248-5044e26b5205 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 404 len: 111 microversion: 2.1 time: 0.080270 Oct 15 21:00:16.211589 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1710/2824] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:16 2022] DELETE /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:16.217148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:16.226805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:16.236529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.249570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.250758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:16.251904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.252277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.252727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:16.593699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:16.669885 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b81e050-5c88-4c1b-9e13-9803ad6a2735 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.457443 Oct 15 21:00:16.679969 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1711/2825] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:16 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:16.697700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-423d78cd-a8da-4115-b3c2-768cc2a36a99 req-8b18ad6b-456c-40ac-ae09-6ec79d3db57b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "60051f47-77db-45a6-9e66-c0b1b1528667", "tag": "dc662ecd-8ebc-4738-a455-d2a57d721b0b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:16.725184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-423d78cd-a8da-4115-b3c2-768cc2a36a99 req-8b18ad6b-456c-40ac-ae09-6ec79d3db57b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.725704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-423d78cd-a8da-4115-b3c2-768cc2a36a99 req-8b18ad6b-456c-40ac-ae09-6ec79d3db57b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.726315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-423d78cd-a8da-4115-b3c2-768cc2a36a99 req-8b18ad6b-456c-40ac-ae09-6ec79d3db57b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:16.754471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-423d78cd-a8da-4115-b3c2-768cc2a36a99 req-8b18ad6b-456c-40ac-ae09-6ec79d3db57b service nova] Creating event network-changed:dc662ecd-8ebc-4738-a455-d2a57d721b0b for instance 60051f47-77db-45a6-9e66-c0b1b1528667 on nch2-master-1 Oct 15 21:00:16.764378 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-423d78cd-a8da-4115-b3c2-768cc2a36a99 req-8b18ad6b-456c-40ac-ae09-6ec79d3db57b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082780 Oct 15 21:00:16.770601 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1712/2826] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:16.779976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5052df0-da54-4fca-aabc-58ae941488d5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe7584fcee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:16.947750 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5052df0-da54-4fca-aabc-58ae941488d5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.175532 Oct 15 21:00:16.949280 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1713/2827] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:16 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:16.967354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:16.970079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:16.987651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.988413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.988861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:16.989674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:16.990151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:16.990588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:17.855358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:17.867731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-815e939d-09dc-4939-87fd-bc6cf0c49b73 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1494 microversion: 2.1 time: 0.915909 Oct 15 21:00:17.868838 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1714/2828] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:16 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1494 bytes in 919 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:17.877028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:17.882339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 170b74a2-0481-4eab-acfd-00157c8cb2c5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:17.901956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:17.902487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:17.902943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:17.903688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:17.907972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:17.908446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:17.923760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:17.976004 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 HTTP/1.1 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '170b74a2-0481-4eab-acfd-00157c8cb2c5'} Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:17.981168 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. Oct 15 21:00:17.985207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:17.990955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 170b74a2-0481-4eab-acfd-00157c8cb2c5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:17.992148 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df061ef2-4906-4dbd-bf74-72f82fe82df1 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5" status: 404 len: 111 microversion: 2.1 time: 0.120369 Oct 15 21:00:17.993948 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1715/2829] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:17 2022] GET /compute/v2.1/servers/170b74a2-0481-4eab-acfd-00157c8cb2c5 => generated 111 bytes in 124 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:18.008680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:18.013950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:18.056628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:18.058151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.030s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:18.059765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:18.060773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:18.061957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:18.062694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:18.063402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:18.150225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:18.710939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:18.736975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3bfd78f-b1a0-4b20-bb9d-614abf4ecdb9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.739593 Oct 15 21:00:18.739447 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1716/2830] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:17 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:18.750730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:18.753150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:18.775146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:18.778636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:18.779874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:18.784603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:18.785434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:18.785781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:18.926106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:18.968766 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b44eede-2e74-4dc1-a496-d70230b6799e tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.226000 Oct 15 21:00:18.970598 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1717/2831] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:18 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:18.978544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54a5d9e1-bf33-450a-94f5-843eb469ca49 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe758525700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:19.058946 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54a5d9e1-bf33-450a-94f5-843eb469ca49 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.086763 Oct 15 21:00:19.061094 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1718/2832] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:18 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:19.067367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:19.068267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:19.077185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.078075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.078741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.079703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.080372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.081083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.103177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 HTTP/1.1 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '04857e01-76b5-45fa-a293-becd94cfd638'} Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:19.109151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.113368 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.115714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:19.123377 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35de19f3-5fbc-43cf-9a3c-98a6d08950b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 404 len: 111 microversion: 2.1 time: 0.055104 Oct 15 21:00:19.124641 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1719/2833] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:19 2022] DELETE /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:19.131007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:19.131931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:19.144668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.145656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.146344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.147288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.147951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.148582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.500721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:19.514457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e110bf93-2364-49bb-a8b5-9639194328fc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 200 len: 1452 microversion: 2.1 time: 0.388263 Oct 15 21:00:19.515069 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1720/2834] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:19 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 1452 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:19.567117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:19.567747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 04857e01-76b5-45fa-a293-becd94cfd638] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:19.616882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.617011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.617502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.618237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.638995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.639467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.672268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 HTTP/1.1 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '04857e01-76b5-45fa-a293-becd94cfd638'} Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:19.674665 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. Oct 15 21:00:19.678928 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:19.680980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 04857e01-76b5-45fa-a293-becd94cfd638 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:19.682187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c1108237-e722-4b55-a10c-d5f7e294a407 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638" status: 404 len: 111 microversion: 2.1 time: 0.133597 Oct 15 21:00:19.687896 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1721/2835] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:19 2022] GET /compute/v2.1/servers/04857e01-76b5-45fa-a293-becd94cfd638 => generated 111 bytes in 172 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:19.696409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-958a1a8d-9de3-4d8b-aea5-6c1972afe1ab req-ffe2d361-1f89-4e04-9095-5a5d59c14e5e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3fba876c-4a9f-4ad3-b241-45b22a885319", "name": "network-vif-deleted", "tag": "9d314f4a-83b9-4ee2-8914-0a7b2c42e245"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:19.708677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-958a1a8d-9de3-4d8b-aea5-6c1972afe1ab req-ffe2d361-1f89-4e04-9095-5a5d59c14e5e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.709567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-958a1a8d-9de3-4d8b-aea5-6c1972afe1ab req-ffe2d361-1f89-4e04-9095-5a5d59c14e5e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.710234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-958a1a8d-9de3-4d8b-aea5-6c1972afe1ab req-ffe2d361-1f89-4e04-9095-5a5d59c14e5e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.734448 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-958a1a8d-9de3-4d8b-aea5-6c1972afe1ab req-ffe2d361-1f89-4e04-9095-5a5d59c14e5e service nova] Creating event network-vif-deleted:9d314f4a-83b9-4ee2-8914-0a7b2c42e245 for instance 3fba876c-4a9f-4ad3-b241-45b22a885319 on nch1-master-1 Oct 15 21:00:19.743587 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-958a1a8d-9de3-4d8b-aea5-6c1972afe1ab req-ffe2d361-1f89-4e04-9095-5a5d59c14e5e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054068 Oct 15 21:00:19.747312 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1722/2836] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:19.780740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:19.781914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:19.806900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.808585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.809816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:19.810860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:19.811785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:19.812663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:20.285121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:20.297804 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6f9a002-4961-4af8-a142-ebee4803fcb7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.532186 Oct 15 21:00:20.301313 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1723/2837] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:19 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:20.317285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:20.326513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:20.339670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:20.345911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:20.349110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:20.350323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:20.352569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:20.353310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:20.576973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:20.627918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d2ff50b-b67a-441d-bd82-3aa80d5c711a tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.323207 Oct 15 21:00:20.628873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1724/2838] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:20 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:20.635494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-467c98c2-46d0-4857-bbbe-0cf0d06dcda5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe758462c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:20.746379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-467c98c2-46d0-4857-bbbe-0cf0d06dcda5 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.110847 Oct 15 21:00:20.751869 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1725/2839] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:20 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:20.766464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:20.767146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:20.785754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:20.786266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:20.786699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:20.787420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:20.788078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:20.788594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:20.830595 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 HTTP/1.1 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3fba876c-4a9f-4ad3-b241-45b22a885319'} Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:20.836765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:20.841347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:20.841347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:20.841347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:20.841347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:20.841347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:00:20.841347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:20.844040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:20.846921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e81dbff3-c0c5-43e8-b172-cb90bf3208fd tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 404 len: 111 microversion: 2.1 time: 0.091573 Oct 15 21:00:20.848843 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1726/2840] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:20 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:20.861862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ba85874-b8f2-4930-89d5-e5dfb1ba28c8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7584fcaf0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:22.470603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ba85874-b8f2-4930-89d5-e5dfb1ba28c8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 204 len: 0 microversion: 2.1 time: 1.616705 Oct 15 21:00:22.471169 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1727/2841] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:20 2022] DELETE /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 0 bytes in 1618 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:22.477402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:22.477937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:22.496370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:22.507643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:22.508368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:22.509326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:22.512955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:22.513947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:22.560085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 HTTP/1.1 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8077ee04-cf77-4ad7-aef9-1d0244cfa551'} Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:22.564331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:22.568625 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:22.573786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:22.575570 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd1adc2f-8550-4069-a3c0-a2d47cac4873 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 404 len: 111 microversion: 2.1 time: 0.103204 Oct 15 21:00:22.576301 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1728/2842] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:22 2022] DELETE /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 111 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:22.586317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:22.589784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:22.599115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:22.599859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:22.600536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:22.607602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:22.608716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:22.611294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:23.463984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:23.481688 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67c2bfe7-ca1f-4e2c-b2de-11fb55e5e296 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.901993 Oct 15 21:00:23.482973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1729/2843] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:22 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:23.488674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:23.511597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:23.525195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:23.526037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:23.526712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:23.528508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:23.531549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:23.532255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:24.256361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:24.319595 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60a03216-8a8b-4d01-878e-7b5c232371d3 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1929 microversion: 2.75 time: 0.834708 Oct 15 21:00:24.345864 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1730/2844] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:23 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1929 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:24.358096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c47d7742-11d3-4cf4-8e0d-b3f70fc1c5dc tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75857fca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:24.512823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c47d7742-11d3-4cf4-8e0d-b3f70fc1c5dc tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.165424 Oct 15 21:00:24.523378 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1731/2845] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:24 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:24.529725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:24.530634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:24.539557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:24.540321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:24.541016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:24.542004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:24.543126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:24.543898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:24.638917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:00:24.931880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25dbb180-55aa-4782-ae9c-cdf13f652221 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 204 len: 0 microversion: 2.1 time: 0.406857 Oct 15 21:00:24.933731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1732/2846] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:24 2022] DELETE /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:24.953898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:24.960554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 8077ee04-cf77-4ad7-aef9-1d0244cfa551] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:24.985934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:24.988674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:24.993572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:24.994682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:24.996806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:24.997561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:25.050294 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 HTTP/1.1 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8077ee04-cf77-4ad7-aef9-1d0244cfa551'} Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:25.064162 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.069180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 8077ee04-cf77-4ad7-aef9-1d0244cfa551 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:25.070929 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-313d12d0-9845-458d-bc00-12f3dd1c130a tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551" status: 404 len: 111 microversion: 2.1 time: 0.126283 Oct 15 21:00:25.076877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1733/2847] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:24 2022] GET /compute/v2.1/servers/8077ee04-cf77-4ad7-aef9-1d0244cfa551 => generated 111 bytes in 142 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:25.097968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:25.099023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:25.115105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:25.117902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:25.120553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:25.121732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:25.122865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:25.126320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:25.697852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:25.710714 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ac73935-700a-4d9b-aac6-b7568c300dfa tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.629124 Oct 15 21:00:25.721093 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1734/2848] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:25 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:25.731295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:25.731295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:25.752069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:25.752973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:25.753739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:25.755811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:25.758252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:25.760009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:25.807892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a HTTP/1.1 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a'} Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:25.822467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:25.826789 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:25.832417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:25.835827 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19968848-d3f4-45a8-9cc6-1009df938eb5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "DELETE /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 404 len: 111 microversion: 2.1 time: 0.117970 Oct 15 21:00:25.842813 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1735/2849] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:25 2022] DELETE /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 111 bytes in 125 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:25.856662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-857135926", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c130cb51-e60d-43b7-af80-10e77fbe9595"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:25.953648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] validate_networks() for [('c130cb51-e60d-43b7-af80-10e77fbe9595', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:00:28.579267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:00:28.582396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:00:28.583571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:00:28.584658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:00:28.587421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:00:28.688060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:28.690961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:28.691772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:28.692950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:28.693616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:28.694339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:28.776509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:28.796085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Getting quotas for user 5d93a680570f4d23b1f6df534bc082b2 and project b62c28804ebd4f1f9ac47aa541500110. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:28.846259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:00:28.870510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:00:29.017149 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bfa00c9a-41c0-4749-af19-9e41a503954c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.170151 Oct 15 21:00:29.017876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1736/2850] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:00:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 3176 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:00:29.065347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:29.065972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:29.109860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:29.111226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:29.111683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:29.112735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:29.113513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:29.113950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:29.321211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:29.379227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef70e17c-e1b5-4bb5-8335-f19a85243693 tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1935 microversion: 2.75 time: 0.348912 Oct 15 21:00:29.391852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1737/2851] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:29 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1935 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:29.419505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0cb354d8-b1e5-4302-97b4-45dd56284091 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75846e430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:29.599490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0cb354d8-b1e5-4302-97b4-45dd56284091 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.206234 Oct 15 21:00:29.600817 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1738/2852] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:29 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:29.614558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:29.623546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] [instance: 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:29.655020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:29.655530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:29.655977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:29.656697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:29.674581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:29.674778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:29.734026 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] HTTP exception thrown: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Request method failure captured: Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a HTTP/1.1 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4jw5-UiB7krGX-pOJaYIckg5_4lhh49cEjK52K9H7CwAyOEm42c1Fq5sHy_Hqw4pKyrRK04je3OOwHcRJFWdsecOBPt_SilfcM5ebXQsXP3GChiI38kkqh3ZHBdo6KSurlsR3OBaM0EA4Yfd-FfjtOXKjOchDjBmFwZRH_oE_IMA8 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ca9af3805eba4f69b474a8b2205f0920"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ca9af3805eba4f69b474a8b2205f0920 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-1725261563 Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2bba6c9b3f674959ad79d4a394ed9f2a Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-1725261563-project Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a'} Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:29.736065 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. Oct 15 21:00:29.740296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:29.746698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] Returning 404 to user: Instance 049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:29.752889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81cf3ed9-9fa7-48f6-bb56-aa8bbeb8d3b5 tempest-DeleteServersTestJSON-1725261563 tempest-DeleteServersTestJSON-1725261563-project] 10.222.0.16 "GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a" status: 404 len: 111 microversion: 2.1 time: 0.146543 Oct 15 21:00:29.753874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1739/2853] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:29 2022] GET /compute/v2.1/servers/049caaa1-b3a9-42b4-9f2c-e7c9115f5e0a => generated 111 bytes in 153 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:00:29.761996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:29.763088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:29.769720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:29.770504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:29.771139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:29.772049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:29.772719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:29.778483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:29.829413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:30.577682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:30.593535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4ad8f04-cb20-42a7-8813-7696d0bd9ffe tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1309 microversion: 2.1 time: 0.837984 Oct 15 21:00:30.619266 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1740/2854] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:29 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1309 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:30.628474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c8538e64-009e-477a-ad58-4ef239cd1d44 req-6cdf3ac6-65bb-4a2b-bb0d-b0d58813da24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "60051f47-77db-45a6-9e66-c0b1b1528667", "name": "network-vif-plugged", "status": "completed", "tag": "dc662ecd-8ebc-4738-a455-d2a57d721b0b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:30.659509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c8538e64-009e-477a-ad58-4ef239cd1d44 req-6cdf3ac6-65bb-4a2b-bb0d-b0d58813da24 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:30.660317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c8538e64-009e-477a-ad58-4ef239cd1d44 req-6cdf3ac6-65bb-4a2b-bb0d-b0d58813da24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:30.660967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c8538e64-009e-477a-ad58-4ef239cd1d44 req-6cdf3ac6-65bb-4a2b-bb0d-b0d58813da24 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:30.675223 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c8538e64-009e-477a-ad58-4ef239cd1d44 req-6cdf3ac6-65bb-4a2b-bb0d-b0d58813da24 service nova] Creating event network-vif-plugged:dc662ecd-8ebc-4738-a455-d2a57d721b0b for instance 60051f47-77db-45a6-9e66-c0b1b1528667 on nch2-master-1 Oct 15 21:00:30.709818 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c8538e64-009e-477a-ad58-4ef239cd1d44 req-6cdf3ac6-65bb-4a2b-bb0d-b0d58813da24 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088336 Oct 15 21:00:30.714123 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1741/2855] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:31.029858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-tempest.common.compute-instance-1740021837", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 2, "networks": [{"uuid": "d675349d-9eba-4ab3-9302-bd7c72913332"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:31.165855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] validate_networks() for [('d675349d-9eba-4ab3-9302-bd7c72913332', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:00:33.160529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:00:33.161680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:00:33.162355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:00:33.162930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:00:33.163507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:00:33.191599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:33.192103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:33.192541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:33.193318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:33.193782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:33.194213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:33.251827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Getting quotas for project 9bc93093ddf44aada9598a14b5417a2d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:33.257485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Getting quotas for user 4e1353ffe42243af82b590bc98e96a42 and project 9bc93093ddf44aada9598a14b5417a2d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:33.289164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Going to run 2 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:00:33.291902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: eebafc32-9d60-4131-92ac-3da43cc822a9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:00:33.312240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:33.346507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=eebafc32-9d60-4131-92ac-3da43cc822a9,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:00:33.431567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03aeb2de-72eb-4c97-873f-95f742810ec1 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.713692 Oct 15 21:00:33.432850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1742/2856] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:00:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 2718 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:00:33.438715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:33.439524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:33.452727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:33.456687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:33.458506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:33.460068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:33.461874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:33.463656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:33.705662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:33.715313 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b831fcf7-e540-43b6-bd92-f6230c67daf2 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1265 microversion: 2.1 time: 0.280150 Oct 15 21:00:33.721984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1743/2857] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:33 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1265 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:33.849808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-793f6bae-72c9-459b-896a-ce7ae8b493e9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'create', calling method: .version_select at 0x7fe7584ca5e0>, body: {"keypair": {"name": "tempest-keypair-1037964320", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCMeeDY6oFj4Cbc+lf8IzNLTJsxlwHKU4FsuQ54wRRO81dQakCOBnyQyskLDFX16yt1hDc9Duj8xujSsUphOyKFMX6ocBa8PIOXR934eBnch4GB/00b87nf8+oA0o32WMg=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:33.888367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-793f6bae-72c9-459b-896a-ce7ae8b493e9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for project c04b2976577d42799a800d21545d1969. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:33.930916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-793f6bae-72c9-459b-896a-ce7ae8b493e9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for user fb8dc371c65f4bbb92d559c7ed78964c and project c04b2976577d42799a800d21545d1969. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:33.999889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-793f6bae-72c9-459b-896a-ce7ae8b493e9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.275892 Oct 15 21:00:34.001994 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1744/2858] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 21:00:33 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 279 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 21:00:34.015239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0632429f-67d1-4182-a46c-1897419888ce req-468a5895-cfb7-4de4-a2af-39d261069eb7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "359b81c9-e8fc-491e-95d7-b2887dfb032f", "name": "network-vif-deleted", "tag": "4ae4edec-e1c4-4b33-a749-85c9de8a0ed8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:34.046049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0632429f-67d1-4182-a46c-1897419888ce req-468a5895-cfb7-4de4-a2af-39d261069eb7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:34.060136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0632429f-67d1-4182-a46c-1897419888ce req-468a5895-cfb7-4de4-a2af-39d261069eb7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:34.065985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0632429f-67d1-4182-a46c-1897419888ce req-468a5895-cfb7-4de4-a2af-39d261069eb7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:34.103862 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0632429f-67d1-4182-a46c-1897419888ce req-468a5895-cfb7-4de4-a2af-39d261069eb7 service nova] Creating event network-vif-deleted:4ae4edec-e1c4-4b33-a749-85c9de8a0ed8 for instance 359b81c9-e8fc-491e-95d7-b2887dfb032f on nch1-master-1 Oct 15 21:00:34.124672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0632429f-67d1-4182-a46c-1897419888ce req-468a5895-cfb7-4de4-a2af-39d261069eb7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120473 Oct 15 21:00:34.127731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1745/2859] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:34.166046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:34.172167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:34.196373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:34.206443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:34.222294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:34.224568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:34.226246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:34.228300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:34.562363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] Lazy-loading 'services' on Instance uuid fac4feea-5ad7-49d5-9777-ec8db4b6a4fe {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:34.610582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f25b4d8e-15ef-4c6f-b1f8-83ca32aa81ab tempest-ServersAdmin275Test-587614346 tempest-ServersAdmin275Test-587614346-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1920 microversion: 2.75 time: 0.477909 Oct 15 21:00:34.627668 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1746/2860] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:34 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1920 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:34.638601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e51bb6b-c076-4b2b-9c4f-dee707b4a994 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe75857f5e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:34.746461 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e51bb6b-c076-4b2b-9c4f-dee707b4a994 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.117383 Oct 15 21:00:34.757204 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1747/2861] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:34 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:34.777259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:34.779328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:34.796946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:34.801365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:34.801869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:34.802663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:34.803135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:34.803609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:35.821931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:35.837601 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa324f91-89f6-4fc2-97a7-fe6dbd530206 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1491 microversion: 2.1 time: 1.079281 Oct 15 21:00:35.839979 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1748/2862] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:34 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1491 bytes in 1082 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:35.857659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:35.880577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:35.885561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Searching by: {'deleted': False, 'project_id': '9bc93093ddf44aada9598a14b5417a2d'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:00:35.913499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:35.916038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:35.916615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:35.921370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:35.922151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:35.922849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:36.020769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:00:36.111836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5938e848-57ef-4405-8df9-eee402d1121b tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 677 microversion: 2.1 time: 0.263603 Oct 15 21:00:36.115431 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1749/2863] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:00:35 2022] GET /compute/v2.1/servers => generated 677 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:36.139360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:36.140402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:36.176696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:36.177558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:36.178700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:36.179631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:36.180284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:36.180900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:36.266822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:00:36.455355 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f31b583-a1a6-4922-8dec-03042952bac3 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 204 len: 0 microversion: 2.75 time: 0.335315 Oct 15 21:00:36.456580 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1750/2864] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:00:36 2022] DELETE /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 0 bytes in 337 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:00:36.466176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:36.467666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:36.477221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:36.480702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:36.481615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:36.482732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:36.483435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:36.484021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:36.920194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:36.946930 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77c2813b-5e8b-4875-aac3-b2a5ac6bba75 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1265 microversion: 2.1 time: 0.484979 Oct 15 21:00:36.953625 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1751/2865] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:36 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1265 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:36.978972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1db03847-5b93-40c7-9340-c0a37e3e8a20 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe7584ca3a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:37.099342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1db03847-5b93-40c7-9340-c0a37e3e8a20 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1428 microversion: 2.1 time: 0.144224 Oct 15 21:00:37.099925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1752/2866] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:36 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1428 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:37.113687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'update', calling method: >, body: {"server": {"accessIPv4": "1.1.1.1", "accessIPv6": "::babe:202:202"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:37.116229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:37.145633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:37.146138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:37.146599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:37.147393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:37.147853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:37.148283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:37.441485 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8d66bb3-e27f-427f-a3ed-3563a873ee2f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "PUT /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1086 microversion: 2.1 time: 0.337359 Oct 15 21:00:37.442542 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1753/2867] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:00:37 2022] PUT /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1086 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:37.451636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:37.453894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:37.471909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:37.476222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:37.476922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:37.479076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:37.483467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:37.484275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:37.874538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:37.893823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39bd935b-a406-4fd6-b641-055c3135b9eb tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1278 microversion: 2.1 time: 0.450008 Oct 15 21:00:37.895020 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1754/2868] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:37 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1278 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:37.899941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:37.900778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:37.922331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:37.923552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:37.924148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:37.925021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:37.925682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:37.926294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:38.315323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:38.326920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:38.335374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '42276cf2f2894712ac91f87e4ce1f6ca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9ea11fd1a5a64f4a9641d7ed07f8e94f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:38.353255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de101660-cc72-4297-9a46-73c794b697f5 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 200 len: 1444 microversion: 2.75 time: 0.456757 Oct 15 21:00:38.382475 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1755/2869] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:37 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 1444 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:38.392769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:38.393567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:38.400813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:38.401428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:38.401853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:38.402603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:38.403076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:38.403586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:39.459784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:39.470744 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddde300c-b086-477c-bfb8-0355a066e462 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1512 microversion: 2.1 time: 1.086998 Oct 15 21:00:39.472062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1756/2870] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:38 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1512 bytes in 1089 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:39.478274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1930689348", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a8bfc216-af8d-45e1-9e10-7c244ff5c0ea"}], "security_groups": [{"name": "tempest-securitygroup--1932595840"}], "key_name": "tempest-keypair-1037964320", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:39.904793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] validate_networks() for [('a8bfc216-af8d-45e1-9e10-7c244ff5c0ea', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:00:42.576973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:00:42.577646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:00:42.577944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:00:42.578772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:00:42.579609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:00:42.600641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:42.601199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:42.601647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:42.602835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:42.603466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:42.604618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:42.630986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for project c04b2976577d42799a800d21545d1969. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:42.641920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for user fb8dc371c65f4bbb92d559c7ed78964c and project c04b2976577d42799a800d21545d1969. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:42.663324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:00:42.667521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:00:42.753073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7476cf3b-8ff1-417d-9401-e719237e89d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 3.279468 Oct 15 21:00:42.754098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1757/2871] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:00:39 2022] POST /compute/v2.1/servers => generated 402 bytes in 3281 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:00:42.764437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:42.765394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:42.780660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:42.783318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:42.791221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:42.793207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:42.793887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:42.795065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:43.593660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:43.614555 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22b0e498-eff2-4da6-bf7f-4aa3c6f76f38 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1307 microversion: 2.1 time: 0.859264 Oct 15 21:00:43.615238 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1758/2872] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:42 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1307 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:43.620168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c27a7b7-83b9-4d51-ad2b-75b0a44058cb tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe7583b93a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:43.690972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c27a7b7-83b9-4d51-ad2b-75b0a44058cb tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1437 microversion: 2.1 time: 0.072959 Oct 15 21:00:43.691786 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1759/2873] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:43 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1437 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:43.697385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:43.699485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:43.706389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:43.707270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:43.707948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:43.708960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:43.709689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:43.710344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:44.518704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:44.522296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:44.529868 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3940bc9-8f8c-4dcc-b7ab-ea4df4def1ac tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1320 microversion: 2.1 time: 0.835683 Oct 15 21:00:44.534878 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1760/2874] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:43 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1320 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:44.543028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:44.544099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] [instance: fac4feea-5ad7-49d5-9777-ec8db4b6a4fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:44.553319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:44.554249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:44.554931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:44.555920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:44.563898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:44.564578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:44.590604 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] HTTP exception thrown: Instance fac4feea-5ad7-49d5-9777-ec8db4b6a4fe could not be found. Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Request method failure captured: Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe HTTP/1.1 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8XCjNnWz8ExDL4jk3NroBQZHJ_dG6ODDdsraACrqpHqB_i0q-L6w0_V22vpIkKWM4KWyiHSHsHghOWbs90mRhmckKN2ZB4NOriL1ECzGdaw-JWcvQkMf9aOak7uG4wRAz0n1gLK4Ga-RHwK2ai65wadbpc03qg6wajSSYXD1gARSc Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.75 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 9ea11fd1a5a64f4a9641d7ed07f8e94f Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdmin275Test-1391528656 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9ea11fd1a5a64f4a9641d7ed07f8e94f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9ea11fd1a5a64f4a9641d7ed07f8e94f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9ea11fd1a5a64f4a9641d7ed07f8e94f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9ea11fd1a5a64f4a9641d7ed07f8e94f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdmin275Test-1391528656 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 9ea11fd1a5a64f4a9641d7ed07f8e94f Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdmin275Test-1391528656 Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdmin275Test-1391528656-project Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 42276cf2f2894712ac91f87e4ce1f6ca Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdmin275Test-1391528656-project Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance fac4feea-5ad7-49d5-9777-ec8db4b6a4fe could not be found. Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'fac4feea-5ad7-49d5-9777-ec8db4b6a4fe'} Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fac4feea-5ad7-49d5-9777-ec8db4b6a4fe could not be found. Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fac4feea-5ad7-49d5-9777-ec8db4b6a4fe could not be found. Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:44.592298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fac4feea-5ad7-49d5-9777-ec8db4b6a4fe could not be found. Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] Returning 404 to user: Instance fac4feea-5ad7-49d5-9777-ec8db4b6a4fe could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:44.598132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d172ecb-41e9-432b-938b-af1a0b5f67e6 tempest-ServersAdmin275Test-1391528656 tempest-ServersAdmin275Test-1391528656-project] 10.222.0.16 "GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe" status: 404 len: 111 microversion: 2.75 time: 0.060271 Oct 15 21:00:44.600680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1761/2875] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:44 2022] GET /compute/v2.1/servers/fac4feea-5ad7-49d5-9777-ec8db4b6a4fe => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:00:44.610098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:44.610776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:44.633115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:44.635432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:44.637891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:44.638835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:44.647077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:44.647875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:45.839024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:45.848558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-028bd9c1-6f49-48a5-a9bd-1b7b64a311f9 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1512 microversion: 2.1 time: 1.246751 Oct 15 21:00:45.857186 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1762/2876] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:44 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1512 bytes in 1248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:45.857186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:45.857186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:45.862087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:45.862588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:45.863428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:45.864311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:45.864916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:45.865516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:46.033983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:46.044324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:46.045652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:46.046789 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d4ebb4b-6370-4a5d-b2b2-7f1b5d61b66e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1340 microversion: 2.20 time: 0.196239 Oct 15 21:00:46.047625 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1763/2877] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:45 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1340 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:46.052434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-15b3cf97-82f6-4094-bab3-364f97f0574f req-839f38ae-3130-43a3-b470-4b6e7764f360 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "02685830-954d-444e-a85a-55088f3e0e72", "tag": "3083b290-a6ed-4c3c-b8fb-0637091c4c38"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:46.062886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-15b3cf97-82f6-4094-bab3-364f97f0574f req-839f38ae-3130-43a3-b470-4b6e7764f360 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:46.063380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-15b3cf97-82f6-4094-bab3-364f97f0574f req-839f38ae-3130-43a3-b470-4b6e7764f360 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:46.063822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-15b3cf97-82f6-4094-bab3-364f97f0574f req-839f38ae-3130-43a3-b470-4b6e7764f360 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:46.078243 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-15b3cf97-82f6-4094-bab3-364f97f0574f req-839f38ae-3130-43a3-b470-4b6e7764f360 service nova] Creating event network-changed:3083b290-a6ed-4c3c-b8fb-0637091c4c38 for instance 02685830-954d-444e-a85a-55088f3e0e72 on nch1-master-1 Oct 15 21:00:46.084144 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-15b3cf97-82f6-4094-bab3-364f97f0574f req-839f38ae-3130-43a3-b470-4b6e7764f360 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034954 Oct 15 21:00:46.085438 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1764/2878] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:46.092530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-451c2bcb-bfa8-4d4e-b6f1-858fba3aee2a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe758352280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:46.155908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-451c2bcb-bfa8-4d4e-b6f1-858fba3aee2a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 200 len: 1437 microversion: 2.1 time: 0.068866 Oct 15 21:00:46.156753 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1765/2879] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:00:46 2022] GET /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 1437 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:46.161892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bd7b35a8-e8f0-48c0-91e3-3a017d82a361 req-d077a179-2642-48a8-b9fe-d07ae29140f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "name": "network-vif-unplugged", "status": "completed", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:46.173822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bd7b35a8-e8f0-48c0-91e3-3a017d82a361 req-d077a179-2642-48a8-b9fe-d07ae29140f1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:46.174622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bd7b35a8-e8f0-48c0-91e3-3a017d82a361 req-d077a179-2642-48a8-b9fe-d07ae29140f1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:46.175235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bd7b35a8-e8f0-48c0-91e3-3a017d82a361 req-d077a179-2642-48a8-b9fe-d07ae29140f1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:46.186853 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-bd7b35a8-e8f0-48c0-91e3-3a017d82a361 req-d077a179-2642-48a8-b9fe-d07ae29140f1 service nova] Creating event network-vif-unplugged:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 21:00:46.205283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bd7b35a8-e8f0-48c0-91e3-3a017d82a361 req-d077a179-2642-48a8-b9fe-d07ae29140f1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.040148 Oct 15 21:00:46.206360 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1766/2880] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:46 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:46.214482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:46.215483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:46.223321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:46.223817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:46.224247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:46.225386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:46.225835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:46.226535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:46.698646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:46.710531 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-681bd33e-bf14-41c5-b485-2b24e389e02c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1307 microversion: 2.1 time: 0.502753 Oct 15 21:00:46.711695 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1767/2881] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:46 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1307 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:46.726658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:46.730570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:46.737259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:46.737941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:46.738703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:46.739535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:46.740400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:46.740941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:47.425520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:47.434861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-49a267fc-ecd5-4773-a9fa-1dac1ba02f87 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1320 microversion: 2.1 time: 0.722042 Oct 15 21:00:47.435750 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1768/2882] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:46 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1320 bytes in 724 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:47.447440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:47.448854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:47.457340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:47.459359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:47.466115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:47.470230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:47.472631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:47.475881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:47.517524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:00:47.673414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f30b6669-f28d-4367-9eae-8cf3eb70b500 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 204 len: 0 microversion: 2.1 time: 0.235748 Oct 15 21:00:47.675774 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1769/2883] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:00:47 2022] DELETE /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:47.695619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b1686e5-bd8b-4ac4-9281-d18beefce6f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Action: 'create', calling method: .version_select at 0x7fe758502dc0>, body: {"keypair": {"name": "tempest-keypair-487010634", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ4qkXTS2VfkDESUYs7EapHXE7g4cSjzg7ynNB2qMjXGbs68+LLX/Kb+NRc3mG1353Lvvykd7CYsR7W7XS+sA7qku4/hXKm5tew0xwaNcU00kTUKyoA96IM6hwXlpWpNkg=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:47.743766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2b1686e5-bd8b-4ac4-9281-d18beefce6f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Getting quotas for project 12a874b752564923ad6a4747c7a44d61. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:47.750180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:INSERT=10,nova_api:SELECT=88,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:47.759351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-2b1686e5-bd8b-4ac4-9281-d18beefce6f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Getting quotas for user 84c19569dc6d49f4a29c814ef921e144 and project 12a874b752564923ad6a4747c7a44d61. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:47.798355 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b1686e5-bd8b-4ac4-9281-d18beefce6f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.42 time: 0.116483 Oct 15 21:00:47.805863 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1770/2884] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 21:00:47 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 120 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 21:00:47.808794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:47.810161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:47.838452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:47.840783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:47.841640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:47.843092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:47.844022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:47.844832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:48.377844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:48.386812 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fd00ad9-bd1e-4cf8-838d-1d261f15ebbd tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1467 microversion: 2.1 time: 0.585667 Oct 15 21:00:48.393852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1771/2885] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:47 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1467 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:48.409248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:48.410237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:48.434822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:48.435474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:48.435861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:48.436548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:48.437009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:48.437491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:48.611010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:48.633929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:48.641316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:48.643953 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1111b5d6-ae6a-44a4-8b1b-c677e41f8f75 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1340 microversion: 2.20 time: 0.248116 Oct 15 21:00:48.657215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1772/2886] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:48 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1340 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:48.677485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:48.679225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:48.702163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:48.702917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:48.703629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:48.704558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:48.705272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:48.705918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:49.603006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:49.620734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d38f6fde-4ffc-4ccf-b051-8c14c2146593 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1518 microversion: 2.1 time: 0.950723 Oct 15 21:00:49.622260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1773/2887] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:48 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1518 bytes in 955 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:49.635242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:49.636933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:49.644296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:49.645023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:49.645582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:49.646302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:49.646796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:49.647229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:50.035378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:50.045310 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22efe4f6-a8f7-4346-85e2-a34be92914b5 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1307 microversion: 2.1 time: 0.421232 Oct 15 21:00:50.046562 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1774/2888] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:49 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1307 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:50.052190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-11cee6df-45e8-4743-997c-91e87bc4d4f0 req-cbf921cd-e7b7-47b0-9398-73b100e93446 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ba5bfc3-c9bd-4949-a2dd-254a4bba450b", "tag": "b758a233-2c34-4df8-a6d5-229769b08481"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:50.075151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-11cee6df-45e8-4743-997c-91e87bc4d4f0 req-cbf921cd-e7b7-47b0-9398-73b100e93446 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:50.075988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-11cee6df-45e8-4743-997c-91e87bc4d4f0 req-cbf921cd-e7b7-47b0-9398-73b100e93446 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:50.076688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-11cee6df-45e8-4743-997c-91e87bc4d4f0 req-cbf921cd-e7b7-47b0-9398-73b100e93446 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:50.103511 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-11cee6df-45e8-4743-997c-91e87bc4d4f0 req-cbf921cd-e7b7-47b0-9398-73b100e93446 service nova] Creating event network-changed:b758a233-2c34-4df8-a6d5-229769b08481 for instance 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b on nch2-master-1 Oct 15 21:00:50.116316 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-11cee6df-45e8-4743-997c-91e87bc4d4f0 req-cbf921cd-e7b7-47b0-9398-73b100e93446 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067189 Oct 15 21:00:50.117666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1775/2889] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:50.130333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21b0cfcb-1af3-4649-8a4e-06c472e6de5a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe7584caf70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:51.014033 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21b0cfcb-1af3-4649-8a4e-06c472e6de5a tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "DELETE /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a" status: 204 len: 0 microversion: 2.1 time: 0.891991 Oct 15 21:00:51.014679 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1776/2890] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:50 2022] DELETE /compute/v2.1/images/5b2d8fbc-682f-4e19-a304-02c3da01df5a => generated 0 bytes in 896 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:00:51.293663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-1449102785", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:51.394899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:00:52.509532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:00:52.510138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:00:52.510934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:00:52.511361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:00:52.511772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:00:52.531622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:52.531946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:52.532490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:52.533992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:52.535726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:52.536536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:52.579673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Getting quotas for project e91f96ee17114009a55fe2bf388a7aa7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:52.592196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Getting quotas for user 573c02d502744876babbdb4fbb4fb9e6 and project e91f96ee17114009a55fe2bf388a7aa7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:52.620190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:00:52.623236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:00:52.731031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43b1a505-cc52-4651-b844-5a87ab9ee9c4 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.715243 Oct 15 21:00:52.732252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1777/2891] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:00:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 1717 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:00:52.747756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:52.749208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:52.759215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:52.759940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:52.761183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:52.762445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:52.763065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:52.765750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:53.398132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:53.411809 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8d33b26-e05c-4b11-8ec6-0421ef2f61e0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1320 microversion: 2.1 time: 0.670280 Oct 15 21:00:53.413678 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1778/2892] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:52 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1320 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:53.431334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:53.432544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:53.457805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:53.458430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:53.459344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:53.460211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:53.461017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:53.461720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:54.300184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:54.318788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:54.320277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:54.328962 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7446e815-2095-4f3e-9c28-6b14cf01b21f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1408 microversion: 2.20 time: 0.911653 Oct 15 21:00:54.329622 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1779/2893] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:00:53 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1408 bytes in 915 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:00:54.343752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:54.344676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:54.355930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:54.357618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:54.358337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:54.359310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:54.360007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:54.361435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:54.475901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:54.476705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:54.477183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:54.477657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lazy-loading 'fault' on Instance uuid 60051f47-77db-45a6-9e66-c0b1b1528667 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:00:54.564303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:54.574646 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-081c372b-f1bf-4a1e-a1ef-644da583bef6 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 200 len: 1320 microversion: 2.1 time: 0.241828 Oct 15 21:00:54.577207 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1780/2894] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:54 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 1320 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:54.585251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bdeacaf-fc3b-43ff-9a07-7dc65974441c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '.version_select at 0x7fe74f14ddc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:54.650825 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6bdeacaf-fc3b-43ff-9a07-7dc65974441c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] HTTP exception thrown: Image not found. Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bdeacaf-fc3b-43ff-9a07-7dc65974441c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Request method failure captured: Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 HTTP/1.1 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8PX0-49snCz46mp-NhhQmPmVLgOYnXRDBm5JZolI1wgX3QUCaT9-lCECUUcQru_YYXFhNc9VKPOfClBYhxcvHWxiBB0zyTeNnuzf3YvcHosWw4P2pG2U_a8aeLFvXKayL7urDAFU5iSPj4NlGYI6PUdshLtdi42vyh8Je7UAoRxGE Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 57ac50a5098942e58c8ab72ff09d5159 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 57ac50a5098942e58c8ab72ff09d5159 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerTestJSON-1585030782-project Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 83d85ef4ebdb452a93f24682e2bbd46c Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerTestJSON-1585030782-project Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f3ca929c-bd96-4eca-a596-b2ee1ac252e8'} Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image f3ca929c-bd96-4eca-a596-b2ee1ac252e8 to delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image f3ca929c-bd96-4eca-a596-b2ee1ac252e8 could not be found. Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.654498 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:00:54.659462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bdeacaf-fc3b-43ff-9a07-7dc65974441c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:00:54.661638 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6bdeacaf-fc3b-43ff-9a07-7dc65974441c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "DELETE /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8" status: 404 len: 62 microversion: 2.1 time: 0.079733 Oct 15 21:00:54.661638 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1781/2895] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:54 2022] DELETE /compute/v2.1/images/f3ca929c-bd96-4eca-a596-b2ee1ac252e8 => generated 62 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:00:54.666535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:00:54.669657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:00:54.704151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:54.704669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:54.705154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:54.705870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:54.706330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:54.737066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.031s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:54.932896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:00:54.937003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:54.943514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef3cde31-0c37-4e29-9e8a-de4fe53f6e8a tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1307 microversion: 2.1 time: 0.281069 Oct 15 21:00:54.944406 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1782/2896] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:00:54 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1307 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:00:54.967499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f08f45cf-f575-494b-a403-fa6ca82f2155 req-cea28335-9aee-41d9-9600-65336ca5a228 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02685830-954d-444e-a85a-55088f3e0e72", "name": "network-vif-plugged", "status": "completed", "tag": "3083b290-a6ed-4c3c-b8fb-0637091c4c38"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:54.989701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f08f45cf-f575-494b-a403-fa6ca82f2155 req-cea28335-9aee-41d9-9600-65336ca5a228 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:54.990486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f08f45cf-f575-494b-a403-fa6ca82f2155 req-cea28335-9aee-41d9-9600-65336ca5a228 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:54.991568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f08f45cf-f575-494b-a403-fa6ca82f2155 req-cea28335-9aee-41d9-9600-65336ca5a228 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:55.015479 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f08f45cf-f575-494b-a403-fa6ca82f2155 req-cea28335-9aee-41d9-9600-65336ca5a228 service nova] Creating event network-vif-plugged:3083b290-a6ed-4c3c-b8fb-0637091c4c38 for instance 02685830-954d-444e-a85a-55088f3e0e72 on nch1-master-1 Oct 15 21:00:55.024713 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f08f45cf-f575-494b-a403-fa6ca82f2155 req-cea28335-9aee-41d9-9600-65336ca5a228 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077809 Oct 15 21:00:55.027584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1783/2897] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:00:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:00:55.034306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-1267559979", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "0caf785e-670e-48e9-9bc9-fcc9689581a7"}, {"port": "d1754b9c-eda8-41d8-99a6-7cc5c3b7c3eb", "tag": "port-1"}, {"port": "d5ed5624-9ab3-4ee7-8a25-54a7b3d3bdef", "tag": "port-2"}, {"uuid": "595a5e8f-1f6e-4bc5-ad7e-bea1c084ff37", "tag": "net-1"}, {"uuid": "595a5e8f-1f6e-4bc5-ad7e-bea1c084ff37"}, {"uuid": "d034ac2e-a33c-4a5f-8062-059da1e4c03f", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "d034ac2e-a33c-4a5f-8062-059da1e4c03f", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "bab8bc29-5550-4fc0-897e-f06ff3f05efc", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "b59c2e3f-399b-485b-b7cb-c12e5d16ef95", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "4c2c4a02-0af0-475a-b644-cecbb2a2056d", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--1775142489"}], "key_name": "tempest-keypair-487010634", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:00:55.239268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] validate_networks() for [('0caf785e-670e-48e9-9bc9-fcc9689581a7', None, None, None, None, None), (None, None, 'd1754b9c-eda8-41d8-99a6-7cc5c3b7c3eb', None, None, None), (None, None, 'd5ed5624-9ab3-4ee7-8a25-54a7b3d3bdef', None, None, None), ('595a5e8f-1f6e-4bc5-ad7e-bea1c084ff37', None, None, None, None, None), ('595a5e8f-1f6e-4bc5-ad7e-bea1c084ff37', None, None, None, None, None), ('d034ac2e-a33c-4a5f-8062-059da1e4c03f', '10.2.2.100', None, None, None, None), ('d034ac2e-a33c-4a5f-8062-059da1e4c03f', '10.2.2.200', None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:00:59.528238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:00:59.528785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:00:59.529653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:00:59.530749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:00:59.531182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:00:59.553975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=33 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:59.562530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:59.563450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:59.565997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:59.566964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:00:59.569256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:00:59.569716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:00:59.581910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=11 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:00:59.596372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Getting quotas for project 12a874b752564923ad6a4747c7a44d61. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:00:59.603563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Getting quotas for user 84c19569dc6d49f4a29c814ef921e144 and project 12a874b752564923ad6a4747c7a44d61. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:00:59.849629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:00:59.855556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='bab8bc29-5550-4fc0-897e-f06ff3f05efc',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='b59c2e3f-399b-485b-b7cb-c12e5d16ef95',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='4c2c4a02-0af0-475a-b644-cecbb2a2056d',volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:01:00.472143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-56134ca0-34ce-48be-a034-17e667024b80 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.42 time: 5.443231 Oct 15 21:01:00.474739 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1784/2898] 10.222.0.16 () {62 vars in 1189 bytes} [Sat Oct 15 21:00:55 2022] POST /compute/v2.1/servers => generated 406 bytes in 5447 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:01:00.492949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eebafc32-9d60-4131-92ac-3da43cc822a9", "tag": "9af00388-8f08-4aba-b946-41ee5181975e"}, {"name": "network-changed", "server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:00.535947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:00.536625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:00.537233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:00.553727 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] Creating event network-changed:9af00388-8f08-4aba-b946-41ee5181975e for instance eebafc32-9d60-4131-92ac-3da43cc822a9 on nch1-master-1 Oct 15 21:01:00.554369 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] Creating event network-changed:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:01:00.561959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9acc36ec-6163-4d6c-bc57-42796d3b85d6 req-3776c676-1d07-4881-8bc6-f63cf0696edf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.085516 Oct 15 21:01:00.563311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1785/2899] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:00 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:00.571582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:00.576632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:00.586247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:00.587292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:00.587988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:00.589489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:00.590224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:00.591899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:00.967208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:00.977987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f16dda9e-0d8b-4472-9043-fdcfb2c32f71 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1322 microversion: 2.1 time: 0.413091 Oct 15 21:01:00.981077 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1786/2900] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:00 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1322 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:00.999581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:01.000294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:01.018506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:01.019431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:01.021505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:01.024986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:01.025780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:01.028551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:01.751581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:01.761958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3fd2c42-3c12-4295-b280-e1a800cf5a12 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1320 microversion: 2.1 time: 0.777705 Oct 15 21:01:01.766179 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1787/2901] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:00 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1320 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:01.772061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:01.777572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:01.790797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:01.793554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:01.796204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:01.798592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:01.799348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:01.800008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:01.856863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:01:02.026654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e291cb0-9bb7-4519-9dca-e1627fcae588 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 204 len: 0 microversion: 2.1 time: 0.257825 Oct 15 21:01:02.030462 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1788/2902] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:01 2022] DELETE /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:02.035236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:02.036133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:02.046815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:02.047411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:02.047919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:02.048751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:02.049418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:02.049898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:02.621307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:02.639537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:02.659144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:02.661809 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e9eb77d-6053-48f7-af88-83072e517e27 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1408 microversion: 2.20 time: 0.632781 Oct 15 21:01:02.683351 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1789/2903] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:02 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1408 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:02.701861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:02.702627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:02.719849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:02.720591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:02.721230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:02.722200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:02.722883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:02.727068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:02.754928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 HTTP/1.1 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '60051f47-77db-45a6-9e66-c0b1b1528667'} Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:02.758165 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:01:02.762296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:02.765073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:02.766414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a03793ad-a3c4-4c07-b6a9-89afae97adf7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 404 len: 111 microversion: 2.1 time: 0.080178 Oct 15 21:01:02.767746 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1790/2904] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:02 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:02.773139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:02.774038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:02.818658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:02.819564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:02.820229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:02.821358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:02.822064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:02.822716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.137258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.151514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6f2e827-e301-436a-ba9f-fa61d536963b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1488 microversion: 2.1 time: 0.382297 Oct 15 21:01:03.152672 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1791/2905] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:02 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1488 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:03.158092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-dacee794-670f-421d-bb0f-c3cfdd87b635 req-bae97216-ce82-41f1-aa68-850abb646103 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ba5bfc3-c9bd-4949-a2dd-254a4bba450b", "name": "network-vif-plugged", "status": "completed", "tag": "b758a233-2c34-4df8-a6d5-229769b08481"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:03.175417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dacee794-670f-421d-bb0f-c3cfdd87b635 req-bae97216-ce82-41f1-aa68-850abb646103 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.178158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dacee794-670f-421d-bb0f-c3cfdd87b635 req-bae97216-ce82-41f1-aa68-850abb646103 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.178763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dacee794-670f-421d-bb0f-c3cfdd87b635 req-bae97216-ce82-41f1-aa68-850abb646103 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.193883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-dacee794-670f-421d-bb0f-c3cfdd87b635 req-bae97216-ce82-41f1-aa68-850abb646103 service nova] Creating event network-vif-plugged:b758a233-2c34-4df8-a6d5-229769b08481 for instance 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b on nch2-master-1 Oct 15 21:01:03.201405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-dacee794-670f-421d-bb0f-c3cfdd87b635 req-bae97216-ce82-41f1-aa68-850abb646103 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046430 Oct 15 21:01:03.202503 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1792/2906] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:03.210936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:03.211787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:03.221358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.222587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.223325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.224160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.224588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.225108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.305234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.320116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.322508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.324076 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-020094c7-3a26-4ae8-aab3-b64a88d14585 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1527 microversion: 2.42 time: 0.120137 Oct 15 21:01:03.326213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1793/2907] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:03 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1527 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:03.347914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:03.348545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:03.355081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.355762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.356204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.357168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.357779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.358362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.554601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.564185 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e71adc40-723f-4277-9ba5-87bdeaa5d6d9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1322 microversion: 2.1 time: 0.236585 Oct 15 21:01:03.564937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1794/2908] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:03 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1322 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:03.570209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:03.571130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:03.578535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.579652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.580478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.581553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.582221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.582958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.740992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.748757 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d95f3dec-ecd6-4410-a641-0c78665a3156 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1473 microversion: 2.1 time: 0.182542 Oct 15 21:01:03.749357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1795/2909] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:03 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1473 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:03.754027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "60051f47-77db-45a6-9e66-c0b1b1528667", "name": "network-vif-deleted", "tag": "dc662ecd-8ebc-4738-a455-d2a57d721b0b"}, {"name": "network-changed", "server_uuid": "c8032d6f-ff10-4add-be09-6de0bbdcf98f", "tag": "f4da8690-5405-4369-97d1-4c91b63a3c57"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:03.765907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.766533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.766866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.781316 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] Creating event network-vif-deleted:dc662ecd-8ebc-4738-a455-d2a57d721b0b for instance 60051f47-77db-45a6-9e66-c0b1b1528667 on nch2-master-1 Oct 15 21:01:03.781863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] Creating event network-changed:f4da8690-5405-4369-97d1-4c91b63a3c57 for instance c8032d6f-ff10-4add-be09-6de0bbdcf98f on nch2-master-1 Oct 15 21:01:03.789059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6a2dee2e-c445-4643-a0a4-bbd67291e6a1 req-54eb2011-97fe-44cd-a62c-8dc072501ee8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.038375 Oct 15 21:01:03.793132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1796/2910] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:03 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:03.798627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:03.799557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:03.805960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.806459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.806906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.807682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:03.808148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:03.808588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:03.957501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:03.968013 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5528eab7-8df5-4da2-94a3-d9a4b89e78db tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1320 microversion: 2.1 time: 0.173331 Oct 15 21:01:03.968533 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1797/2911] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:03 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1320 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:03.978110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-100906625", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "56cf6f47-4330-4f93-b857-826f2cc81b20"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:04.060768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] validate_networks() for [('56cf6f47-4330-4f93-b857-826f2cc81b20', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:01:04.872740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:01:04.873435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:01:04.873666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:01:04.874076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:01:04.874492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:01:04.895787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:04.896277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:04.896713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:04.897806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:04.898438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:04.899184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:04.915872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:04.924532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:01:04.931748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Getting quotas for user 6adaa12c2ca9469299994300bc856ac5 and project 44a01230c0fc49a6b60d553d085f3808. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:01:04.945754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:01:04.948816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:01:04.993892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d9eb5343-284c-42ed-88f5-ad3ce919a6a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.023945 Oct 15 21:01:04.995131 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1798/2912] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:01:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1025 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:01:04.999879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1777250915"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:05.002594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:05.008731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:05.009493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:05.010079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:05.011553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:05.012022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:05.012542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:05.358673 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4e6d730c-2444-4323-810a-8841d4524c2d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "POST /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72/action" status: 202 len: 0 microversion: 2.1 time: 0.362125 Oct 15 21:01:05.365100 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1799/2913] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:01:04 2022] POST /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72/action => generated 0 bytes in 364 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 21:01:05.367353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:05.368750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:05.427414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:05.430952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:05.432790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:05.434219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:05.438239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:05.439270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:05.798252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:05.807948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:05.810640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:05.812158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e0d577e-17b1-4238-9dd0-a6b271af9e93 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1408 microversion: 2.20 time: 0.450364 Oct 15 21:01:05.825850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1800/2914] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:05 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1408 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:05.837074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:05.837764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:05.844986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:05.845895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:05.846589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:05.847768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:05.848578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:05.849271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:05.854948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:05.979053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:05.994834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:05.996353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:05.997480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16fbb08d-7555-4d17-931c-6cb6593a7a8a tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1575 microversion: 2.42 time: 0.170223 Oct 15 21:01:06.008309 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1801/2915] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:05 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1575 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:06.021127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:06.021738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:06.033924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:06.034766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:06.036378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:06.038530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:06.039031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:06.040641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:06.526359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:06.575880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfcdf4c2-53df-4388-9415-584aefe6525a tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1322 microversion: 2.1 time: 0.547635 Oct 15 21:01:06.579703 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1802/2916] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:06 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1322 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:06.601603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:06.605512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:06.623103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:06.623674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:06.624122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:06.631037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:06.638249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:06.641160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:06.850887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '83d85ef4ebdb452a93f24682e2bbd46c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57ac50a5098942e58c8ab72ff09d5159', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:06.882554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9d7887a-ccff-4fca-aca5-eb7a56976af3 tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 200 len: 1431 microversion: 2.1 time: 0.301639 Oct 15 21:01:06.898500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1803/2917] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:06 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 1431 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:06.906740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:06.907639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:06.925875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:06.928132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:06.931334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:06.938758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:06.939532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:06.942043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:07.611528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:07.622212 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-646f8ec8-18b8-4a84-8475-2dac7ddd8137 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 200 len: 1508 microversion: 2.1 time: 0.722502 Oct 15 21:01:07.645526 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1804/2918] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:06 2022] GET /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 1508 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:07.659072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:07.659695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:07.676839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:07.677365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:07.677830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:07.678516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:07.679158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:07.679593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:07.841400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:07.855297 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-941a7290-7b91-4126-bda5-de9cb84e5726 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1266 microversion: 2.1 time: 0.208338 Oct 15 21:01:07.863071 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1805/2919] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:07 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1266 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:07.876622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "name": "network-vif-plugged", "status": "completed", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}, {"server_uuid": "eebafc32-9d60-4131-92ac-3da43cc822a9", "name": "network-vif-plugged", "status": "completed", "tag": "9af00388-8f08-4aba-b946-41ee5181975e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:07.910124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:07.913238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:07.913798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:07.934130 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] Creating event network-vif-plugged:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:01:07.934932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] Creating event network-vif-plugged:9af00388-8f08-4aba-b946-41ee5181975e for instance eebafc32-9d60-4131-92ac-3da43cc822a9 on nch1-master-1 Oct 15 21:01:07.942282 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-339ffd5e-9b88-4dbc-9ce0-5b8c2c19afc1 req-cc2ab72b-42a7-4aa4-b542-970bb367dae0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.076485 Oct 15 21:01:07.943054 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1806/2920] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:07 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:07.950247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6280439e-9faa-4dae-989e-ed5a26c364dc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe75846e0d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:08.044540 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6280439e-9faa-4dae-989e-ed5a26c364dc tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed" status: 200 len: 1414 microversion: 2.1 time: 0.097103 Oct 15 21:01:08.047033 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1807/2921] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:01:07 2022] GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed => generated 1414 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:08.061419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:08.062542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:08.072882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.074085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.074867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:08.075776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.076448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.077225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:08.557680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:08.568332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:08.577823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:08.591092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79c3e17c-8fbf-436d-980d-df5f80fde231 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1408 microversion: 2.20 time: 0.533007 Oct 15 21:01:08.594365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1808/2922] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:08 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1408 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:08.606858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-137cb066-3185-4d65-9069-4496064919d0 req-b9143127-3f59-4e58-8a42-f27b0120650a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d26d6751-7e16-4e2e-b5d7-fd1761b57031", "name": "network-vif-deleted", "tag": "45798da7-59f5-4021-a379-a96fb37423f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:08.639509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-137cb066-3185-4d65-9069-4496064919d0 req-b9143127-3f59-4e58-8a42-f27b0120650a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.643419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-137cb066-3185-4d65-9069-4496064919d0 req-b9143127-3f59-4e58-8a42-f27b0120650a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.644140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-137cb066-3185-4d65-9069-4496064919d0 req-b9143127-3f59-4e58-8a42-f27b0120650a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:08.677353 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-137cb066-3185-4d65-9069-4496064919d0 req-b9143127-3f59-4e58-8a42-f27b0120650a service nova] Creating event network-vif-deleted:45798da7-59f5-4021-a379-a96fb37423f4 for instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 on nch2-master-1 Oct 15 21:01:08.700011 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-137cb066-3185-4d65-9069-4496064919d0 req-b9143127-3f59-4e58-8a42-f27b0120650a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103770 Oct 15 21:01:08.706788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1809/2923] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:08.715543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:08.716169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:08.754560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.755287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.755925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:08.758221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.758861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.759696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:08.910311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:08.920676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:08.926981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:08.928179 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75b90873-a665-4188-8e49-17de8d87db40 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1593 microversion: 2.42 time: 0.220230 Oct 15 21:01:08.928751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1810/2924] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:08 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1593 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:08.946976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:08.949773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:08.969431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.971731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.973335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:08.975418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:08.977279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:08.977984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:09.612455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:09.622052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6b83090-8af3-4961-a5c3-ce3e8fc611e6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1322 microversion: 2.1 time: 0.692218 Oct 15 21:01:09.627213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1811/2925] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:08 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1322 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:09.632058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:09.641834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: eebafc32-9d60-4131-92ac-3da43cc822a9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:09.654968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:09.655478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:09.659113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:09.662328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:09.663050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:09.663926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:10.347169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:10.362347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4f56e8a8-0ddc-4167-a019-8e5f9433d84e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/eebafc32-9d60-4131-92ac-3da43cc822a9" status: 200 len: 1320 microversion: 2.1 time: 0.733834 Oct 15 21:01:10.363767 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1812/2926] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:09 2022] GET /compute/v2.1/servers/eebafc32-9d60-4131-92ac-3da43cc822a9 => generated 1320 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:10.369210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:10.370719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] [instance: d26d6751-7e16-4e2e-b5d7-fd1761b57031] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:10.377306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:10.378141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:10.378797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:10.379710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:10.380532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:10.381182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:10.398893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] HTTP exception thrown: Instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 could not be found. Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Request method failure captured: Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 HTTP/1.1 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx8PX0-49snCz46mp-NhhQmPmVLgOYnXRDBm5JZolI1wgX3QUCaT9-lCECUUcQru_YYXFhNc9VKPOfClBYhxcvHWxiBB0zyTeNnuzf3YvcHosWw4P2pG2U_a8aeLFvXKayL7urDAFU5iSPj4NlGYI6PUdshLtdi42vyh8Je7UAoRxGE Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 57ac50a5098942e58c8ab72ff09d5159 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57ac50a5098942e58c8ab72ff09d5159"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 57ac50a5098942e58c8ab72ff09d5159 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-1585030782 Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerTestJSON-1585030782-project Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 83d85ef4ebdb452a93f24682e2bbd46c Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerTestJSON-1585030782-project Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 could not be found. Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd26d6751-7e16-4e2e-b5d7-fd1761b57031'} Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 could not be found. Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 could not be found. Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:10.401873 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 could not be found. Oct 15 21:01:10.405992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:10.418070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] Returning 404 to user: Instance d26d6751-7e16-4e2e-b5d7-fd1761b57031 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:10.418947 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80cb3ad8-f43b-4cf4-aa09-e0f3d27b557c tempest-ImagesOneServerTestJSON-1585030782 tempest-ImagesOneServerTestJSON-1585030782-project] 10.222.0.16 "GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031" status: 404 len: 111 microversion: 2.1 time: 0.053415 Oct 15 21:01:10.419520 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1813/2927] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:10 2022] GET /compute/v2.1/servers/d26d6751-7e16-4e2e-b5d7-fd1761b57031 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:10.425610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d2cd1778-5904-4e19-beff-7ef65479cc3c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f14d700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:10.478170 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d2cd1778-5904-4e19-beff-7ef65479cc3c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed" status: 200 len: 1414 microversion: 2.1 time: 0.057217 Oct 15 21:01:10.479043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1814/2928] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:01:10 2022] GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed => generated 1414 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:10.496019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:10.499895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:10.521078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:10.521610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:10.522114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:10.529898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:10.530417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:10.530866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:11.117091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:11.127366 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e15f1c5a-cae3-4f48-bd15-224c0ff3c579 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1308 microversion: 2.1 time: 0.646910 Oct 15 21:01:11.131232 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1815/2929] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:10 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1308 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:11.140711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:11.144456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:11.164324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:11.165462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:11.167567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:11.168549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:11.177010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:11.178001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:12.099315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:12.118895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:12.123218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:12.129856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d8efdf1f-80ca-41cb-b58d-126509e5475d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1600 microversion: 2.20 time: 0.997181 Oct 15 21:01:12.135366 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1816/2930] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:11 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1600 bytes in 1003 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:12.163246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:12.164361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:12.195127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:12.196137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:12.197983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:12.200199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:12.205030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:12.207774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:13.225324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:13.250097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:13.255088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:13.258977 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f4663c5-6214-4d71-ae44-aa353784f57b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1708 microversion: 2.42 time: 1.109704 Oct 15 21:01:13.262701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1817/2931] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:12 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1708 bytes in 1120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:13.269828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-136765f9-297f-4be3-a776-ec2413817b52 req-87da944a-a511-4926-80aa-e4356e5f5449 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c8032d6f-ff10-4add-be09-6de0bbdcf98f", "name": "network-vif-plugged", "status": "completed", "tag": "f4da8690-5405-4369-97d1-4c91b63a3c57"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:13.324940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-136765f9-297f-4be3-a776-ec2413817b52 req-87da944a-a511-4926-80aa-e4356e5f5449 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:13.333721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-136765f9-297f-4be3-a776-ec2413817b52 req-87da944a-a511-4926-80aa-e4356e5f5449 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:13.334667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-136765f9-297f-4be3-a776-ec2413817b52 req-87da944a-a511-4926-80aa-e4356e5f5449 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:13.373025 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-136765f9-297f-4be3-a776-ec2413817b52 req-87da944a-a511-4926-80aa-e4356e5f5449 service nova] Creating event network-vif-plugged:f4da8690-5405-4369-97d1-4c91b63a3c57 for instance c8032d6f-ff10-4add-be09-6de0bbdcf98f on nch2-master-1 Oct 15 21:01:13.387063 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-136765f9-297f-4be3-a776-ec2413817b52 req-87da944a-a511-4926-80aa-e4356e5f5449 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122298 Oct 15 21:01:13.405006 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1818/2932] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:13.426773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5723658f-fc4e-4fdc-871b-114541ad9833 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe758352550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:13.768228 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5723658f-fc4e-4fdc-871b-114541ad9833 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed" status: 204 len: 0 microversion: 2.1 time: 0.354794 Oct 15 21:01:13.770112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1819/2933] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:13 2022] DELETE /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed => generated 0 bytes in 362 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:13.814059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:13.821355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:13.837838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:13.839887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:13.840774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:13.843247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:13.845887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:13.848072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:14.482149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:14.492885 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0494a38-7e8b-4a1d-b224-c9f0f09b05b9 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1322 microversion: 2.1 time: 0.697432 Oct 15 21:01:14.497638 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1820/2934] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:13 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1322 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:14.504387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:14.505403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: eebafc32-9d60-4131-92ac-3da43cc822a9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:14.538098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:14.540172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:14.540810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:14.551522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:14.552780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:14.554129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:14.911620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:14.939825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=11 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:15.451415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:15.473543 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bde66290-39ec-468c-9d42-c83d97e73024 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/eebafc32-9d60-4131-92ac-3da43cc822a9" status: 200 len: 1509 microversion: 2.1 time: 0.971764 Oct 15 21:01:15.476842 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1821/2935] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:14 2022] GET /compute/v2.1/servers/eebafc32-9d60-4131-92ac-3da43cc822a9 => generated 1509 bytes in 979 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:15.506067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:15.507426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:15.516109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:15.517143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:15.517807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:15.518749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:15.519506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:15.520350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:15.871761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:15.883797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d1ef0194-85ca-486f-8943-b41d1f835fd3 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1308 microversion: 2.1 time: 0.385632 Oct 15 21:01:15.888146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1822/2936] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:15 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1308 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:15.913991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:15.914904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:15.928242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:15.930553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:15.934071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:15.935031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:15.935674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:15.938232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:15.941201 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:01:15.941910 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:01:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:01:15.977649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:16.307857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-973af232-cb67-46b8-917b-7657e9616356 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.413573 Oct 15 21:01:16.309208 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:01:16.309577 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:01:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:01:16.328480 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1823/2937] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 21:01:15 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-interface => generated 285 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:01:16.335812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4a4b12a-e9da-4541-8869-540b8c8afb93 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Action: 'create', calling method: .version_select at 0x7fe74f1619d0>, body: {"keypair": {"name": "tempest-keypair-1763522385", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA8BTOukvL2gsa6YnQW7Ofrm9yL/BUpE4HrWSbD7P0uxzbqHxLRpRE2B+EY8itoS74QLMg8wUzKiqoEvkPE8bZ+kr/0FXLVqDHPWcsbfaoVmeS76zCugisBuJ7rocI5JOQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:16.357297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a4a4b12a-e9da-4541-8869-540b8c8afb93 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Getting quotas for project 90ca0b4db2b64900aaa1a4e05dec8a6f. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:01:16.366978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a4a4b12a-e9da-4541-8869-540b8c8afb93 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Getting quotas for user f00da962dc434ae18a1ab29228846688 and project 90ca0b4db2b64900aaa1a4e05dec8a6f. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:01:16.405376 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4a4b12a-e9da-4541-8869-540b8c8afb93 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.32 time: 0.074910 Oct 15 21:01:16.406706 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1824/2938] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 21:01:16 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 77 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 21:01:16.413299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80980047-570c-4422-a339-add16e689f50 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe7584d1dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:16.440917 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-80980047-570c-4422-a339-add16e689f50 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Image not found. Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80980047-570c-4422-a339-add16e689f50 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed HTTP/1.1 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '177923f4-8e9d-468e-bdd3-f1bafce8a2ed'} Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 177923f4-8e9d-468e-bdd3-f1bafce8a2ed Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 21:01:16.445509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 177923f4-8e9d-468e-bdd3-f1bafce8a2ed could not be found. Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:01:16.452212 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:16.458080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80980047-570c-4422-a339-add16e689f50 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:16.461543 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80980047-570c-4422-a339-add16e689f50 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed" status: 404 len: 62 microversion: 2.1 time: 0.051633 Oct 15 21:01:16.462755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1825/2939] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:01:16 2022] GET /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed => generated 62 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:01:16.473874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0e971ced-a70e-4547-8f08-a9c388dc69d1 req-96ef5cdf-1c6f-422f-921e-a80cdff349c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64e679aa-559b-4d18-9d8f-2b69f97d4a2a", "tag": "525185da-02e4-46c2-a615-abe5d1963ad9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:16.489827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e971ced-a70e-4547-8f08-a9c388dc69d1 req-96ef5cdf-1c6f-422f-921e-a80cdff349c0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:16.490706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e971ced-a70e-4547-8f08-a9c388dc69d1 req-96ef5cdf-1c6f-422f-921e-a80cdff349c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:16.491439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e971ced-a70e-4547-8f08-a9c388dc69d1 req-96ef5cdf-1c6f-422f-921e-a80cdff349c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:16.509009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0e971ced-a70e-4547-8f08-a9c388dc69d1 req-96ef5cdf-1c6f-422f-921e-a80cdff349c0 service nova] Creating event network-changed:525185da-02e4-46c2-a615-abe5d1963ad9 for instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a on nch1-master-1 Oct 15 21:01:16.518045 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0e971ced-a70e-4547-8f08-a9c388dc69d1 req-96ef5cdf-1c6f-422f-921e-a80cdff349c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052598 Oct 15 21:01:16.518989 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1826/2940] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:16.525023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:16.538516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:16.556441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:16.557391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:16.558138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:16.559124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:16.559841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:16.560719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:17.927635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:17.954103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:17.963077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:17.971601 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99edea7e-5e8c-41aa-b886-97235059d450 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1743 microversion: 2.42 time: 1.451051 Oct 15 21:01:17.974389 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1827/2941] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:16 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1743 bytes in 1454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:17.985438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4d49dabc-72ef-48a4-9a1f-7656b5d32a2f req-20485cdc-7755-472a-a63b-63d9c0fbfc72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02685830-954d-444e-a85a-55088f3e0e72", "name": "network-vif-plugged", "status": "completed", "tag": "3083b290-a6ed-4c3c-b8fb-0637091c4c38"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:18.015131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4d49dabc-72ef-48a4-9a1f-7656b5d32a2f req-20485cdc-7755-472a-a63b-63d9c0fbfc72 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.015663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4d49dabc-72ef-48a4-9a1f-7656b5d32a2f req-20485cdc-7755-472a-a63b-63d9c0fbfc72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.016168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4d49dabc-72ef-48a4-9a1f-7656b5d32a2f req-20485cdc-7755-472a-a63b-63d9c0fbfc72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:18.045654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4d49dabc-72ef-48a4-9a1f-7656b5d32a2f req-20485cdc-7755-472a-a63b-63d9c0fbfc72 service nova] Creating event network-vif-plugged:3083b290-a6ed-4c3c-b8fb-0637091c4c38 for instance 02685830-954d-444e-a85a-55088f3e0e72 on nch1-master-1 Oct 15 21:01:18.058952 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4d49dabc-72ef-48a4-9a1f-7656b5d32a2f req-20485cdc-7755-472a-a63b-63d9c0fbfc72 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082825 Oct 15 21:01:18.060270 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1828/2942] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:18.073893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:18.074666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: eebafc32-9d60-4131-92ac-3da43cc822a9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:18.092462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.093090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.093641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:18.094492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.095065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.095599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:18.157561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: eebafc32-9d60-4131-92ac-3da43cc822a9] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:01:18.358427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ebb50366-3492-46f1-b8c7-fbd01f55d804 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "DELETE /compute/v2.1/servers/eebafc32-9d60-4131-92ac-3da43cc822a9" status: 204 len: 0 microversion: 2.1 time: 0.295602 Oct 15 21:01:18.359199 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1829/2943] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:18 2022] DELETE /compute/v2.1/servers/eebafc32-9d60-4131-92ac-3da43cc822a9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:18.379327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:18.380166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:18.395215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.395973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.397302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:18.399294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.402761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.403551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:18.929089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:18.941943 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35b86016-8901-46cb-a38c-4bd7462a42b6 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1471 microversion: 2.1 time: 0.576756 Oct 15 21:01:18.942811 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1830/2944] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:18 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1471 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:18.949675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:18.967156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:18.988856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.989602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.990204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:18.991069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:18.993897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:18.996592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:19.922729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:20.029288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8cad7fce-68f8-4ff3-b71d-d45ac723b62b tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1501 microversion: 2.1 time: 1.082614 Oct 15 21:01:20.040659 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1831/2945] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:18 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1501 bytes in 1095 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:20.060220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:20.064876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:20.100482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:20.103181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:20.104489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:20.107145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:20.110110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:20.113857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:21.812975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:21.831726 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-977f2c7a-740f-4fda-b118-4334a505fe72 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1308 microversion: 2.1 time: 1.786178 Oct 15 21:01:21.832534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1832/2946] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:20 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1308 bytes in 1791 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:21.840816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:21.845760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:21.861697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:21.862210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:21.862661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:21.863398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:21.864122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:21.864559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:21.906797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:01:22.044546 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e61216df-da73-4688-a724-8e2ce8a464e7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b" status: 204 len: 0 microversion: 2.1 time: 0.210251 Oct 15 21:01:22.053648 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1833/2947] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:21 2022] DELETE /compute/v2.1/servers/7ba5bfc3-c9bd-4949-a2dd-254a4bba450b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:22.400025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5b22b73-cff4-48e2-9b09-cbad9f29f8a2 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:22.401292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5b22b73-cff4-48e2-9b09-cbad9f29f8a2 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] [instance: d0d9d1ae-d5cb-46d1-b143-fa1f1069170b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:22.412316 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b5b22b73-cff4-48e2-9b09-cbad9f29f8a2 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] HTTP exception thrown: Instance d0d9d1ae-d5cb-46d1-b143-fa1f1069170b could not be found. Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5b22b73-cff4-48e2-9b09-cbad9f29f8a2 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Request method failure captured: Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/d0d9d1ae-d5cb-46d1-b143-fa1f1069170b/action HTTP/1.1 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-AiIDnlOQ8tG7NIktVfdKbc9SkVp70gcbX7QV5sELop672omJSVxJkfqusuJ2uN6aeOP9agzAOF67p5FLUVm3fg78576Pm0MTW3GY5OVeYEKLy_7sHh8MR30q6HiXYbznIG0tnRSNQd8mHMkTyQCwp7DIuhAXGH7VnFI5_JDcudqA Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e0fd11023fb64e60bdeeeb473be1e5ef Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: {"migrate": {}} Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d0d9d1ae-d5cb-46d1-b143-fa1f1069170b could not be found. Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd0d9d1ae-d5cb-46d1-b143-fa1f1069170b', 'body': {'migrate': {}}} Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d0d9d1ae-d5cb-46d1-b143-fa1f1069170b could not be found. Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 47, in _migrate Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id, Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d0d9d1ae-d5cb-46d1-b143-fa1f1069170b could not be found. Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:22.414035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d0d9d1ae-d5cb-46d1-b143-fa1f1069170b could not be found. Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5b22b73-cff4-48e2-9b09-cbad9f29f8a2 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Returning 404 to user: Instance d0d9d1ae-d5cb-46d1-b143-fa1f1069170b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5b22b73-cff4-48e2-9b09-cbad9f29f8a2 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/d0d9d1ae-d5cb-46d1-b143-fa1f1069170b/action" status: 404 len: 111 microversion: 2.1 time: 0.354151 Oct 15 21:01:22.431445 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1834/2948] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:01:22 2022] POST /compute/v2.1/servers/d0d9d1ae-d5cb-46d1-b143-fa1f1069170b/action => generated 111 bytes in 362 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:22.434272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:22.434272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:22.451745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:22.452254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:22.452713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:22.453470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:22.453940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:22.454377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:23.570667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:23.582323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:23.594707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:23.595916 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bebb8e56-b25f-4554-9e2c-24830811d429 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1790 microversion: 2.42 time: 1.170705 Oct 15 21:01:23.609455 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1835/2949] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:22 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1790 bytes in 1192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:23.621428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:23.622041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:23.635658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=4,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:23.641913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:23.642647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:23.644901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:23.647821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:23.649507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:23.651147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:24.226035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:24.242925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35b76b4b-b2ab-47e4-a100-e64ee51bfc4c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1501 microversion: 2.1 time: 0.631999 Oct 15 21:01:24.247534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1836/2950] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:23 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1501 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:24.264656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6d73cf4a-0491-41ea-8f24-6aa21400b657 req-e9f01085-5fa6-4270-a1ea-7dbf9baed079 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64e679aa-559b-4d18-9d8f-2b69f97d4a2a", "name": "network-vif-plugged", "status": "completed", "tag": "525185da-02e4-46c2-a615-abe5d1963ad9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:24.288254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d73cf4a-0491-41ea-8f24-6aa21400b657 req-e9f01085-5fa6-4270-a1ea-7dbf9baed079 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:24.288958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d73cf4a-0491-41ea-8f24-6aa21400b657 req-e9f01085-5fa6-4270-a1ea-7dbf9baed079 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:24.289590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d73cf4a-0491-41ea-8f24-6aa21400b657 req-e9f01085-5fa6-4270-a1ea-7dbf9baed079 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:24.305368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6d73cf4a-0491-41ea-8f24-6aa21400b657 req-e9f01085-5fa6-4270-a1ea-7dbf9baed079 service nova] Creating event network-vif-plugged:525185da-02e4-46c2-a615-abe5d1963ad9 for instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a on nch1-master-1 Oct 15 21:01:24.312241 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6d73cf4a-0491-41ea-8f24-6aa21400b657 req-e9f01085-5fa6-4270-a1ea-7dbf9baed079 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063222 Oct 15 21:01:24.313619 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1837/2951] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:24.320702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateTestJSON-server-525447152", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 1, "max_count": 2, "return_reservation_id": true, "networks": [{"uuid": "d675349d-9eba-4ab3-9302-bd7c72913332"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:24.455759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] validate_networks() for [('d675349d-9eba-4ab3-9302-bd7c72913332', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:01:25.797208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:01:25.798776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:01:25.799888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:01:25.802070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:01:25.802733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:01:25.823721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:25.824875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:25.825576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:25.831825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:25.832556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:25.833137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:25.866060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Getting quotas for project 9bc93093ddf44aada9598a14b5417a2d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:01:25.872868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Getting quotas for user 4e1353ffe42243af82b590bc98e96a42 and project 9bc93093ddf44aada9598a14b5417a2d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:01:25.895852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Going to run 2 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:01:25.898696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:01:25.970188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=fd9910fa-b048-419c-8e20-08c4c1e4e6ac,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:01:26.097325 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3950edec-ddb1-4fcd-bedb-effcbc054a0e tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 1.781637 Oct 15 21:01:26.098281 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1838/2952] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:01:24 2022] POST /compute/v2.1/servers => generated 32 bytes in 1784 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Oct 15 21:01:26.108790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-662857002", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:26.225654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:01:27.190576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:01:27.192045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:01:27.192724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:01:27.193242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:01:27.193719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:01:27.218728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:27.220460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:27.221734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:27.223816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:27.225123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:27.226580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:27.260657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Getting quotas for project e91f96ee17114009a55fe2bf388a7aa7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:01:27.269183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Getting quotas for user 573c02d502744876babbdb4fbb4fb9e6 and project e91f96ee17114009a55fe2bf388a7aa7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:01:27.314118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:01:27.320245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:01:27.432834 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f3be90d-64b1-44fc-a015-4d5616661e30 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.329835 Oct 15 21:01:27.434818 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1839/2953] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:01:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 1336 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:01:27.727395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:27.728835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:27.745369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:27.752492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:27.753316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:27.757748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:27.758578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:27.759182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:28.016182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:28.024216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dae8cbc3-3458-47f3-988e-9114e51a3844 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1490 microversion: 2.1 time: 0.588176 Oct 15 21:01:28.029385 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1840/2954] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:27 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1490 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:28.043512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ef8b29c2-895b-4a9e-b806-d3483d30c1b1 req-50a71692-6145-46a2-a97f-abaec17296f5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:28.063292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef8b29c2-895b-4a9e-b806-d3483d30c1b1 req-50a71692-6145-46a2-a97f-abaec17296f5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:28.063792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef8b29c2-895b-4a9e-b806-d3483d30c1b1 req-50a71692-6145-46a2-a97f-abaec17296f5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:28.064219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ef8b29c2-895b-4a9e-b806-d3483d30c1b1 req-50a71692-6145-46a2-a97f-abaec17296f5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:28.090340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ef8b29c2-895b-4a9e-b806-d3483d30c1b1 req-50a71692-6145-46a2-a97f-abaec17296f5 service nova] Creating event network-changed:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:01:28.104644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ef8b29c2-895b-4a9e-b806-d3483d30c1b1 req-50a71692-6145-46a2-a97f-abaec17296f5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073852 Oct 15 21:01:28.114000 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1841/2955] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:28.121611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-947553264", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "aca1d30f-c9a2-452b-9d71-67ed17d6c09f"}, {"port": "6e28f7f5-2809-45be-9dd9-d05f11e48e6f", "tag": "port-1"}, {"port": "97f99e08-5287-4631-83bc-e4e52565d3e4", "tag": "port-2"}, {"uuid": "f850e581-e948-4ca3-b70c-1ed22cb1c880", "tag": "net-1"}, {"uuid": "f850e581-e948-4ca3-b70c-1ed22cb1c880"}, {"uuid": "4ee70dee-c8f4-4090-aeca-3d7d04b4d53e", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "4ee70dee-c8f4-4090-aeca-3d7d04b4d53e", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "b3c6e585-b0bc-4f79-b8dd-b0ff70c18019", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "b8ecbdfa-373e-4d79-a855-440c96853a59", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "fd2289e9-1128-4cfd-a309-9f8e7a58a1b2", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--961488699"}], "key_name": "tempest-keypair-1763522385", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:28.843005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] validate_networks() for [('aca1d30f-c9a2-452b-9d71-67ed17d6c09f', None, None, None, None, None), (None, None, '6e28f7f5-2809-45be-9dd9-d05f11e48e6f', None, None, None), (None, None, '97f99e08-5287-4631-83bc-e4e52565d3e4', None, None, None), ('f850e581-e948-4ca3-b70c-1ed22cb1c880', None, None, None, None, None), ('f850e581-e948-4ca3-b70c-1ed22cb1c880', None, None, None, None, None), ('4ee70dee-c8f4-4090-aeca-3d7d04b4d53e', '10.2.2.100', None, None, None, None), ('4ee70dee-c8f4-4090-aeca-3d7d04b4d53e', '10.2.2.200', None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:01:37.244401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:37.302828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=16 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:38.009937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:01:38.013410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:01:38.014060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:01:38.014614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:01:38.015045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:01:38.024176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:38.024774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:38.026218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:38.026965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:38.027428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:38.027857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:38.046399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Getting quotas for project 90ca0b4db2b64900aaa1a4e05dec8a6f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:01:38.051718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Getting quotas for user f00da962dc434ae18a1ab29228846688 and project 90ca0b4db2b64900aaa1a4e05dec8a6f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:01:38.222096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:01:38.225082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='b3c6e585-b0bc-4f79-b8dd-b0ff70c18019',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='b8ecbdfa-373e-4d79-a855-440c96853a59',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='fd2289e9-1128-4cfd-a309-9f8e7a58a1b2',volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:01:38.555355 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b5a0668-9f15-4e07-bdbd-a5c2bfdbdca4 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.32 time: 10.439979 Oct 15 21:01:38.557263 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1842/2956] 10.222.0.16 () {62 vars in 1189 bytes} [Sat Oct 15 21:01:28 2022] POST /compute/v2.1/servers => generated 405 bytes in 10443 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:01:38.563132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:38.563782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:38.570946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:38.571506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:38.572001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:38.572706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:38.573203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:38.573645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.236196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:39.247337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:39.248856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:39.250061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6f9e732f-0d81-45cd-b411-8f5c1860acb0 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.691594 Oct 15 21:01:39.250611 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1843/2957] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:38 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:39.255513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:39.256322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:39.263597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.264100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.264782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.265474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.266096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.266689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.404095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5d93a680570f4d23b1f6df534bc082b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62c28804ebd4f1f9ac47aa541500110', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:39.413336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:39.418957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b70da826-eeb5-4191-a456-84555810f5de tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 200 len: 1488 microversion: 2.1 time: 0.167126 Oct 15 21:01:39.419612 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1844/2958] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:39 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 1488 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:39.424027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:39.428751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:39.429544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Searching by: {'deleted': False, 'project_id': '9bc93093ddf44aada9598a14b5417a2d'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:01:39.436460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.437029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.437924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.438652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.439106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.439535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.467620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:01:39.500288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf690137-d801-4e49-874d-4f753ad6a7a0 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 671 microversion: 2.1 time: 0.079476 Oct 15 21:01:39.500728 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1845/2959] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:01:39 2022] GET /compute/v2.1/servers => generated 671 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:39.505325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-dd9de80e-0fcc-4ff9-9dbb-0ae67684e675 req-53f91af0-66a6-40bc-879d-2c9664c7b4cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eebafc32-9d60-4131-92ac-3da43cc822a9", "name": "network-vif-unplugged", "status": "completed", "tag": "9af00388-8f08-4aba-b946-41ee5181975e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:39.515732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd9de80e-0fcc-4ff9-9dbb-0ae67684e675 req-53f91af0-66a6-40bc-879d-2c9664c7b4cf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.516208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd9de80e-0fcc-4ff9-9dbb-0ae67684e675 req-53f91af0-66a6-40bc-879d-2c9664c7b4cf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.516638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd9de80e-0fcc-4ff9-9dbb-0ae67684e675 req-53f91af0-66a6-40bc-879d-2c9664c7b4cf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.527346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-dd9de80e-0fcc-4ff9-9dbb-0ae67684e675 req-53f91af0-66a6-40bc-879d-2c9664c7b4cf service nova] Creating event network-vif-unplugged:9af00388-8f08-4aba-b946-41ee5181975e for instance eebafc32-9d60-4131-92ac-3da43cc822a9 on nch1-master-1 Oct 15 21:01:39.532178 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-dd9de80e-0fcc-4ff9-9dbb-0ae67684e675 req-53f91af0-66a6-40bc-879d-2c9664c7b4cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030302 Oct 15 21:01:39.532929 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1846/2960] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:39 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:39.537763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:39.538386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:39.544902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.545487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.545860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.546640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.547100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.547526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.682097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:39.689764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1820b9eb-d07a-4afe-afef-fd416819b543 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1321 microversion: 2.1 time: 0.155895 Oct 15 21:01:39.690328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1847/2961] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:39 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1321 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:39.694732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-\u00cd\u00f1st\u00e1\u00f1c\u00e9-1836042239"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:39.697958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:39.703672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.704145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.704575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.705351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:39.705805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:39.706227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:39.772124 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46a5fc03-045a-4241-a805-8fc65b95394a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "PUT /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1076 microversion: 2.1 time: 0.080745 Oct 15 21:01:39.772770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1848/2962] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:01:39 2022] PUT /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1076 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:39.829809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f91abdda-7963-4dbe-83a2-9dcbad09ccc7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe758406430>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1286643323", "description": "tempest-description-1844386963"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:39.981689 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f91abdda-7963-4dbe-83a2-9dcbad09ccc7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 256 microversion: 2.1 time: 0.207599 Oct 15 21:01:39.982360 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1849/2963] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:01:39 2022] POST /compute/v2.1/os-security-groups => generated 256 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:39.988220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eebafc32-9d60-4131-92ac-3da43cc822a9", "name": "network-vif-deleted", "tag": "9af00388-8f08-4aba-b946-41ee5181975e"}, {"server_uuid": "7ba5bfc3-c9bd-4949-a2dd-254a4bba450b", "name": "network-vif-deleted", "tag": "b758a233-2c34-4df8-a6d5-229769b08481"}, {"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "142ac24e-4393-42c7-a3ab-d8d47ad61698"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:40.002045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.002595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.003111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.017483 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Creating event network-vif-deleted:9af00388-8f08-4aba-b946-41ee5181975e for instance eebafc32-9d60-4131-92ac-3da43cc822a9 on nch1-master-1 Oct 15 21:01:40.018109 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Creating event network-vif-deleted:b758a233-2c34-4df8-a6d5-229769b08481 for instance 7ba5bfc3-c9bd-4949-a2dd-254a4bba450b on nch2-master-1 Oct 15 21:01:40.018711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] Creating event network-changed:142ac24e-4393-42c7-a3ab-d8d47ad61698 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:40.028935 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-596daac7-e808-4e30-8e1b-a721eac0f464 req-43aa728d-1513-4622-8c35-02e9e57baef3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 521 microversion: 2.1 time: 0.045007 Oct 15 21:01:40.029426 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1850/2964] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:39 2022] POST /compute/v2.1/os-server-external-events => generated 521 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:40.034753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:40.035452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:40.041959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.042454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.042902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.043752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.044224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.044783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.112160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:40.122597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:40.124316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:40.125749 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ee28464-1018-42f0-b209-2e2dd53e95df tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1575 microversion: 2.32 time: 0.095237 Oct 15 21:01:40.126336 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1851/2965] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:40 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1575 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:40.131125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:40.131632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:40.137677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.138167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.138612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.139348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.139769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.140200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.165365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:01:40.244806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a8412c5-7767-44f2-b62e-52611f1ac7f7 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 204 len: 0 microversion: 2.1 time: 0.117083 Oct 15 21:01:40.245410 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1852/2966] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:40 2022] DELETE /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:40.250397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:40.251220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:40.256782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.257307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.257744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.258433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.258894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.259343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.397012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:40.406470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f7f9baf-84a4-4d93-b7e5-1ab2636b931d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6" status: 200 len: 1317 microversion: 2.1 time: 0.159847 Oct 15 21:01:40.407352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1853/2967] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:40 2022] GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 => generated 1317 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:40.412056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:40.412668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:40.419377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.419863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.421730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.422441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:40.422905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:40.423331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:40.736450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:40.744747 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ce14b87-fd85-4a4c-a24c-8e4b3c2d535c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1502 microversion: 2.1 time: 0.336295 Oct 15 21:01:40.745362 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1854/2968] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:40 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1502 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:40.749862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89314893-d26e-4cb7-9567-b033f935ddb0 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ec00430>, body: {"security_group_rule": {"parent_group_id": "bd188229-5236-40ac-a0c7-33c0ad61f297", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:41.539118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-89314893-d26e-4cb7-9567-b033f935ddb0 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.791843 Oct 15 21:01:41.539776 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1855/2969] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:01:40 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:41.545380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2ffb48b-9e9d-4970-9fda-1a4c59e8d8ed tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe75846ed30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:41.568391 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c2ffb48b-9e9d-4970-9fda-1a4c59e8d8ed tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Image not found. Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2ffb48b-9e9d-4970-9fda-1a4c59e8d8ed tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed HTTP/1.1 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '177923f4-8e9d-468e-bdd3-f1bafce8a2ed'} Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 177923f4-8e9d-468e-bdd3-f1bafce8a2ed to delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 177923f4-8e9d-468e-bdd3-f1bafce8a2ed could not be found. Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.570310 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2ffb48b-9e9d-4970-9fda-1a4c59e8d8ed tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2ffb48b-9e9d-4970-9fda-1a4c59e8d8ed tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed" status: 404 len: 62 microversion: 2.1 time: 0.030706 Oct 15 21:01:41.576775 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1856/2970] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:41 2022] DELETE /compute/v2.1/images/177923f4-8e9d-468e-bdd3-f1bafce8a2ed => generated 62 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:01:41.582017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:41.582641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:41.589184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:41.589680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:41.590128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:41.590836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:41.591294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:41.591724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:41.945199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:41.955638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:41.957114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:41.958580 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47fd6b2a-de86-4ba3-bce1-a6cb52c5b68c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.381423 Oct 15 21:01:41.959134 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1857/2971] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:41 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:41.979104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:41.979818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:41.987042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:41.987657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:41.988196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:41.989351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:41.989975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:41.990399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:42.457666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:42.468132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4cc952f9-fb4f-443d-b432-bf363e2d0d88 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1321 microversion: 2.1 time: 0.492735 Oct 15 21:01:42.469366 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1858/2972] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:41 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1321 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:42.479022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:42.479975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:42.489821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:42.490320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:42.490767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:42.491489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:42.492214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:42.492656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.187166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:43.203466 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51e53629-a815-4e24-9b8a-fb33912dc6d0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1502 microversion: 2.1 time: 0.731869 Oct 15 21:01:43.217076 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1859/2973] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:42 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1502 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:43.217076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:43.217076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:43.225708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.225708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.225708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.225708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.225708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.225708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.337763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:43.349961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:43.352276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:43.353527 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47186115-c03d-49f0-990a-b534ad4d8687 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1593 microversion: 2.32 time: 0.146011 Oct 15 21:01:43.355214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1860/2974] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:43 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1593 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:43.359956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:43.360566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:43.367304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.368001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.369271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.370320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.370784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.371213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.695087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:43.704717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e2f9ae3a-57e8-44e2-90eb-b651acbcf0cc tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6" status: 200 len: 1317 microversion: 2.1 time: 0.348298 Oct 15 21:01:43.705333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1861/2975] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:43 2022] GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 => generated 1317 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:43.711131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d5f9b8b9-4854-4a78-8086-26dca77adca0 req-23892559-f31c-4869-be44-3fc9a5a93d72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02685830-954d-444e-a85a-55088f3e0e72", "name": "network-vif-unplugged", "status": "completed", "tag": "3083b290-a6ed-4c3c-b8fb-0637091c4c38"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:43.722850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=12,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:43.724990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d5f9b8b9-4854-4a78-8086-26dca77adca0 req-23892559-f31c-4869-be44-3fc9a5a93d72 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.725508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d5f9b8b9-4854-4a78-8086-26dca77adca0 req-23892559-f31c-4869-be44-3fc9a5a93d72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.726061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d5f9b8b9-4854-4a78-8086-26dca77adca0 req-23892559-f31c-4869-be44-3fc9a5a93d72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.746122 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d5f9b8b9-4854-4a78-8086-26dca77adca0 req-23892559-f31c-4869-be44-3fc9a5a93d72 service nova] Creating event network-vif-unplugged:3083b290-a6ed-4c3c-b8fb-0637091c4c38 for instance 02685830-954d-444e-a85a-55088f3e0e72 on nch1-master-1 Oct 15 21:01:43.753705 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d5f9b8b9-4854-4a78-8086-26dca77adca0 req-23892559-f31c-4869-be44-3fc9a5a93d72 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046626 Oct 15 21:01:43.754670 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1862/2976] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:43 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:43.761352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4db428db-fc3c-4635-bd46-d246a82cc6b7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ed6d3a0>, body: {"security_group_rule": {"parent_group_id": "bd188229-5236-40ac-a0c7-33c0ad61f297", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:43.895425 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4db428db-fc3c-4635-bd46-d246a82cc6b7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: This rule already exists in group bd188229-5236-40ac-a0c7-33c0ad61f297 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4db428db-fc3c-4635-bd46-d246a82cc6b7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 138 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "bd188229-5236-40ac-a0c7-33c0ad61f297", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: exception: This rule already exists in group bd188229-5236-40ac-a0c7-33c0ad61f297 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'bd188229-5236-40ac-a0c7-33c0ad61f297', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 324, in create Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group_api.create_security_group_rule( Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 186, in create_security_group_rule Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.Invalid(msg) Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: This rule already exists in group bd188229-5236-40ac-a0c7-33c0ad61f297 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 327, in create Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: This rule already exists in group bd188229-5236-40ac-a0c7-33c0ad61f297 Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:43.897376 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: This rule already exists in group bd188229-5236-40ac-a0c7-33c0ad61f297 Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4db428db-fc3c-4635-bd46-d246a82cc6b7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 400 to user: This rule already exists in group bd188229-5236-40ac-a0c7-33c0ad61f297 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4db428db-fc3c-4635-bd46-d246a82cc6b7 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 114 microversion: 2.1 time: 0.142838 Oct 15 21:01:43.906132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1863/2977] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:01:43 2022] POST /compute/v2.1/os-security-group-rules => generated 114 bytes in 149 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:01:43.910425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:43.911075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:43.920357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.920856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.921328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.922177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:43.922642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:43.923070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:43.958655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:01:43.969633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:44.111898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2d69b3e-de44-4f93-8ae5-4afa6198fda4 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 204 len: 0 microversion: 2.1 time: 0.205336 Oct 15 21:01:44.116813 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1864/2978] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:43 2022] DELETE /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:44.128812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "d1754b9c-eda8-41d8-99a6-7cc5c3b7c3eb"}, {"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "d5ed5624-9ab3-4ee7-8a25-54a7b3d3bdef"}, {"name": "network-changed", "server_uuid": "dfaae26e-0dc7-4c4e-8219-cb78c44d5070", "tag": "4b8860d7-6686-44ce-b530-e8b8214dff80"}, {"name": "network-changed", "server_uuid": "5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6", "tag": "18e73793-968c-429b-bc0b-c9b535576589"}, {"name": "network-changed", "server_uuid": "fd9910fa-b048-419c-8e20-08c4c1e4e6ac", "tag": "9dcbb0e4-cd91-4589-ba63-84c1689a5346"}, {"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "fd2bd5a8-8477-474b-8090-dc55b657fa33"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:44.160134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:44.160824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:44.161416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:44.211493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Creating event network-changed:d1754b9c-eda8-41d8-99a6-7cc5c3b7c3eb for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:44.212154 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Creating event network-changed:d5ed5624-9ab3-4ee7-8a25-54a7b3d3bdef for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:44.212915 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Creating event network-changed:4b8860d7-6686-44ce-b530-e8b8214dff80 for instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 on nch2-master-1 Oct 15 21:01:44.215054 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Creating event network-changed:18e73793-968c-429b-bc0b-c9b535576589 for instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 on nch1-master-1 Oct 15 21:01:44.216675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Creating event network-changed:9dcbb0e4-cd91-4589-ba63-84c1689a5346 for instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac on nch2-master-1 Oct 15 21:01:44.216675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] Creating event network-changed:fd2bd5a8-8477-474b-8090-dc55b657fa33 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:44.228922 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-adb6ed04-e3fb-498f-840c-3c77ea43ee91 req-d5dbb5c4-ff66-4971-b2bb-1e012ef7d698 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 1014 microversion: 2.1 time: 0.109748 Oct 15 21:01:44.230069 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1865/2979] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:44 2022] POST /compute/v2.1/os-server-external-events => generated 1014 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:44.235847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:44.236705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:44.247961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:44.249695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:44.250272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:44.251804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:44.253660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:44.255529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:45.023054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:45.039408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:45.042871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:45.045253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8dfef70e-3785-421c-980d-6a299f2feb04 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.813756 Oct 15 21:01:45.046498 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1866/2980] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:44 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 816 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:45.051754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-207cdc10-73d3-4438-abc0-1277ff48b3d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:45.060364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-207cdc10-73d3-4438-abc0-1277ff48b3d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:45.061212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-207cdc10-73d3-4438-abc0-1277ff48b3d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:45.061887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-207cdc10-73d3-4438-abc0-1277ff48b3d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:45.091859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-207cdc10-73d3-4438-abc0-1277ff48b3d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:01:45.148358 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-207cdc10-73d3-4438-abc0-1277ff48b3d7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "POST /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a/action" status: 202 len: 0 microversion: 2.1 time: 0.100325 Oct 15 21:01:45.153311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1867/2981] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:01:45 2022] POST /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a/action => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:01:45.159615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:45.160315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:45.167615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:45.168351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:45.168962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:45.170002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:45.170835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:45.171428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:45.782255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:45.793175 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b457a9c-03aa-4c37-ab68-a9ddaa36a77f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1321 microversion: 2.1 time: 0.638465 Oct 15 21:01:45.793885 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1868/2982] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:45 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1321 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:45.807427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e81ed355-53fb-4229-b16b-70da7989bb6f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe74f12d9d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:46.988228 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e81ed355-53fb-4229-b16b-70da7989bb6f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-group-rules/c2e58bd7-cba5-464e-9ef5-33910418dc1d" status: 202 len: 0 microversion: 2.1 time: 1.193164 Oct 15 21:01:46.988875 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1869/2983] 10.222.0.16 () {58 vars in 1287 bytes} [Sat Oct 15 21:01:45 2022] DELETE /compute/v2.1/os-security-group-rules/c2e58bd7-cba5-464e-9ef5-33910418dc1d => generated 0 bytes in 1194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:01:46.996683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:46.997345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 02685830-954d-444e-a85a-55088f3e0e72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:47.005993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:47.006493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:47.006935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:47.007645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:47.008102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:47.008663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:47.029252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 02685830-954d-444e-a85a-55088f3e0e72 could not be found. Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 HTTP/1.1 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 02685830-954d-444e-a85a-55088f3e0e72 could not be found. Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '02685830-954d-444e-a85a-55088f3e0e72'} Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 02685830-954d-444e-a85a-55088f3e0e72 could not be found. Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 02685830-954d-444e-a85a-55088f3e0e72 could not be found. Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:47.031166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 02685830-954d-444e-a85a-55088f3e0e72 could not be found. Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 02685830-954d-444e-a85a-55088f3e0e72 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4374d62-a335-4c23-acdf-508b8c883f09 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72" status: 404 len: 111 microversion: 2.1 time: 0.042561 Oct 15 21:01:47.036904 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1870/2984] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:46 2022] GET /compute/v2.1/servers/02685830-954d-444e-a85a-55088f3e0e72 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:47.043385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:47.044062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:47.051325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:47.052326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:47.053372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:47.054904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:47.056544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:47.057392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:47.514206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:47.525187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:47.526780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:47.527959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e22aafb7-3395-4db3-a0f6-2ae9ee3cf00e tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1648 microversion: 2.32 time: 0.493842 Oct 15 21:01:47.541070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1871/2985] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:47 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1648 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:47.541070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:47.541070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:47.552400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:47.554167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:47.554615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:47.555339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:47.557128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:47.557571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:48.051614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:48.059290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:48.061737 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a24318d6-2531-4417-b087-67fa8964a301 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6" status: 200 len: 1317 microversion: 2.1 time: 0.531751 Oct 15 21:01:48.062324 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1872/2986] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:47 2022] GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 => generated 1317 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:48.066819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:48.067437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:48.074548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:48.075046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:48.075556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:48.076256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:48.076735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:48.077183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:48.783652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:48.791861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8797bb32-4842-48c0-8a10-6a7777090f06 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1512 microversion: 2.1 time: 0.728520 Oct 15 21:01:48.792426 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1873/2987] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:48 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1512 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:48.796935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6", "name": "network-vif-plugged", "status": "completed", "tag": "18e73793-968c-429b-bc0b-c9b535576589"}, {"server_uuid": "dfaae26e-0dc7-4c4e-8219-cb78c44d5070", "name": "network-vif-plugged", "status": "completed", "tag": "4b8860d7-6686-44ce-b530-e8b8214dff80"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:48.807774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:48.808260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:48.808692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:48.821462 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] Creating event network-vif-plugged:18e73793-968c-429b-bc0b-c9b535576589 for instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 on nch1-master-1 Oct 15 21:01:48.822774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] Creating event network-vif-plugged:4b8860d7-6686-44ce-b530-e8b8214dff80 for instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 on nch2-master-1 Oct 15 21:01:48.830474 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b6524c4a-a44c-4243-ad52-828fe10c4860 req-25de9ddf-c12c-4ec2-b250-da5f0ded89c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.036891 Oct 15 21:01:48.831147 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1874/2988] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:48 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:48.837904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:48.838612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:48.845369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:48.845862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:48.846304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:48.847012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:48.847489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:48.847899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:49.452027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:49.524927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:49.534386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:49.535762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:49.536916 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d5c0f86-51a8-41ff-ae40-726eb6f58e96 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.704694 Oct 15 21:01:49.537509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1875/2989] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:48 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 706 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:49.542668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "a3ed0c87-6f02-402a-a80b-a011756b969c"}, {"server_uuid": "02685830-954d-444e-a85a-55088f3e0e72", "name": "network-vif-deleted", "tag": "3083b290-a6ed-4c3c-b8fb-0637091c4c38"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:49.552714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:49.553235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:49.553671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:49.566799 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] Creating event network-changed:a3ed0c87-6f02-402a-a80b-a011756b969c for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:49.567552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] Creating event network-vif-deleted:3083b290-a6ed-4c3c-b8fb-0637091c4c38 for instance 02685830-954d-444e-a85a-55088f3e0e72 on nch1-master-1 Oct 15 21:01:49.576250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2763ebcb-782f-4f81-91e1-bca2fc141b4f req-4e4e541b-2993-4173-aa64-293edceca1aa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.037172 Oct 15 21:01:49.576844 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1876/2990] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:49 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:49.583031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:49.583734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:49.590350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:49.590834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:49.591278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:49.592029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:49.592433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:49.592965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:50.031167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:50.040869 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-12aa54e0-34be-4cf2-bc6f-f2380ee22956 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1321 microversion: 2.1 time: 0.462824 Oct 15 21:01:50.041798 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1877/2991] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:49 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1321 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:50.047632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec02fb6b-5f8b-42e2-8c6c-28f0112b081e tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74f1b0040>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1440203721", "description": "tempest-description-43326295"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:50.458573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec02fb6b-5f8b-42e2-8c6c-28f0112b081e tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 254 microversion: 2.1 time: 0.415090 Oct 15 21:01:50.460507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1878/2992] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:01:50 2022] POST /compute/v2.1/os-security-groups => generated 254 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:50.470226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:50.470955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:50.491012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:50.491533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:50.491983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:50.492699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:50.493378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:50.494037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:50.508288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 HTTP/1.1 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3fba876c-4a9f-4ad3-b241-45b22a885319'} Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:50.511015 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:50.514904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:50.516990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:50.518106 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26f6e165-fab2-43e6-bc3e-3ffb36b01a1c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 404 len: 111 microversion: 2.1 time: 0.056075 Oct 15 21:01:50.518935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1879/2993] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:50 2022] DELETE /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:50.524089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:50.525327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:50.531868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:50.532453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:50.532963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:50.533762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:50.534291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:50.534795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:50.565206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6607e1ad-b84e-40b5-ad8a-01e00dfddebd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/ips" status: 200 len: 152 microversion: 2.20 time: 0.044864 Oct 15 21:01:50.569211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1880/2994] 10.222.0.16 () {60 vars in 1289 bytes} [Sat Oct 15 21:01:50 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/ips => generated 152 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:01:50.570641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:50.571263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:50.583325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:50.584376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:50.584833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:50.585932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:50.586411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:50.586850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:51.194466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:51.199597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:51.201056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:51.202397 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-889e61aa-961a-4699-9a3e-b509eeb0c129 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1740 microversion: 2.32 time: 0.635149 Oct 15 21:01:51.202956 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1881/2995] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:50 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1740 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:51.208596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:51.209236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:51.215797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:51.216500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:51.216935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:51.217672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:51.218129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:51.218551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:51.953529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:51.961617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-91d44c30-4d2f-401a-a0ef-500b735462ad tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6" status: 200 len: 1505 microversion: 2.1 time: 0.757251 Oct 15 21:01:51.962411 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1882/2996] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:51 2022] GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 => generated 1505 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:51.967132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:51.967887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:51.973947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:51.974658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:51.975285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:51.976147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:51.976608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:51.977015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:52.410910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:52.419583 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f590a19-cff5-408e-a634-bd036825ce94 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1512 microversion: 2.1 time: 0.455975 Oct 15 21:01:52.420165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1883/2997] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:51 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1512 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:52.439791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d16d76c3-64cd-43c0-b703-8ca6a83d59ad tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74f14d040>, body: {"security_group_rule": {"parent_group_id": "82b07114-cbd9-48cc-9278-7b5b7b24a702", "ip_protocol": "tcp", "from_port": 834496416, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:52.638304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d16d76c3-64cd-43c0-b703-8ca6a83d59ad tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Invalid port range 834496416:22. Former value cannot be greater than the later Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d16d76c3-64cd-43c0-b703-8ca6a83d59ad tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 145 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "82b07114-cbd9-48cc-9278-7b5b7b24a702", "ip_protocol": "tcp", "from_port": 834496416, "to_port": 22}} Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid port range 834496416:22. Former value cannot be greater than the later Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': '82b07114-cbd9-48cc-9278-7b5b7b24a702', 'ip_protocol': 'tcp', 'from_port': 834496416, 'to_port': 22}}} Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 150, in _new_ingress_rule Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 834496416:22. Former value cannot be greater than the later Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 834496416:22. Former value cannot be greater than the later Oct 15 21:01:52.639863 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 834496416:22. Former value cannot be greater than the later Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d16d76c3-64cd-43c0-b703-8ca6a83d59ad tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 400 to user: Invalid port range 834496416:22. Former value cannot be greater than the later {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d16d76c3-64cd-43c0-b703-8ca6a83d59ad tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 122 microversion: 2.1 time: 0.207331 Oct 15 21:01:52.644123 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1884/2998] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:01:52 2022] POST /compute/v2.1/os-security-group-rules => generated 122 bytes in 222 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:01:52.647973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:52.648184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 3fba876c-4a9f-4ad3-b241-45b22a885319] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:52.656270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:52.656854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:52.657320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:52.658036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:52.658600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:52.659039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:52.672905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 HTTP/1.1 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3fba876c-4a9f-4ad3-b241-45b22a885319'} Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.674507 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 3fba876c-4a9f-4ad3-b241-45b22a885319 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-42812f80-25be-4cc4-a5b4-716c24bca908 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319" status: 404 len: 111 microversion: 2.1 time: 0.032058 Oct 15 21:01:52.682869 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1885/2999] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:52 2022] GET /compute/v2.1/servers/3fba876c-4a9f-4ad3-b241-45b22a885319 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:52.683548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:52.683806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:52.690727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:52.690992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:52.691437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:52.692156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:52.692623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:52.693096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:52.698393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:53.132118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:53.142168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:53.143644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:53.144818 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c9ce6e7-ebdf-46e8-901f-b4903c92ca97 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.466973 Oct 15 21:01:53.145398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1886/3000] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:52 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:53.149827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ad0bd7b4-1ab6-4108-b811-f1576c66e468 req-e7ffc373-6de4-42dd-9673-d0cb3dddc529 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd9910fa-b048-419c-8e20-08c4c1e4e6ac", "name": "network-vif-plugged", "status": "completed", "tag": "9dcbb0e4-cd91-4589-ba63-84c1689a5346"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:53.159127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad0bd7b4-1ab6-4108-b811-f1576c66e468 req-e7ffc373-6de4-42dd-9673-d0cb3dddc529 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.159600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad0bd7b4-1ab6-4108-b811-f1576c66e468 req-e7ffc373-6de4-42dd-9673-d0cb3dddc529 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.160211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad0bd7b4-1ab6-4108-b811-f1576c66e468 req-e7ffc373-6de4-42dd-9673-d0cb3dddc529 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.172331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ad0bd7b4-1ab6-4108-b811-f1576c66e468 req-e7ffc373-6de4-42dd-9673-d0cb3dddc529 service nova] Creating event network-vif-plugged:9dcbb0e4-cd91-4589-ba63-84c1689a5346 for instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac on nch2-master-1 Oct 15 21:01:53.177346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ad0bd7b4-1ab6-4108-b811-f1576c66e468 req-e7ffc373-6de4-42dd-9673-d0cb3dddc529 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030735 Oct 15 21:01:53.178114 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1887/3001] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:53.183703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:53.184444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:53.197203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.197688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.198782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.199403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.200056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.200759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.564173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:53.572287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2255269-2b30-4af1-ac5f-44b7085f48f2 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1471 microversion: 2.1 time: 0.393048 Oct 15 21:01:53.572861 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1888/3002] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:53 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1471 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:53.577219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "79d61df5-38a3-4192-a9e3-ed2c945abba0"}, {"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "77b4ac77-9b8d-4c4f-a944-9e3a506c9e67"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:53.586573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.587544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.587975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.599556 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] Creating event network-changed:79d61df5-38a3-4192-a9e3-ed2c945abba0 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:53.600070 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] Creating event network-changed:77b4ac77-9b8d-4c4f-a944-9e3a506c9e67 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:01:53.605881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-805acfad-5da3-4d0d-bcea-f9a797f73969 req-f58843e8-dd26-44bb-86b4-3780b469ff3d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.031873 Oct 15 21:01:53.606475 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1889/3003] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:53 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:53.611388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:53.612076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:53.618333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.618811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.619246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.619931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.620385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.620808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.852304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:53.861703 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2332c908-032f-41ec-88e1-819d4d8cf5e2 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac" status: 200 len: 1317 microversion: 2.1 time: 0.254036 Oct 15 21:01:53.862285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1890/3004] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:53 2022] GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac => generated 1317 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:53.867157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:53.867761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:53.874020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.874518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.874951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:53.875645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:53.876102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:53.876527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:54.150031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:54.172210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:54.174679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:54.176128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d776a23-1211-4f43-9191-86efb61ec244 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1786 microversion: 2.32 time: 0.312895 Oct 15 21:01:54.176808 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1891/3005] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:53 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1786 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:54.194288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:54.195507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:54.203840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:54.204532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:54.204981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:54.205758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:54.206419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:54.206862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:54.235070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lazy-loading 'flavor' on Instance uuid 8dc308ad-212c-4df0-baa6-f446dac1d979 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:01:54.449577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f695b47-1fb8-449c-af86-211943138ad6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/action" status: 202 len: 0 microversion: 2.20 time: 0.271205 Oct 15 21:01:54.450371 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1892/3006] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 21:01:54 2022] POST /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/action => generated 0 bytes in 273 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:54.467201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7077dd3-a973-4f80-aa78-0dbaffe79a23 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ebc0b80>, body: {"security_group_rule": {"parent_group_id": "tempest-non_int_id-1360098806", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:54.469199 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e7077dd3-a973-4f80-aa78-0dbaffe79a23 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Security group id should be uuid Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7077dd3-a973-4f80-aa78-0dbaffe79a23 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 131 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "tempest-non_int_id-1360098806", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group id should be uuid Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'tempest-non_int_id-1360098806', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 285, in create Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi parent_group_id = security_group_api.validate_id( Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 51, in validate_id Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.Invalid(msg) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security group id should be uuid Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security group id should be uuid Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:54.472465 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:54.479441 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group id should be uuid Oct 15 21:01:54.479441 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.479441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7077dd3-a973-4f80-aa78-0dbaffe79a23 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 400 to user: Security group id should be uuid {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:54.479441 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e7077dd3-a973-4f80-aa78-0dbaffe79a23 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 76 microversion: 2.1 time: 0.022363 Oct 15 21:01:54.484873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1893/3007] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:01:54 2022] POST /compute/v2.1/os-security-group-rules => generated 76 bytes in 34 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:54.490793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6063afea-60db-42ed-ab2d-b99fe693c412 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ebcf040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:54.516095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6063afea-60db-42ed-ab2d-b99fe693c412 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Image not found. Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6063afea-60db-42ed-ab2d-b99fe693c412 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d HTTP/1.1 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b7725f7d-013b-4e75-8927-10802f221d8d'} Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image b7725f7d-013b-4e75-8927-10802f221d8d to delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image b7725f7d-013b-4e75-8927-10802f221d8d could not be found. Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.517889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:01:54.522741 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:54.526133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6063afea-60db-42ed-ab2d-b99fe693c412 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:54.527015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6063afea-60db-42ed-ab2d-b99fe693c412 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d" status: 404 len: 62 microversion: 2.1 time: 0.041054 Oct 15 21:01:54.527585 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1894/3008] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:54 2022] DELETE /compute/v2.1/images/b7725f7d-013b-4e75-8927-10802f221d8d => generated 62 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:01:54.532166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:54.532894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:54.539481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:54.539973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:54.540827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:54.541629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:54.542089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:54.542520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.049909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.062983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a191692a-4624-492b-aac3-34a1c7fcdddc tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1512 microversion: 2.1 time: 0.534055 Oct 15 21:01:55.063718 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1895/3009] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:54 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1512 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:55.068623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:55.069269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:55.077428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:55.077955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:55.078467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.079325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:55.080118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:55.080723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.107968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lazy-loading 'flavor' on Instance uuid dfaae26e-0dc7-4c4e-8219-cb78c44d5070 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:01:55.199354 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a921bf9-a3cf-4360-918b-c5be164cbdf8 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070/action" status: 202 len: 0 microversion: 2.1 time: 0.133942 Oct 15 21:01:55.200057 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1896/3010] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:01:55 2022] POST /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070/action => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:01:55.206687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:55.207401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:55.214576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:55.215150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:55.215561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.216278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:55.216739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:55.217220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.524841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.534908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.536444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.537762 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6441c2f8-c35a-4d7b-bebb-1548980ffc03 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.336314 Oct 15 21:01:55.538402 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1897/3011] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:55 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:55.543439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:55.544041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:55.549870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:55.550568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:55.551065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.551765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:55.552221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:55.552646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:55.877380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.886589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.888655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:55.889883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f75dfdba-2c43-4b2a-8459-70c0f390b610 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1725 microversion: 2.20 time: 0.350555 Oct 15 21:01:55.890594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1898/3012] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:55 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1725 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:55.895025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35d38f77-cc2f-4ede-ab9a-f97390a01d16 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74f1b0670>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1508882401", "description": "tempest-description-1383254929"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:56.456627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35d38f77-cc2f-4ede-ab9a-f97390a01d16 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 256 microversion: 2.1 time: 0.565078 Oct 15 21:01:56.457267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1899/3013] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:01:55 2022] POST /compute/v2.1/os-security-groups => generated 256 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:56.465295 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:01:56.466046 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:01:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:01:56.475154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:56.475657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:56.483010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:56.483720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:56.484874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:56.485107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:56.486696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:56.487153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:56.502144 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f HTTP/1.1 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '359b81c9-e8fc-491e-95d7-b2887dfb032f'} Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:56.504182 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4ada3ff8-a6b2-4f33-8288-b339d90bcc04 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 404 len: 111 microversion: 2.1 time: 0.038199 Oct 15 21:01:56.508256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1900/3014] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:56 2022] DELETE /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:56.511530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:56.512144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:56.517964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:56.518448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:56.518981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:56.519757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:56.520222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:56.520642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:56.841823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:56.849738 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-798c4fe4-6821-4304-b436-53ca13e5865d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac" status: 200 len: 1506 microversion: 2.1 time: 0.342307 Oct 15 21:01:56.850309 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1901/3015] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:56 2022] GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac => generated 1506 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:56.854596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:56.855198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:56.861356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:56.861850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:56.862965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:56.863689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:56.864412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:56.864833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:57.465443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-68553558-3d30-4643-b975-bebb25edb27e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1613 microversion: 2.1 time: 0.614016 Oct 15 21:01:57.478745 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1902/3016] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:56 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1613 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:57.487960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:57.488671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:57.495608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:57.496098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:57.496544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:57.497290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:57.497756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:57.498187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.197844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:58.207659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:58.209152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:58.210316 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d9024ad-613e-4ff5-acb4-5247c3e0b21c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1853 microversion: 2.32 time: 0.730499 Oct 15 21:01:58.210875 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1903/3017] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:57 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1853 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:58.215943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-507bb81f-008e-467d-89ee-d2bd7383c689 req-28fc0a1c-5e26-40e8-b872-1128d449dd5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "675ba1c4-0b55-4936-9005-97673bfc6b22"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:58.225353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-507bb81f-008e-467d-89ee-d2bd7383c689 req-28fc0a1c-5e26-40e8-b872-1128d449dd5f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.225845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-507bb81f-008e-467d-89ee-d2bd7383c689 req-28fc0a1c-5e26-40e8-b872-1128d449dd5f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.226285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-507bb81f-008e-467d-89ee-d2bd7383c689 req-28fc0a1c-5e26-40e8-b872-1128d449dd5f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.230982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:01:58.240273 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-507bb81f-008e-467d-89ee-d2bd7383c689 req-28fc0a1c-5e26-40e8-b872-1128d449dd5f service nova] Creating event network-changed:675ba1c4-0b55-4936-9005-97673bfc6b22 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:01:58.246379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-507bb81f-008e-467d-89ee-d2bd7383c689 req-28fc0a1c-5e26-40e8-b872-1128d449dd5f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034287 Oct 15 21:01:58.247074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1904/3018] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:01:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:01:58.252390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:58.253120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:58.260184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.261597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.262094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.262998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.263659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.264269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.708320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:58.717002 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d1e1a0f-d004-4456-b231-c40808a1b27a tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1512 microversion: 2.1 time: 0.468805 Oct 15 21:01:58.717588 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1905/3019] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:58 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1512 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:01:58.721999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9895077e-9a58-48c4-979f-1d3da8215214 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74f1b0820>, body: {"security_group_rule": {"parent_group_id": "ba1d872f-43b1-4746-a846-04ba8efc9c3f", "ip_protocol": "tempest-999-1781121627", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:58.900026 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9895077e-9a58-48c4-979f-1d3da8215214 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Invalid IP protocol tempest-999-1781121627. Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9895077e-9a58-48c4-979f-1d3da8215214 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 157 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "ba1d872f-43b1-4746-a846-04ba8efc9c3f", "ip_protocol": "tempest-999-1781121627", "from_port": 22, "to_port": 22}} Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid IP protocol tempest-999-1781121627. Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'ba1d872f-43b1-4746-a846-04ba8efc9c3f', 'ip_protocol': 'tempest-999-1781121627', 'from_port': 22, 'to_port': 22}}} Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 144, in _new_ingress_rule Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidIpProtocol(protocol=ip_protocol) Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidIpProtocol: Invalid IP protocol tempest-999-1781121627. Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid IP protocol tempest-999-1781121627. Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.901742 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid IP protocol tempest-999-1781121627. Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9895077e-9a58-48c4-979f-1d3da8215214 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 400 to user: Invalid IP protocol tempest-999-1781121627. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9895077e-9a58-48c4-979f-1d3da8215214 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 87 microversion: 2.1 time: 0.184668 Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1906/3020] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:01:58 2022] POST /compute/v2.1/os-security-group-rules => generated 87 bytes in 186 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:58.912076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 359b81c9-e8fc-491e-95d7-b2887dfb032f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:58.916409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.916909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.917396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.918103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.918582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.919000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.935137 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f HTTP/1.1 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '359b81c9-e8fc-491e-95d7-b2887dfb032f'} Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.936948 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 359b81c9-e8fc-491e-95d7-b2887dfb032f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:58.941221 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44668b08-2d57-4db3-be70-c1b5295f6c0e tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f" status: 404 len: 111 microversion: 2.1 time: 0.033748 Oct 15 21:01:58.942020 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1907/3021] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:01:58 2022] GET /compute/v2.1/servers/359b81c9-e8fc-491e-95d7-b2887dfb032f => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:01:58.947411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:58.948035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:58.953969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.954618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.955014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:58.955880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:58.956244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:58.959004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.240316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:59.249538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:59.250999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:59.252161 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-338b499e-ab84-4d19-9dd3-e6d9583c5777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.309150 Oct 15 21:01:59.252725 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1908/3022] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:58 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:59.257246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:59.257846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:59.264281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:59.264769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:59.265251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.265952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:59.266432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:59.266887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.293733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:01:59.379889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4e5a681a-4bcc-4d7f-99e3-6143cb3a6ea7 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac" status: 204 len: 0 microversion: 2.1 time: 0.125887 Oct 15 21:01:59.380581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1909/3023] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:01:59 2022] DELETE /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:01:59.386634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:01:59.387368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:59.393515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:59.394145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:59.394589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.395549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:59.395745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:59.396173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.638454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:59.647805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:59.649310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:01:59.650503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d670cbc1-62c3-4a06-a544-efe0a47d67aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1725 microversion: 2.20 time: 0.268781 Oct 15 21:01:59.651061 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1910/3024] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:01:59 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1725 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:01:59.665251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:01:59.709467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] HTTP exception thrown: Cannot 'migrate' instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 while it is in vm_state suspended Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Request method failure captured: Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070/action HTTP/1.1 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-AiIDnlOQ8tG7NIktVfdKbc9SkVp70gcbX7QV5sELop672omJSVxJkfqusuJ2uN6aeOP9agzAOF67p5FLUVm3fg78576Pm0MTW3GY5OVeYEKLy_7sHh8MR30q6HiXYbznIG0tnRSNQd8mHMkTyQCwp7DIuhAXGH7VnFI5_JDcudqA Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e0fd11023fb64e60bdeeeb473be1e5ef Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: {"migrate": {}} Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'migrate' instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 while it is in vm_state suspended Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'dfaae26e-0dc7-4c4e-8219-cb78c44d5070', 'body': {'migrate': {}}} Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 58, in _migrate Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.resize(req.environ['nova.context'], instance, Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 in vm_state suspended. Cannot resize while the instance is in this state. Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:01:59.711177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 69, in _migrate Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'migrate' instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 while it is in vm_state suspended Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'migrate' instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 while it is in vm_state suspended Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Returning 409 to user: Cannot 'migrate' instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 while it is in vm_state suspended {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1e94961-4f78-4a37-9700-964aceb50482 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070/action" status: 409 len: 148 microversion: 2.1 time: 0.060717 Oct 15 21:01:59.717393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1911/3025] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:01:59 2022] POST /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070/action => generated 148 bytes in 62 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:01:59.719017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57c75023-777f-4322-8bdb-418219d1099f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ebe6ee0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-402605450", "description": "tempest-description-119567523"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:00.076268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57c75023-777f-4322-8bdb-418219d1099f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 254 microversion: 2.1 time: 0.361398 Oct 15 21:02:00.076973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1912/3026] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:01:59 2022] POST /compute/v2.1/os-security-groups => generated 254 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:00.081740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd10a926-95ab-4db9-9966-cf2dadde2c0f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74f1f8430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:00.105274 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-dd10a926-95ab-4db9-9966-cf2dadde2c0f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Image not found. Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd10a926-95ab-4db9-9966-cf2dadde2c0f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b HTTP/1.1 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b'} Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b to delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b could not be found. Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.107411 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd10a926-95ab-4db9-9966-cf2dadde2c0f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:00.112097 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd10a926-95ab-4db9-9966-cf2dadde2c0f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b" status: 404 len: 62 microversion: 2.1 time: 0.031631 Oct 15 21:02:00.114932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1913/3027] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:00 2022] DELETE /compute/v2.1/images/aa8f8dd3-986a-4c5b-bdf6-a25d75e7653b => generated 62 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:02:00.124038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:00.124773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:00.131259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.131753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.132193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.132896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.133419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.133816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.394151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:00.404062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:00.405553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:00.406715 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73ddc31b-79ba-4b13-85e6-9f8376dda63d tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.290768 Oct 15 21:02:00.407631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1914/3028] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:00 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:00.412807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:00.413070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:00.420707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.421403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.422039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.422746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.423205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.423629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.452303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:02:00.526138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:00.543549 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd9a781c-9a68-4138-8af8-e73cc643c60d tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6" status: 204 len: 0 microversion: 2.1 time: 0.134069 Oct 15 21:02:00.544852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1915/3029] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:00 2022] DELETE /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:02:00.550828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c8d2737-5280-4632-a992-ad0d09d9a3a4 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "invalid"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c8d2737-5280-4632-a992-ad0d09d9a3a4 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Request method failure captured: Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f/action HTTP/1.1 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 39 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-AiIDnlOQ8tG7NIktVfdKbc9SkVp70gcbX7QV5sELop672omJSVxJkfqusuJ2uN6aeOP9agzAOF67p5FLUVm3fg78576Pm0MTW3GY5OVeYEKLy_7sHh8MR30q6HiXYbznIG0tnRSNQd8mHMkTyQCwp7DIuhAXGH7VnFI5_JDcudqA Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e0fd11023fb64e60bdeeeb473be1e5ef Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: {"os-resetState": {"state": "invalid"}} Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c8032d6f-ff10-4add-be09-6de0bbdcf98f', 'body': {'os-resetState': {'state': 'invalid'}}} Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['active', 'error'] Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['os-resetState']['properties']['state']: Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'enum': ['active', 'error'], 'type': 'string'} Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['os-resetState']: Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'invalid' Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:02:00.554836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c8d2737-5280-4632-a992-ad0d09d9a3a4 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Returning 400 to user: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c8d2737-5280-4632-a992-ad0d09d9a3a4 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f/action" status: 400 len: 144 microversion: 2.1 time: 0.010734 Oct 15 21:02:00.564653 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1916/3030] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:02:00 2022] POST /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f/action => generated 144 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:02:00.565838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:00.566508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:00.573184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.573666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.574099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.574794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.575250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.575676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.822879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:00.830692 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-96cbd32e-3e51-4911-b9d8-9b9e224861d5 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1512 microversion: 2.1 time: 0.271955 Oct 15 21:02:00.831252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1917/3031] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:00 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1512 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:00.835969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8ff9857-731e-427e-a67f-c67b539497ac tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ebc05e0>, body: {"security_group_rule": {"parent_group_id": "adab5309-6f22-49da-ac27-8c3409d16390", "ip_protocol": "tcp", "from_port": 22, "to_port": 21}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:00.922980 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c8ff9857-731e-427e-a67f-c67b539497ac tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Invalid port range 22:21. Former value cannot be greater than the later Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8ff9857-731e-427e-a67f-c67b539497ac tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 138 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "adab5309-6f22-49da-ac27-8c3409d16390", "ip_protocol": "tcp", "from_port": 22, "to_port": 21}} Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid port range 22:21. Former value cannot be greater than the later Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'adab5309-6f22-49da-ac27-8c3409d16390', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 21}}} Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 150, in _new_ingress_rule Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 22:21. Former value cannot be greater than the later Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 22:21. Former value cannot be greater than the later Oct 15 21:02:00.924530 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 22:21. Former value cannot be greater than the later Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8ff9857-731e-427e-a67f-c67b539497ac tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 400 to user: Invalid port range 22:21. Former value cannot be greater than the later {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8ff9857-731e-427e-a67f-c67b539497ac tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 115 microversion: 2.1 time: 0.093603 Oct 15 21:02:00.928764 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1918/3032] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:02:00 2022] POST /compute/v2.1/os-security-group-rules => generated 115 bytes in 95 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:02:00.934241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:00.934749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:00.942010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.942506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.942949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.943649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.944111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.944609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.958148 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f HTTP/1.1 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e14662ce-cafb-4dcf-add1-d49193e3ed0f'} Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:00.959712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac2ffb31-ae49-41bc-8052-a4f843f814f8 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 404 len: 111 microversion: 2.1 time: 0.033685 Oct 15 21:02:00.964063 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1919/3033] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:00 2022] DELETE /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:00.966449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:00.967004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:00.972837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.973437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.973879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:00.974584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:00.975046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:00.975476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:01.336471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:01.346411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:01.347900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:01.349107 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-853eb198-74b9-43ec-af97-ef7fe4142d94 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.386523 Oct 15 21:02:01.349676 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1920/3034] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:00 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:01.354747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "6e28f7f5-2809-45be-9dd9-d05f11e48e6f"}, {"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "97f99e08-5287-4631-83bc-e4e52565d3e4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:01.366203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:01.366203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:01.366710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:01.379656 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] Creating event network-changed:6e28f7f5-2809-45be-9dd9-d05f11e48e6f for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:01.380239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] Creating event network-changed:97f99e08-5287-4631-83bc-e4e52565d3e4 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:01.386517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1cce87ea-4d85-4ee4-a1d9-ba2ed7bacd19 req-14f8e0bd-f364-4e9b-bdb9-c36840d843d9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.035425 Oct 15 21:02:01.387122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1921/3035] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:01 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:01.392463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:01.393326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:01.399842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:01.400589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:01.401047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:01.401772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:01.402239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:01.402672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:01.807952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:01.815679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc32f99e-e35c-40e0-9391-126f5d675d15 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac" status: 200 len: 1512 microversion: 2.1 time: 0.427380 Oct 15 21:02:01.816256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1922/3036] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:01 2022] GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac => generated 1512 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:01.821829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-617e59c8-cd30-46c1-ad8d-2797702d310e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": 1}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-617e59c8-cd30-46c1-ad8d-2797702d310e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Request method failure captured: Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f/action HTTP/1.1 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 31 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-AiIDnlOQ8tG7NIktVfdKbc9SkVp70gcbX7QV5sELop672omJSVxJkfqusuJ2uN6aeOP9agzAOF67p5FLUVm3fg78576Pm0MTW3GY5OVeYEKLy_7sHh8MR30q6HiXYbznIG0tnRSNQd8mHMkTyQCwp7DIuhAXGH7VnFI5_JDcudqA Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e0fd11023fb64e60bdeeeb473be1e5ef Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: {"os-resetState": {"state": 1}} Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c8032d6f-ff10-4add-be09-6de0bbdcf98f', 'body': {'os-resetState': {'state': 1}}} Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 1 is not of type 'string' Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['os-resetState']['properties']['state']: Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'enum': ['active', 'error'], 'type': 'string'} Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['os-resetState']: Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 1 Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Oct 15 21:02:01.825488 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-617e59c8-cd30-46c1-ad8d-2797702d310e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Returning 400 to user: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:01.832869 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-617e59c8-cd30-46c1-ad8d-2797702d310e tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f/action" status: 400 len: 120 microversion: 2.1 time: 0.009490 Oct 15 21:02:01.835305 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1923/3037] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:02:01 2022] POST /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f/action => generated 120 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:02:01.838839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:01.839426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:01.846152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:01.846650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:01.847090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:01.847845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:01.848309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:01.848737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:02.840598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:02.848827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:02.850339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:02.851509 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a792ad2f-8c90-4f8b-a61e-c99a6ea7c880 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1725 microversion: 2.20 time: 1.016485 Oct 15 21:02:02.852068 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1924/3038] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:01 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1725 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:02.857180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e70ef8b2-f1b4-4c40-be09-1ec8854ad9fb tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ebeb700>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-181966306", "description": "tempest-description-131315889"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:03.812768 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e70ef8b2-f1b4-4c40-be09-1ec8854ad9fb tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 254 microversion: 2.1 time: 0.958906 Oct 15 21:02:03.813642 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1925/3039] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:02:02 2022] POST /compute/v2.1/os-security-groups => generated 254 bytes in 961 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:03.818749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:03.819046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: e14662ce-cafb-4dcf-add1-d49193e3ed0f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:03.827815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:03.842268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:03.842970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:03.843718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:03.844213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:03.844618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:03.862035 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f HTTP/1.1 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e14662ce-cafb-4dcf-add1-d49193e3ed0f'} Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:03.863828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:03.869990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance e14662ce-cafb-4dcf-add1-d49193e3ed0f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:03.870891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a47dfab6-536b-4428-b886-8a467e04740f tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f" status: 404 len: 111 microversion: 2.1 time: 0.054920 Oct 15 21:02:03.874758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1926/3040] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:03 2022] GET /compute/v2.1/servers/e14662ce-cafb-4dcf-add1-d49193e3ed0f => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:03.879527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:03.885531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:03.890754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:03.890980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:03.891427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:03.892568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:03.892779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:03.893479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:04.664967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:04.677132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:04.678755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:04.680164 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-408c3885-66b5-4163-b550-02d31d590d60 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.805863 Oct 15 21:02:04.680901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1927/3041] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:03 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 807 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:04.685560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c946e590-011d-43ef-81c3-89041c573da5 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:04.688085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c946e590-011d-43ef-81c3-89041c573da5 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Failed to fetch instance by id 999 {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 21:02:04.688896 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c946e590-011d-43ef-81c3-89041c573da5 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] HTTP exception thrown: Instance 999 could not be found. Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c946e590-011d-43ef-81c3-89041c573da5 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Request method failure captured: Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/999/action HTTP/1.1 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 37 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-AiIDnlOQ8tG7NIktVfdKbc9SkVp70gcbX7QV5sELop672omJSVxJkfqusuJ2uN6aeOP9agzAOF67p5FLUVm3fg78576Pm0MTW3GY5OVeYEKLy_7sHh8MR30q6HiXYbznIG0tnRSNQd8mHMkTyQCwp7DIuhAXGH7VnFI5_JDcudqA Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a484be1f1a70400ea15a4560a26f7dbe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: a484be1f1a70400ea15a4560a26f7dbe Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-58638062 Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e0fd11023fb64e60bdeeeb473be1e5ef Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-58638062-project-admin Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: {"os-resetState": {"state": "error"}} Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 999 could not be found. Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '999', 'body': {'os-resetState': {'state': 'error'}}} Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 999 could not be found. Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/admin_actions.py", line 66, in _reset_state Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 999 could not be found. Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:04.692723 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:04.698202 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:04.698202 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 999 could not be found. Oct 15 21:02:04.698202 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:04.699329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c946e590-011d-43ef-81c3-89041c573da5 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] Returning 404 to user: Instance 999 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:04.700400 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c946e590-011d-43ef-81c3-89041c573da5 tempest-ServersAdminNegativeTestJSON-58638062 tempest-ServersAdminNegativeTestJSON-58638062-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/999/action" status: 404 len: 78 microversion: 2.1 time: 0.018331 Oct 15 21:02:04.701406 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1928/3042] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Oct 15 21:02:04 2022] POST /compute/v2.1/servers/999/action => generated 78 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:02:04.706162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:04.706773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:04.715505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:04.716207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:04.716714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:04.717693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:04.718244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:04.718755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:04.723487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:04.985746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:04.994389 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9068dcaa-1110-4340-a110-880493c3a4f2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1512 microversion: 2.1 time: 0.291815 Oct 15 21:02:04.995346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1929/3043] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:04 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1512 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:04.999610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:05.000228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:05.006106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.006615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.007038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.007734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.008185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.009260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.355090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:05.364851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:05.366735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:05.368651 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09f23fbb-efc8-47a8-8102-a4b8d91b2e70 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.372198 Oct 15 21:02:05.369782 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1930/3044] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:04 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:05.374726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:05.375761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:05.382302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.382921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.383461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.384489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.385104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.385794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.419562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.420054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.420497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.420951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lazy-loading 'fault' on Instance uuid fd9910fa-b048-419c-8e20-08c4c1e4e6ac {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:02:05.479831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e1353ffe42243af82b590bc98e96a42', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bc93093ddf44aada9598a14b5417a2d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:05.487990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-480b88b5-fd11-48c0-89ff-9dc354462f36 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac" status: 200 len: 1307 microversion: 2.1 time: 0.117195 Oct 15 21:02:05.488558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1931/3045] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:05 2022] GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac => generated 1307 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:05.494080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "c874ec72-d847-4302-a82c-1a15a1e24a5f"}, {"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "f425e84b-b112-4afe-9505-a001d8f8ce16"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:05.503472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.503953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.504393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.518031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] Creating event network-changed:c874ec72-d847-4302-a82c-1a15a1e24a5f for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:05.518777 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] Creating event network-changed:f425e84b-b112-4afe-9505-a001d8f8ce16 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:05.523725 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3daffd30-a2f3-40cf-9288-8a17a7e25bfa req-332e3a3c-b300-438e-9ac9-7a3dab20a3b8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.033508 Oct 15 21:02:05.525645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1932/3046] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:05 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:05.533267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3cf7957-3b65-4d8e-886a-a7d01bf666ca tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ebfa3a0>, body: {"security_group_rule": {"parent_group_id": "da4f3341-836b-4413-a101-b9fd102c2a63", "ip_protocol": "tcp", "from_port": 22, "to_port": 2094594896}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:05.686470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c3cf7957-3b65-4d8e-886a-a7d01bf666ca tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Invalid port range 22:2094594896. Valid TCP ports should be between 1-65535 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3cf7957-3b65-4d8e-886a-a7d01bf666ca tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 146 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "da4f3341-836b-4413-a101-b9fd102c2a63", "ip_protocol": "tcp", "from_port": 22, "to_port": 2094594896}} Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid port range 22:2094594896. Valid TCP ports should be between 1-65535 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'da4f3341-836b-4413-a101-b9fd102c2a63', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 2094594896}}} Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 157, in _new_ingress_rule Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 22:2094594896. Valid TCP ports should be between 1-65535 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 22:2094594896. Valid TCP ports should be between 1-65535 Oct 15 21:02:05.688135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 22:2094594896. Valid TCP ports should be between 1-65535 Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3cf7957-3b65-4d8e-886a-a7d01bf666ca tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 400 to user: Invalid port range 22:2094594896. Valid TCP ports should be between 1-65535 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3cf7957-3b65-4d8e-886a-a7d01bf666ca tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 119 microversion: 2.1 time: 0.161896 Oct 15 21:02:05.693911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1933/3047] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:02:05 2022] POST /compute/v2.1/os-security-group-rules => generated 119 bytes in 164 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:02:05.695600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38af7911-09d3-4e17-b341-aeacc6a5e507 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe754344820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:05.718640 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-38af7911-09d3-4e17-b341-aeacc6a5e507 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Image not found. Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38af7911-09d3-4e17-b341-aeacc6a5e507 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 HTTP/1.1 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'fa8e2e21-c1ce-4609-a448-21af646a7c10'} Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image fa8e2e21-c1ce-4609-a448-21af646a7c10 to delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image fa8e2e21-c1ce-4609-a448-21af646a7c10 could not be found. Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.720396 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38af7911-09d3-4e17-b341-aeacc6a5e507 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:05.725445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38af7911-09d3-4e17-b341-aeacc6a5e507 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10" status: 404 len: 62 microversion: 2.1 time: 0.030363 Oct 15 21:02:05.727203 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1934/3048] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:05 2022] DELETE /compute/v2.1/images/fa8e2e21-c1ce-4609-a448-21af646a7c10 => generated 62 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:02:05.732080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:05.732814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:05.739612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.740109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.740612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:05.741365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:05.742209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:05.742429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.109130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:06.117891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:06.119550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:06.120740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4261651-a5af-466a-8261-144a9c735728 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1725 microversion: 2.20 time: 0.392813 Oct 15 21:02:06.121357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1935/3049] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:05 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1725 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:06.125948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:06.126452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:06.132631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.133156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.134095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.134807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.135513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.136112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.167287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:02:06.314126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea2546af-b816-4102-8ef4-178756c388f3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "DELETE /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 204 len: 0 microversion: 2.1 time: 0.191233 Oct 15 21:02:06.318043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1936/3050] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:06 2022] DELETE /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:02:06.323925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Action: 'create', calling method: .version_select at 0x7fe74ebc0b80>, body: {"security_group_rule": {"parent_group_id": "89c8c1be-d74f-46da-a02d-4b124825cc07", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:06.419098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 21:02:06.420361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Neutron security group 89c8c1be-d74f-46da-a02d-4b124825cc07 not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 21:02:06.420905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.420905 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 138 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: {"security_group_rule": {"parent_group_id": "89c8c1be-d74f-46da-a02d-4b124825cc07", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'security_group_rule': {'parent_group_id': '89c8c1be-d74f-46da-a02d-4b124825cc07', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 21:02:06.422951 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 287, in create Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get( Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 303, in create Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 404 to user: Security group 89c8c1be-d74f-46da-a02d-4b124825cc07 does not exist Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-ba550064-00cb-4ce7-bd05-da52d4237d35'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bdfd6d83-57ab-4983-b07c-ee490561950a tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 404 len: 194 microversion: 2.1 time: 0.105205 Oct 15 21:02:06.427639 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1937/3051] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:02:06 2022] POST /compute/v2.1/os-security-group-rules => generated 194 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:06.433074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:06.433074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:06.439894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.440833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.441332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.442283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.442960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.443397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.769687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:06.781107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:06.782573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:06.783745 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-64894dd0-ddca-4e5d-912f-ffee88b45e04 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.357236 Oct 15 21:02:06.788507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1938/3052] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:06 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:06.788900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:06.789328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:06.796262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.796754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.797228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.797940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.798405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.798847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.815470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da HTTP/1.1 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4da129cd-0c8c-4e46-bcc6-beb0550690da'} Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:06.817204 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:06.822648 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:06.824164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:06.825017 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc638d40-d721-432f-985f-e419a2094876 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 404 len: 111 microversion: 2.1 time: 0.039775 Oct 15 21:02:06.825889 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1939/3053] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:06 2022] DELETE /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:06.830377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:06.830951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:06.837377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.837941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.838378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:06.839077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:06.839633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:06.840062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.061947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:07.070348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51d4e677-d8e6-4a50-a24c-1bdb71541813 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1489 microversion: 2.1 time: 0.243505 Oct 15 21:02:07.071043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1940/3054] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:06 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1489 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:07.075787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:07.076406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:07.082418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.083235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.083862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.084752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.085252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.085956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.311779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:07.320899 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f0b8eb0-9aa9-4a2a-848d-6c271972661b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 200 len: 1468 microversion: 2.1 time: 0.248768 Oct 15 21:02:07.322534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1941/3055] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:07 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 1468 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:07.329955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:07.330813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:07.338622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.339128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.339574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.340338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.340794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.341249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.747447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:07.757666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:07.760182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:07.761521 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b38f1209-d738-4ea3-871c-4572ea95faca tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.437414 Oct 15 21:02:07.762260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1942/3056] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:07 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:07.767196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe74ebfa280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:07.827618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Error message: {"NeutronError": {"type": "SecurityGroupRuleNotFound", "message": "Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 21:02:07.828788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Neutron security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 not found {{(pid=138688) get_rule /opt/stack/nova/nova/network/security_group_api.py:463}} Oct 15 21:02:07.829630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] HTTP exception thrown: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.829630 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Request method failure captured: Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-security-group-rules/33376f57-f61d-4a46-a8c9-5d3893359f97 HTTP/1.1 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-pmWMwjeB4V255mcWvJ8-N0d_ApEidcTZdg2bdmZAbVkBhwyQJ3n08-Xhrow1gIMIha9erT7h-V52UJfe76y5M_P9J0DDnmTTRMGAGgC0odhSQtKWz0C1_PDd1KvoCuZTDGANumQWCx7je9P_rHOY8-40FKKl-4DJHZp43b0ius0w Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/453ad8b708754a7ea5788abe2cc688d7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 453ad8b708754a7ea5788abe2cc688d7 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d625eb28f0946f5905d4e99543fa014 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-169800727-project Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '33376f57-f61d-4a46-a8c9-5d3893359f97'} Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 459, in get_rule Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi rule = neutron.show_security_group_rule( Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1092, in show_security_group_rule Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_rule_path % (security_group_rule), Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 21:02:07.832273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 359, in delete Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi rule = security_group_api.get_rule(context, id) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 464, in get_rule Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 365, in delete Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Returning 404 to user: Security group rule 33376f57-f61d-4a46-a8c9-5d3893359f97 does not exist Oct 15 21:02:07.837849 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-e939ebb6-787b-455e-a90e-4281bc6d0c08'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:07.846839 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37fc1393-be10-44a1-a4c8-6b27d476ee3f tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-group-rules/33376f57-f61d-4a46-a8c9-5d3893359f97" status: 404 len: 199 microversion: 2.1 time: 0.076199 Oct 15 21:02:07.846839 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1943/3057] 10.222.0.16 () {58 vars in 1287 bytes} [Sat Oct 15 21:02:07 2022] DELETE /compute/v2.1/os-security-group-rules/33376f57-f61d-4a46-a8c9-5d3893359f97 => generated 199 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:07.848623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:07.849282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: fd9910fa-b048-419c-8e20-08c4c1e4e6ac] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:07.856324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.856828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.857304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.858017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.858486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.858921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.874009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] HTTP exception thrown: Instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac could not be found. Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Request method failure captured: Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac HTTP/1.1 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9spXps-tidFtykZP3kNBioK_CBvtVdWMXl0N_tCzwuR3S5aAPAOpajp6iiyRhYjTx-6gFyt3uwZD5wwRKm2HuTI7PcorBDZHlvvIMOH5QE4KOpgC1CsLGL016n4bnI3bCitlLwQ3QoBmYINxdQhXJXR81oYTtHMzmhtnL-19aB0fA Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 9bc93093ddf44aada9598a14b5417a2d Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateTestJSON-123314571 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateTestJSON-123314571 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 9bc93093ddf44aada9598a14b5417a2d Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateTestJSON-123314571 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateTestJSON-123314571-project Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4e1353ffe42243af82b590bc98e96a42 Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateTestJSON-123314571-project Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac could not be found. Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'fd9910fa-b048-419c-8e20-08c4c1e4e6ac'} Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac could not be found. Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac could not be found. Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.875573 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac could not be found. Oct 15 21:02:07.880307 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.881594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Returning 404 to user: Instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:07.882470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5c36037-2051-458c-ad58-d1a3b53e23fe tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac" status: 404 len: 111 microversion: 2.1 time: 0.038924 Oct 15 21:02:07.883036 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1944/3058] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:07 2022] GET /compute/v2.1/servers/fd9910fa-b048-419c-8e20-08c4c1e4e6ac => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:07.888705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:07.889352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 4da129cd-0c8c-4e46-bcc6-beb0550690da] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:07.895823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.896319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.896767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.897578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.898048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.898484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.915544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da HTTP/1.1 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '4da129cd-0c8c-4e46-bcc6-beb0550690da'} Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.916745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 4da129cd-0c8c-4e46-bcc6-beb0550690da could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce721062-897d-4d04-9273-d509d755ceb6 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da" status: 404 len: 111 microversion: 2.1 time: 0.033336 Oct 15 21:02:07.921527 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1945/3059] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:07 2022] GET /compute/v2.1/servers/4da129cd-0c8c-4e46-bcc6-beb0550690da => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:07.923864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-\u00cd\u00f1st\u00e1\u00f1c\u00e9-555900812"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:07.926473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:07.933542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.934045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.934492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:07.935233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:07.935704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:07.936142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:08.019010 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8803632b-0121-4d48-b722-68000c2a85bd tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "PUT /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1061 microversion: 2.1 time: 0.099050 Oct 15 21:02:08.033081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1946/3060] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:02:07 2022] PUT /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1061 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:08.033081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:08.033081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:08.034005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:08.034703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:08.035354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:08.036437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:08.037237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:08.037885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:08.621190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:08.629463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:08.630903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:08.632069 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b0a6a7a-b108-4ea4-be9b-145d6d4480ec tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1725 microversion: 2.20 time: 0.610818 Oct 15 21:02:08.632616 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1947/3061] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:08 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1725 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:08.655093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd9910fa-b048-419c-8e20-08c4c1e4e6ac", "name": "network-vif-deleted", "tag": "9dcbb0e4-cd91-4589-ba63-84c1689a5346"}, {"server_uuid": "5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6", "name": "network-vif-deleted", "tag": "18e73793-968c-429b-bc0b-c9b535576589"}, {"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "2a2ab61a-72e6-4577-a7e1-d1b4987a2389"}, {"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "5621f2f9-6408-416e-9e9b-6029170044f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:08.666659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:08.667135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:08.667565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:08.682234 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Creating event network-vif-deleted:9dcbb0e4-cd91-4589-ba63-84c1689a5346 for instance fd9910fa-b048-419c-8e20-08c4c1e4e6ac on nch2-master-1 Oct 15 21:02:08.682740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Creating event network-vif-deleted:18e73793-968c-429b-bc0b-c9b535576589 for instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 on nch1-master-1 Oct 15 21:02:08.683240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Creating event network-changed:2a2ab61a-72e6-4577-a7e1-d1b4987a2389 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:08.683736 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] Creating event network-changed:5621f2f9-6408-416e-9e9b-6029170044f4 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:08.693532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-45a3d810-b4b2-48ec-aa12-577ed0eb85b6 req-93abc6b9-339e-44a4-996b-2ed9772638d4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 688 microversion: 2.1 time: 0.043268 Oct 15 21:02:08.694129 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1948/3062] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:08 2022] POST /compute/v2.1/os-server-external-events => generated 688 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:08.699892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:08.700584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:08.706640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:08.707125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:08.707553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:08.708251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:08.708709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:08.709160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:09.185707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:09.197399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:09.198934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:09.200138 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d8fc133-882f-4450-bab7-60a6a6bad460 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.504749 Oct 15 21:02:09.201436 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1949/3063] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:08 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:09.206328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7d78081-5854-4152-b028-208f9ae4b291 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe74eb90790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:09.913128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7d78081-5854-4152-b028-208f9ae4b291 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/da4f3341-836b-4413-a101-b9fd102c2a63" status: 202 len: 0 microversion: 2.1 time: 0.710161 Oct 15 21:02:09.913929 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1950/3064] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:02:09 2022] DELETE /compute/v2.1/os-security-groups/da4f3341-836b-4413-a101-b9fd102c2a63 => generated 0 bytes in 712 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:02:09.919292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:09.919852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] [instance: 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:09.926672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:09.927381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:09.928004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:09.928903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:09.929381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:09.929818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:09.946582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] HTTP exception thrown: Instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 could not be found. Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Request method failure captured: Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 HTTP/1.1 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9spXps-tidFtykZP3kNBioK_CBvtVdWMXl0N_tCzwuR3S5aAPAOpajp6iiyRhYjTx-6gFyt3uwZD5wwRKm2HuTI7PcorBDZHlvvIMOH5QE4KOpgC1CsLGL016n4bnI3bCitlLwQ3QoBmYINxdQhXJXR81oYTtHMzmhtnL-19aB0fA Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 9bc93093ddf44aada9598a14b5417a2d Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateTestJSON-123314571 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9bc93093ddf44aada9598a14b5417a2d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateTestJSON-123314571 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 9bc93093ddf44aada9598a14b5417a2d Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateTestJSON-123314571 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateTestJSON-123314571-project Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4e1353ffe42243af82b590bc98e96a42 Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateTestJSON-123314571-project Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 could not be found. Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6'} Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 could not be found. Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 could not be found. Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:09.948405 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 could not be found. Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.954053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] Returning 404 to user: Instance 5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:09.955880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3babbc19-b9f6-42fa-8b55-b732e4a507e8 tempest-MultipleCreateTestJSON-123314571 tempest-MultipleCreateTestJSON-123314571-project] 10.222.0.16 "GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6" status: 404 len: 111 microversion: 2.1 time: 0.040968 Oct 15 21:02:09.961408 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1951/3065] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:09 2022] GET /compute/v2.1/servers/5741dcf8-ae52-4fbc-b6d3-e9e90cb753d6 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:09.965869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-847ee6cc-7d44-4f71-9144-0d05d57e133d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '.version_select at 0x7fe74ebfa280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:09.992111 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-847ee6cc-7d44-4f71-9144-0d05d57e133d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Image not found. Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-847ee6cc-7d44-4f71-9144-0d05d57e133d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 HTTP/1.1 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '89596dba-f9dc-4736-bf36-b23de99f5de2'} Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 89596dba-f9dc-4736-bf36-b23de99f5de2 to delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 89596dba-f9dc-4736-bf36-b23de99f5de2 could not be found. Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.994995 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-847ee6cc-7d44-4f71-9144-0d05d57e133d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:09.999658 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-847ee6cc-7d44-4f71-9144-0d05d57e133d tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2" status: 404 len: 62 microversion: 2.1 time: 0.034254 Oct 15 21:02:10.001675 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1952/3066] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:09 2022] DELETE /compute/v2.1/images/89596dba-f9dc-4736-bf36-b23de99f5de2 => generated 62 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:02:10.004463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:10.005106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:10.263512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.264008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.264638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.265564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.266021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.266608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.288261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:10.521006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:10.531937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7ace755-f45e-458a-a217-8dc7d439eb3e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1488 microversion: 2.1 time: 0.530690 Oct 15 21:02:10.532970 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1953/3067] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:09 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1488 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:10.538185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:10.538835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: dfaae26e-0dc7-4c4e-8219-cb78c44d5070] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:10.545819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.546591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.547042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.547752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.548217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.548930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.567030 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] HTTP exception thrown: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 could not be found. Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Request method failure captured: Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 HTTP/1.1 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9_fHQVJ85WBd3l_m4euhtG4cgODYMwB9qe_LsLrsZyd7RcR_k3x7moGPwq6DGXl7w2IR101-eyun8Uu0_Ld5-Ju9CovfjJp6f0I33yl9rfVukwnW54aMbElqpH1uWjq-K9S-k8io8QIvn_sR1i2_Mc6BljPJGTEWvtcM2hg8Hfut4 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e91f96ee17114009a55fe2bf388a7aa7 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1372966968 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1372966968 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e91f96ee17114009a55fe2bf388a7aa7 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1372966968 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-1372966968-project Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 573c02d502744876babbdb4fbb4fb9e6 Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1372966968-project Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 could not be found. Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'dfaae26e-0dc7-4c4e-8219-cb78c44d5070'} Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 could not be found. Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 could not be found. Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:10.568763 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 could not be found. Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:10.574425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Returning 404 to user: Instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:10.576190 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfac9f66-3871-436d-91c5-cf32b7bf250f tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070" status: 404 len: 111 microversion: 2.1 time: 0.041924 Oct 15 21:02:10.576751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1954/3068] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:10 2022] GET /compute/v2.1/servers/dfaae26e-0dc7-4c4e-8219-cb78c44d5070 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:10.581457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:10.582071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:10.587967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.588454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.588885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.589766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.590225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.590648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.888386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:10.898735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:10.901194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:10.902426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac07f3c3-3f29-472a-8e1d-1b4341b89286 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.324692 Oct 15 21:02:10.903084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1955/3069] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:10 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:10.908018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:10.908631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:10.916130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.917918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.918533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:10.919430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:10.920058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:10.920655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:11.148755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:11.159730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:11.170042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:11.171413 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee77d477-4caf-4e3f-8b64-17f1568d7c6c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1746 microversion: 2.20 time: 0.267162 Oct 15 21:02:11.172535 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1956/3070] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:10 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1746 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:11.177479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3a80c8a-9160-45f1-9c5f-3825b65c5057 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe74eeeef70>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:11.935150 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3a80c8a-9160-45f1-9c5f-3825b65c5057 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/adab5309-6f22-49da-ac27-8c3409d16390" status: 202 len: 0 microversion: 2.1 time: 0.761237 Oct 15 21:02:11.936327 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1957/3071] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:02:11 2022] DELETE /compute/v2.1/os-security-groups/adab5309-6f22-49da-ac27-8c3409d16390 => generated 0 bytes in 763 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:02:11.941565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:11.942416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:11.954390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:11.957474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:11.958117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:11.959424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:11.961662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:11.963145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:11.994311 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 HTTP/1.1 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '7dafc5b2-733a-433a-bbcc-5136827624c7'} Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:11.996006 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:12.000265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:12.003118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:12.004332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-34f47d88-5884-4ca2-bfb3-c122c2558b7c tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 404 len: 111 microversion: 2.1 time: 0.066581 Oct 15 21:02:12.005448 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1958/3072] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:11 2022] DELETE /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:12.011695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:12.012857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:12.019312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.020131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.020803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.021774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.022510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.023325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.366121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:12.379872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:12.381380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:12.382608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0dfb44f3-1eb7-4207-9f6e-4c23b96c6d76 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.375976 Oct 15 21:02:12.383867 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1959/3073] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:12 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:12.389064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:12.390088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:12.408397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.408886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.409486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.410211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.410676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.411113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.587427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:12.596189 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ed818f6-a625-4fe7-bd45-ed3d008a9944 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1488 microversion: 2.1 time: 0.210928 Oct 15 21:02:12.598369 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1960/3074] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:12 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1488 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:12.603760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:12.604675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:12.611587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.612414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.613120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.614711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.615507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.616188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.652098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:02:12.813865 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87545a14-c947-48d3-8d9b-fe97ba751bd3 tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 204 len: 0 microversion: 2.1 time: 0.213811 Oct 15 21:02:12.815486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1961/3075] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:12 2022] DELETE /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:02:12.825647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fc5270f3-217e-433f-97f7-08b961f25c2b req-b1be132b-863c-4a6b-ac36-9ba9cc324461 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfaae26e-0dc7-4c4e-8219-cb78c44d5070", "name": "network-vif-deleted", "tag": "4b8860d7-6686-44ce-b530-e8b8214dff80"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:12.839022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc5270f3-217e-433f-97f7-08b961f25c2b req-b1be132b-863c-4a6b-ac36-9ba9cc324461 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.839593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc5270f3-217e-433f-97f7-08b961f25c2b req-b1be132b-863c-4a6b-ac36-9ba9cc324461 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.840103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc5270f3-217e-433f-97f7-08b961f25c2b req-b1be132b-863c-4a6b-ac36-9ba9cc324461 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.861319 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fc5270f3-217e-433f-97f7-08b961f25c2b req-b1be132b-863c-4a6b-ac36-9ba9cc324461 service nova] Creating event network-vif-deleted:4b8860d7-6686-44ce-b530-e8b8214dff80 for instance dfaae26e-0dc7-4c4e-8219-cb78c44d5070 on nch2-master-1 Oct 15 21:02:12.891669 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fc5270f3-217e-433f-97f7-08b961f25c2b req-b1be132b-863c-4a6b-ac36-9ba9cc324461 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072307 Oct 15 21:02:12.897238 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1962/3076] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:12.912102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:12.914338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:12.942298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.943529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.943988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:12.945473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:12.946023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:12.946475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:13.415552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:13.432630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:13.434136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:13.444776 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-306ae59d-9d23-40fb-b04a-1005d7e27090 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.542850 Oct 15 21:02:13.448196 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1963/3077] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:12 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:13.455817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c543a2a5-9be6-4550-80fd-4efc957b6d6d tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe74f12d670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:14.031991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c543a2a5-9be6-4550-80fd-4efc957b6d6d tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/ba1d872f-43b1-4746-a846-04ba8efc9c3f" status: 202 len: 0 microversion: 2.1 time: 0.581631 Oct 15 21:02:14.033851 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1964/3078] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:02:13 2022] DELETE /compute/v2.1/os-security-groups/ba1d872f-43b1-4746-a846-04ba8efc9c3f => generated 0 bytes in 585 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:02:14.040136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:14.041116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] [instance: 7dafc5b2-733a-433a-bbcc-5136827624c7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:14.057342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.057926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.058387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.059135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.063845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.064609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.094873 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] HTTP exception thrown: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Request method failure captured: Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 HTTP/1.1 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx4sgnyuuI3wSqOUl96vTxFYQ3Uli6OuLcZXKtY18TX6X9u4e628VqmshtFKYaQTsPgs9niLR0sa0fQF9EJZK-DEnadgoWfnqQdepeEAPZjE2MTNfdiYtMPVoP4T7XKEMKki4Db0eRT8Ke39wxuTK0Bm5ZBiB8-iwA89sHxXEIxJ85g Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b62c28804ebd4f1f9ac47aa541500110"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b62c28804ebd4f1f9ac47aa541500110 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesTestJSON-1612481233 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 5d93a680570f4d23b1f6df534bc082b2 Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesTestJSON-1612481233-project Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '7dafc5b2-733a-433a-bbcc-5136827624c7'} Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:14.104025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] Returning 404 to user: Instance 7dafc5b2-733a-433a-bbcc-5136827624c7 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:14.109747 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e6045bc-9e5f-4118-a09d-10939cd2c0b1 tempest-ImagesTestJSON-1612481233 tempest-ImagesTestJSON-1612481233-project] 10.222.0.16 "GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7" status: 404 len: 111 microversion: 2.1 time: 0.070317 Oct 15 21:02:14.113398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1965/3079] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:14 2022] GET /compute/v2.1/servers/7dafc5b2-733a-433a-bbcc-5136827624c7 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:14.119585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:14.125702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:14.145021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.146079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.147004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.148675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.150394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.152748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.426839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:14.435759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:14.444900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:14.448498 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35f5da6a-78d9-43b9-940c-4b999654a8dd tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.333824 Oct 15 21:02:14.476460 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1966/3080] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:14 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:14.481823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:14.488717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:14.517666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.518306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.518767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.521546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.522042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.522476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.613931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:02:14.817389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:14.837504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a265764d-a68b-4793-9397-3846b51fe00f tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 204 len: 0 microversion: 2.1 time: 0.359004 Oct 15 21:02:14.843162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1967/3081] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:14 2022] DELETE /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:02:14.850516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:14.851105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:14.882449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.882966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.883454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:14.885728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:14.888296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:14.888800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:15.186522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '573c02d502744876babbdb4fbb4fb9e6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e91f96ee17114009a55fe2bf388a7aa7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:15.209616 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-705e8f1e-1c6d-459b-a7f2-bc86b25ec57b tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 200 len: 1477 microversion: 2.1 time: 0.356653 Oct 15 21:02:15.214029 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1968/3082] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:14 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 1477 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:15.223922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-13fdf270-f36e-4e4e-85ba-df286040dae3 req-574909b1-1ecf-4b1e-bc30-9350ef6ffc72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "77b4ac77-9b8d-4c4f-a944-9e3a506c9e67"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:15.269947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-13fdf270-f36e-4e4e-85ba-df286040dae3 req-574909b1-1ecf-4b1e-bc30-9350ef6ffc72 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:15.272265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-13fdf270-f36e-4e4e-85ba-df286040dae3 req-574909b1-1ecf-4b1e-bc30-9350ef6ffc72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:15.273128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-13fdf270-f36e-4e4e-85ba-df286040dae3 req-574909b1-1ecf-4b1e-bc30-9350ef6ffc72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:15.307850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-13fdf270-f36e-4e4e-85ba-df286040dae3 req-574909b1-1ecf-4b1e-bc30-9350ef6ffc72 service nova] Creating event network-vif-plugged:77b4ac77-9b8d-4c4f-a944-9e3a506c9e67 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:15.326732 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-13fdf270-f36e-4e4e-85ba-df286040dae3 req-574909b1-1ecf-4b1e-bc30-9350ef6ffc72 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.110034 Oct 15 21:02:15.330634 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1969/3083] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:15.340744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:15.341539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:15.366309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:15.366829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:15.367287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:15.373180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:15.375236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:15.377547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:15.791624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:15.804809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:15.807033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:15.808601 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-795c7111-cf8f-416f-a144-c719d5adabe1 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.476445 Oct 15 21:02:15.809180 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1970/3084] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:15 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:15.813633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8cfcdd4-cf64-4e17-8261-7ba49eb7bc78 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe74eeeee50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:16.484601 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8cfcdd4-cf64-4e17-8261-7ba49eb7bc78 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/82b07114-cbd9-48cc-9278-7b5b7b24a702" status: 202 len: 0 microversion: 2.1 time: 0.674331 Oct 15 21:02:16.489506 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1971/3085] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:02:15 2022] DELETE /compute/v2.1/os-security-groups/82b07114-cbd9-48cc-9278-7b5b7b24a702 => generated 0 bytes in 679 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:02:16.495241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:16.495885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:16.515005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:16.516638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:16.517472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:16.518478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:16.519106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:16.519808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:16.894559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:16.903049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:16.906880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:16.908887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:16.910372 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30cadff2-64e7-4141-a016-53519089285e tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.419602 Oct 15 21:02:16.914207 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1972/3086] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:16 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:16.931531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:16.932408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:16.940728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:16.942049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:16.942686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:16.944096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:16.945271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:16.946022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:17.480471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:17.494695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7602fd11-12ce-401a-a2e4-7263a1b2f7e8 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 200 len: 1494 microversion: 2.1 time: 0.578318 Oct 15 21:02:17.497930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1973/3087] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:16 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 1494 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:17.511870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:17.513750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:17.541364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:17.544157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:17.545615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:17.547914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:17.548631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:17.551035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:18.089023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:18.101099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:18.106239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:18.110771 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6f4962c6-1cc6-4c63-acf4-001043d60bb8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.610163 Oct 15 21:02:18.126937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1974/3088] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:17 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:18.140087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-80cb15bf-4e6d-4a8c-bfa5-e9f37261e313 req-9ffb59e6-96f6-48c3-ab4f-879623073b0d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c8032d6f-ff10-4add-be09-6de0bbdcf98f", "name": "network-vif-deleted", "tag": "f4da8690-5405-4369-97d1-4c91b63a3c57"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:18.160269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-80cb15bf-4e6d-4a8c-bfa5-e9f37261e313 req-9ffb59e6-96f6-48c3-ab4f-879623073b0d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:18.164133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-80cb15bf-4e6d-4a8c-bfa5-e9f37261e313 req-9ffb59e6-96f6-48c3-ab4f-879623073b0d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:18.166095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-80cb15bf-4e6d-4a8c-bfa5-e9f37261e313 req-9ffb59e6-96f6-48c3-ab4f-879623073b0d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:18.195680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-80cb15bf-4e6d-4a8c-bfa5-e9f37261e313 req-9ffb59e6-96f6-48c3-ab4f-879623073b0d service nova] Creating event network-vif-deleted:f4da8690-5405-4369-97d1-4c91b63a3c57 for instance c8032d6f-ff10-4add-be09-6de0bbdcf98f on nch2-master-1 Oct 15 21:02:18.201527 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-80cb15bf-4e6d-4a8c-bfa5-e9f37261e313 req-9ffb59e6-96f6-48c3-ab4f-879623073b0d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071970 Oct 15 21:02:18.206581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1975/3089] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:18.212261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:18.213287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] [instance: c8032d6f-ff10-4add-be09-6de0bbdcf98f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:18.222045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:18.222996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:18.223699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:18.224655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:18.225508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:18.226156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:18.249571 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] HTTP exception thrown: Instance c8032d6f-ff10-4add-be09-6de0bbdcf98f could not be found. Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Request method failure captured: Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f HTTP/1.1 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9_fHQVJ85WBd3l_m4euhtG4cgODYMwB9qe_LsLrsZyd7RcR_k3x7moGPwq6DGXl7w2IR101-eyun8Uu0_Ld5-Ju9CovfjJp6f0I33yl9rfVukwnW54aMbElqpH1uWjq-K9S-k8io8QIvn_sR1i2_Mc6BljPJGTEWvtcM2hg8Hfut4 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e91f96ee17114009a55fe2bf388a7aa7 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1372966968 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e91f96ee17114009a55fe2bf388a7aa7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1372966968 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e91f96ee17114009a55fe2bf388a7aa7 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1372966968 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-1372966968-project Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 573c02d502744876babbdb4fbb4fb9e6 Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1372966968-project Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c8032d6f-ff10-4add-be09-6de0bbdcf98f could not be found. Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c8032d6f-ff10-4add-be09-6de0bbdcf98f'} Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c8032d6f-ff10-4add-be09-6de0bbdcf98f could not be found. Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c8032d6f-ff10-4add-be09-6de0bbdcf98f could not be found. Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:18.251644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c8032d6f-ff10-4add-be09-6de0bbdcf98f could not be found. Oct 15 21:02:18.255627 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:18.258296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] Returning 404 to user: Instance c8032d6f-ff10-4add-be09-6de0bbdcf98f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:18.259564 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a66c907a-75dd-4156-b33a-309024bf056c tempest-ServersAdminNegativeTestJSON-1372966968 tempest-ServersAdminNegativeTestJSON-1372966968-project] 10.222.0.16 "GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f" status: 404 len: 111 microversion: 2.1 time: 0.051696 Oct 15 21:02:18.264146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1976/3090] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:18 2022] GET /compute/v2.1/servers/c8032d6f-ff10-4add-be09-6de0bbdcf98f => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:18.287030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e113d699-a0c7-454d-8c6b-c1c42afa5c70 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] Calling method '.version_select at 0x7fe758344670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:19.975959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e113d699-a0c7-454d-8c6b-c1c42afa5c70 tempest-SecurityGroupRulesNegativeTestJSON-169800727 tempest-SecurityGroupRulesNegativeTestJSON-169800727-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/bd188229-5236-40ac-a0c7-33c0ad61f297" status: 202 len: 0 microversion: 2.1 time: 1.708951 Oct 15 21:02:19.980732 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1977/3091] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:02:18 2022] DELETE /compute/v2.1/os-security-groups/bd188229-5236-40ac-a0c7-33c0ad61f297 => generated 0 bytes in 1714 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:02:19.985690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:19.986311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:20.007789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:20.017714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:20.022201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:20.025776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:20.028541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:20.029698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:20.620141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:20.645953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:20.647727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:20.650240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77ca075e-2006-4501-b95b-3354715f93eb tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.667979 Oct 15 21:02:20.652357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1978/3092] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:19 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:20.684392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "79d61df5-38a3-4192-a9e3-ed2c945abba0"}, {"server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "name": "network-vif-plugged", "status": "completed", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}, {"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "142ac24e-4393-42c7-a3ab-d8d47ad61698"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:20.704656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:20.705578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:20.706444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:20.726584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Creating event network-vif-plugged:79d61df5-38a3-4192-a9e3-ed2c945abba0 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:20.727733 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Creating event network-vif-plugged:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:02:20.728624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] Creating event network-vif-plugged:142ac24e-4393-42c7-a3ab-d8d47ad61698 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:20.745811 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-46580d29-a0b8-4d6d-8633-d93102a283d2 req-cc3d96b6-0a77-4c9e-b192-9c4bde6d477a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.088296 Oct 15 21:02:20.746454 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1979/3093] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:20 2022] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:21.326453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV257Test-server-768217058", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:21.421778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:02:21.422203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:02:21.428636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:02:21.434271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:02:21.434784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:02:21.465696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:21.467622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:21.467622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:21.467875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:21.468161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:21.483108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:21.528896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Getting quotas for project 26a4431be325443c85b0ecef861d42ff. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:02:21.545879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Getting quotas for user c50ab103943e4e3bacdc2777892c3bfe and project 26a4431be325443c85b0ecef861d42ff. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:02:21.567970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:02:21.572660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:02:21.678276 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-560183b4-b6ef-4168-bc52-9f3bec572189 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.57 time: 0.927738 Oct 15 21:02:21.681470 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1980/3094] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:02:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 934 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:02:21.699558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:21.700056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:21.718816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:21.733548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:21.734095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:21.734864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:21.735347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:21.735784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:22.408708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:22.421338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:22.434134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:22.437384 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9de1645-1445-410e-8ea4-0969c19972d6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.749799 Oct 15 21:02:22.439053 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1981/3095] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:21 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:22.444471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:22.445422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:22.457239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:22.458129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:22.458823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:22.459802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:22.460593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:22.466722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:22.511673 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a HTTP/1.1 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '64e679aa-559b-4d18-9d8f-2b69f97d4a2a'} Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:22.518123 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:22.525707 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:22.534115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:22.536135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0aa1a9c4-6675-4738-83dd-b25d2cd8593b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 404 len: 111 microversion: 2.1 time: 0.095510 Oct 15 21:02:22.547623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1982/3096] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:22 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 111 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:22.567588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:22.573636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:22.596681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:22.603956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:22.604417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:22.608130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:22.612231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:22.615212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:22.980787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.003926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.006417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.007961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7d5b631-1048-4004-b2a4-8bf7f5d7d8f3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.458012 Oct 15 21:02:23.012073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1983/3097] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:22 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:23.017640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-92557bab-146c-4551-bf16-6f4790f94dbc req-604dc9cd-2bbb-4023-9337-4dee3c26a9fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64e679aa-559b-4d18-9d8f-2b69f97d4a2a", "name": "network-vif-deleted", "tag": "525185da-02e4-46c2-a615-abe5d1963ad9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:23.030950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-92557bab-146c-4551-bf16-6f4790f94dbc req-604dc9cd-2bbb-4023-9337-4dee3c26a9fd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.031718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-92557bab-146c-4551-bf16-6f4790f94dbc req-604dc9cd-2bbb-4023-9337-4dee3c26a9fd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.032310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-92557bab-146c-4551-bf16-6f4790f94dbc req-604dc9cd-2bbb-4023-9337-4dee3c26a9fd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.046617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-92557bab-146c-4551-bf16-6f4790f94dbc req-604dc9cd-2bbb-4023-9337-4dee3c26a9fd service nova] Creating event network-vif-deleted:525185da-02e4-46c2-a615-abe5d1963ad9 for instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a on nch1-master-1 Oct 15 21:02:23.052670 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-92557bab-146c-4551-bf16-6f4790f94dbc req-604dc9cd-2bbb-4023-9337-4dee3c26a9fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038969 Oct 15 21:02:23.059770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1984/3098] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:23.066255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:23.067086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:23.082195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.082713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.083164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.083903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.084379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.084811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.520838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.543655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.547902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.552833 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec826218-d5b9-4d10-8f75-8aaa03239003 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.487813 Oct 15 21:02:23.554382 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1985/3099] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:23 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:23.565305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:23.566829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:23.586843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.587753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.588443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.590042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.592000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.595377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.759133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.772745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.776250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:23.782847 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ecf327e-7fec-4abd-8877-35938dea056d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1329 microversion: 2.57 time: 0.223816 Oct 15 21:02:23.785953 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1986/3100] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:23 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1329 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:23.804816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:23.805634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:23.818905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.819699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.820303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.821249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.821882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.825292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.859542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a HTTP/1.1 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '64e679aa-559b-4d18-9d8f-2b69f97d4a2a'} Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:23.861442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:23.867188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:23.869692 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a46f45b-2fb9-4212-a141-04bee59a153b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 404 len: 111 microversion: 2.1 time: 0.081809 Oct 15 21:02:23.870723 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1987/3101] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:23 2022] DELETE /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:23.876675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "a3ed0c87-6f02-402a-a80b-a011756b969c"}, {"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "fd2bd5a8-8477-474b-8090-dc55b657fa33"}, {"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "d5ed5624-9ab3-4ee7-8a25-54a7b3d3bdef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:23.910225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.921750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.922362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.950808 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Creating event network-vif-plugged:a3ed0c87-6f02-402a-a80b-a011756b969c for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:23.951687 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Creating event network-vif-plugged:fd2bd5a8-8477-474b-8090-dc55b657fa33 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:23.953016 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] Creating event network-vif-plugged:d5ed5624-9ab3-4ee7-8a25-54a7b3d3bdef for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:23.966593 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-176cdc08-7401-445f-badd-1ff5fdbb0f81 req-2670e8c5-aff3-4a77-9776-ef08bfa61e49 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.094514 Oct 15 21:02:23.967785 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1988/3102] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:23 2022] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:23.973759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:23.974831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:23.985613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.986379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.987200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:23.988161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:23.988995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:23.990086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:24.285631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:24.313243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:24.319977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:24.323427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79a8ff5c-44ad-455c-9f21-727c45af25a3 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.353948 Oct 15 21:02:24.330649 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1989/3103] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:23 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:24.790527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb689d24-12db-458e-a3f2-8fd59116f1a8 tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:24.810836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb689d24-12db-458e-a3f2-8fd59116f1a8 tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.478700 Oct 15 21:02:24.812018 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1990/3104] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 21:02:24 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:24.826450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:24.829458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 64e679aa-559b-4d18-9d8f-2b69f97d4a2a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:24.845268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:24.848295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:24.849857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:24.853872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:24.854607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:24.855323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:24.880467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a HTTP/1.1 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '64e679aa-559b-4d18-9d8f-2b69f97d4a2a'} Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:24.896544 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. Oct 15 21:02:24.900841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:24.905271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 64e679aa-559b-4d18-9d8f-2b69f97d4a2a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:24.906481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90016e02-0142-4f33-9cab-5fcc63e0bdb2 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a" status: 404 len: 111 microversion: 2.1 time: 0.090741 Oct 15 21:02:24.910475 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1991/3105] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:24 2022] GET /compute/v2.1/servers/64e679aa-559b-4d18-9d8f-2b69f97d4a2a => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:24.917220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:24.918275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:24.927468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:24.928856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:24.929579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:24.931592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:24.932304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:24.933530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:25.407172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:25.423484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:25.426487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:25.431675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70075c4a-fa39-4aa1-958a-4df63c05bb74 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.519718 Oct 15 21:02:25.432288 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1992/3106] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:24 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:25.469160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:25.479729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:25.505460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:25.506268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:25.508150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:25.510033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:25.515082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:25.515852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:25.888236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:25.912007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:25.916178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:25.918725 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3bb8f6b4-8902-4b79-a182-8240ac56ddb5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.457070 Oct 15 21:02:25.933067 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1993/3107] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:25 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:25.933067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:25.933369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:25.945278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:25.947317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:25.961942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:25.965839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:25.968599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:25.972004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:26.166120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:26.186702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:26.205696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:26.207311 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7064973b-e41e-4b6a-a2f8-9716b121399d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1335 microversion: 2.57 time: 0.284057 Oct 15 21:02:26.210113 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1994/3108] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:25 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1335 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:26.248155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:26.252582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:26.266294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:26.270666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:26.271928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:26.273189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:26.274229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:26.276748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:26.311747 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 HTTP/1.1 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '60051f47-77db-45a6-9e66-c0b1b1528667'} Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:26.314265 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:26.318188 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:26.331479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:26.334202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2efecf5b-91b2-4026-90b0-de13d40d1477 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 404 len: 111 microversion: 2.1 time: 0.121566 Oct 15 21:02:26.335026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1995/3109] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:26 2022] DELETE /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 111 bytes in 124 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:26.340575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:26.341551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:26.382177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:26.384290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:26.391984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:26.392924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:26.393715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:26.395216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:26.938481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:26.961644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:26.967464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:26.968907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-449587c1-8590-4248-944e-01d014ce0784 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.632503 Oct 15 21:02:26.970348 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1996/3110] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:26 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:26.981094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cddf4a62-1a64-480c-b277-132ac71559b8 req-a95a67f3-be58-4aa4-b9f4-59f5eeaecbf4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-plugged", "status": "completed", "tag": "d1754b9c-eda8-41d8-99a6-7cc5c3b7c3eb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:26.993065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cddf4a62-1a64-480c-b277-132ac71559b8 req-a95a67f3-be58-4aa4-b9f4-59f5eeaecbf4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:26.994065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cddf4a62-1a64-480c-b277-132ac71559b8 req-a95a67f3-be58-4aa4-b9f4-59f5eeaecbf4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:26.994782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cddf4a62-1a64-480c-b277-132ac71559b8 req-a95a67f3-be58-4aa4-b9f4-59f5eeaecbf4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:27.008714 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cddf4a62-1a64-480c-b277-132ac71559b8 req-a95a67f3-be58-4aa4-b9f4-59f5eeaecbf4 service nova] Creating event network-vif-plugged:d1754b9c-eda8-41d8-99a6-7cc5c3b7c3eb for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:27.017996 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cddf4a62-1a64-480c-b277-132ac71559b8 req-a95a67f3-be58-4aa4-b9f4-59f5eeaecbf4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046099 Oct 15 21:02:27.019362 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1997/3111] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:27.026738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:27.027759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 60051f47-77db-45a6-9e66-c0b1b1528667] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:27.036265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:27.038133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:27.038834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:27.040575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:27.041522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:27.042310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:27.064825 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 HTTP/1.1 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '60051f47-77db-45a6-9e66-c0b1b1528667'} Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:27.066836 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. Oct 15 21:02:27.071161 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.073160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 60051f47-77db-45a6-9e66-c0b1b1528667 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:27.074792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e24be0f5-36d5-48d1-a069-993c5370e2e7 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667" status: 404 len: 111 microversion: 2.1 time: 0.053628 Oct 15 21:02:27.079574 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1998/3112] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:27 2022] GET /compute/v2.1/servers/60051f47-77db-45a6-9e66-c0b1b1528667 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:27.087029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:27.088771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:27.098073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:27.098845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:27.099350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:27.100240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:27.100851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:27.101427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:27.139859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:27.414234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:27.432858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:27.440851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:27.447156 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ad229bb-2361-4870-ba22-363fc8bd90d7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.361468 Oct 15 21:02:27.448003 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 1999/3113] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:27 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:27.464137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15328f5a-0f3a-4bc8-b244-8b7595c64cec tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FlavorsV2NegativeTest-server-1565898376", "imageRef": "8ef889e1-b51f-4294-8433-f4dcccfe0451", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:27.546078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-15328f5a-0f3a-4bc8-b244-8b7595c64cec tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:02:27.791516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-15328f5a-0f3a-4bc8-b244-8b7595c64cec tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] HTTP exception thrown: Flavor's memory is too small for requested image. Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15328f5a-0f3a-4bc8-b244-8b7595c64cec tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] Request method failure captured: Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 143 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_fXWuFFy1Q6gpNRRV65lBLNSZX2MFS4MCwf_dBjk4Ac98bbL4Y0dS9Bi0fYv-DGIX0s912kZch0erBucKu8BfmSSls0dCp1Ngq9hzxL7vorpC2IK96nvotxMj2CACfllpOHoxXFvcLHzBtu-rPfdGAGFmfL9-M_zOkgCIIcv7XL58 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 5430106b642d476ebfe6866f130b3a56 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FlavorsV2NegativeTest-928759552 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5430106b642d476ebfe6866f130b3a56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5430106b642d476ebfe6866f130b3a56"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5430106b642d476ebfe6866f130b3a56"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5430106b642d476ebfe6866f130b3a56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FlavorsV2NegativeTest-928759552 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 5430106b642d476ebfe6866f130b3a56 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FlavorsV2NegativeTest-928759552 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FlavorsV2NegativeTest-928759552-project Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 0db8e2fbd08c4b658e95dd345cae7fb7 Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FlavorsV2NegativeTest-928759552-project Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-FlavorsV2NegativeTest-server-1565898376", "imageRef": "8ef889e1-b51f-4294-8433-f4dcccfe0451", "flavorRef": "452"}} Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: exception: Flavor's memory is too small for requested image. Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-FlavorsV2NegativeTest-server-1565898376', 'imageRef': '8ef889e1-b51f-4294-8433-f4dcccfe0451', 'flavorRef': '452'}}} Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1724, in _create_instance Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._checks_for_create_and_rebuild(context, image_id, boot_meta, Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1019, in _checks_for_create_and_rebuild Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._validate_flavor_image(context, image_id, image, Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 683, in _validate_flavor_image Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._validate_flavor_image_nostatus( Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 751, in _validate_flavor_image_nostatus Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FlavorMemoryTooSmall() Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FlavorMemoryTooSmall: Flavor's memory is too small for requested image. Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:02:27.793362 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 868, in create Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor's memory is too small for requested image. Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor's memory is too small for requested image. Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15328f5a-0f3a-4bc8-b244-8b7595c64cec tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] Returning 400 to user: Flavor's memory is too small for requested image. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:27.798421 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15328f5a-0f3a-4bc8-b244-8b7595c64cec tempest-FlavorsV2NegativeTest-928759552 tempest-FlavorsV2NegativeTest-928759552-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 93 microversion: 2.1 time: 0.345662 Oct 15 21:02:27.810734 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2000/3114] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:02:27 2022] POST /compute/v2.1/servers => generated 93 bytes in 362 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 21:02:27.819274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:27.821634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:27.847066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:27.847565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:27.848004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:27.848720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:27.849210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:27.849641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:28.625309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:28.641474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:28.644896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:28.649692 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87369a03-cdb5-4a76-8994-5b6a2e830766 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.837823 Oct 15 21:02:28.651300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2001/3115] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:27 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:28.660677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:28.665779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:28.674720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:28.678628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:28.679416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:28.680429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:28.681927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:28.684865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:28.732576 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 HTTP/1.1 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1910f53a-ef2a-4f63-836d-9850556d1762'} Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:28.734556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:28.738893 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:28.744016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:28.746741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-045ed54c-3d6a-4c99-8dca-cf314c11e29b tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 404 len: 111 microversion: 2.1 time: 0.089844 Oct 15 21:02:28.749157 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2002/3116] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:28 2022] DELETE /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:28.758426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:28.761902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:28.781924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:28.782829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:28.783496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:28.784438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:28.785178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:28.787110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:28.961978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:28.972008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:28.978387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:28.981653 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecd65db9-7922-4d27-a249-9a5856383553 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1356 microversion: 2.57 time: 0.227096 Oct 15 21:02:28.997535 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2003/3117] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:28 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1356 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:29.012163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:29.013056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:29.020376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:29.021274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:29.021937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:29.022891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:29.023725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:29.024419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:29.921738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:29.936266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:29.937805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:29.939009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe87146b-dcf3-4c12-aaae-3e26cce530d4 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1879 microversion: 2.42 time: 0.940058 Oct 15 21:02:29.939573 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2004/3118] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:28 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1879 bytes in 941 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:29.960342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:29.965839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:29.984654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:29.988083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:29.988752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:29.991637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:29.992923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:29.994449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:30.352294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:30.366896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:30.391424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:30.393227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce2fec31-e5c2-4e25-ab5b-0279f5790eeb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.449279 Oct 15 21:02:30.418704 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2005/3119] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:29 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:30.429514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:30.434756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 1910f53a-ef2a-4f63-836d-9850556d1762] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:30.450106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:30.451883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:30.452513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:30.453451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:30.454085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:30.454685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:30.475032 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 HTTP/1.1 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1910f53a-ef2a-4f63-836d-9850556d1762'} Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:30.480289 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 1910f53a-ef2a-4f63-836d-9850556d1762 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c31bb5d-cd5d-412c-aeb3-3c05703692e0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762" status: 404 len: 111 microversion: 2.1 time: 0.063151 Oct 15 21:02:30.485208 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2006/3120] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:30 2022] GET /compute/v2.1/servers/1910f53a-ef2a-4f63-836d-9850556d1762 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:30.496553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Action: 'action', calling method: >, body: {"rebuild": {"user_data": "ZWNobyAiaGVsbG8gd29ybGQi", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:30.500039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:30.516723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:30.517439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:30.517878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:30.518593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:30.520991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:30.523592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:30.765471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:02:30.766816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:02:30.768611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:02:30.770215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:02:30.772976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:02:30.933668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:30.941364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:30.942033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:30.943437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:30.946597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:30.947471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:30.954721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.022652 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c2492df-015a-460f-b30e-eb56c93a8ae6 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "POST /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d/action" status: 202 len: 1137 microversion: 2.57 time: 0.530545 Oct 15 21:02:31.024540 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2007/3121] 10.222.0.16 () {62 vars in 1320 bytes} [Sat Oct 15 21:02:30 2022] POST /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d/action => generated 1137 bytes in 539 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Oct 15 21:02:31.030042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-33a5f79e-ac47-45e4-9afc-441cfa44f9f0 req-aebab00e-8370-4041-af87-5de69ab89387 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "6e28f7f5-2809-45be-9dd9-d05f11e48e6f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:31.063857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-33a5f79e-ac47-45e4-9afc-441cfa44f9f0 req-aebab00e-8370-4041-af87-5de69ab89387 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.064450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-33a5f79e-ac47-45e4-9afc-441cfa44f9f0 req-aebab00e-8370-4041-af87-5de69ab89387 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.065121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-33a5f79e-ac47-45e4-9afc-441cfa44f9f0 req-aebab00e-8370-4041-af87-5de69ab89387 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.085822 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-33a5f79e-ac47-45e4-9afc-441cfa44f9f0 req-aebab00e-8370-4041-af87-5de69ab89387 service nova] Creating event network-vif-plugged:6e28f7f5-2809-45be-9dd9-d05f11e48e6f for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:31.094987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-33a5f79e-ac47-45e4-9afc-441cfa44f9f0 req-aebab00e-8370-4041-af87-5de69ab89387 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068630 Oct 15 21:02:31.097779 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2008/3122] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:31.116678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:31.119082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:31.125252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:UPDATE=1,nova_api:SELECT=96,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:31.129709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.130435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.131080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.132051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.132713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.133426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.501367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:31.558166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:31.622924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:31.637628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:31.639188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:31.640566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6047796a-3b31-424d-b384-18686c7b5243 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.540590 Oct 15 21:02:31.645398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2009/3123] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:31 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:31.660031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:31.660541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:31.671018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.671589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.672095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.672882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.673419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.673850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.703557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 HTTP/1.1 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '87c2e279-0350-4f47-b7f3-68c0054c0100'} Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:31.724579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9293258-2822-47b8-b577-e4b51c2598a0 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 404 len: 111 microversion: 2.1 time: 0.080038 Oct 15 21:02:31.731314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2010/3124] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:31 2022] DELETE /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:31.735298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:31.735918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:31.766854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.767660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.768324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:31.769371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:31.770082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:31.770761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:32.573138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:32.589324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:32.595188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:32.599335 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4221ecc8-55da-4a1e-9d58-2819a61f7117 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2838 microversion: 2.42 time: 0.870226 Oct 15 21:02:32.600991 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2011/3125] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:31 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2838 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:32.612962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:32.618016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:32.629764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:32.632013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:32.634784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:32.635663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:32.636148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:32.636708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:32.773342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:32.783383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:32.798981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:32.800756 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-712d9a70-c687-468d-9465-e50787d9ec61 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.197313 Oct 15 21:02:32.801952 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2012/3126] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:32 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:32.808011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:32.808843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:32.825347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:32.829068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:32.830065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:32.832256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:32.834439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:32.836142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:33.598921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:33.613210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:33.618482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:33.620057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21c06ee5-7bc6-4a74-85bc-0af3ad06cf0e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.816792 Oct 15 21:02:33.623211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2013/3127] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:32 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:33.630812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:33.633457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 87c2e279-0350-4f47-b7f3-68c0054c0100] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:33.649820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:33.650599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:33.651261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:33.652209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:33.652956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:33.653659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:33.674695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 HTTP/1.1 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '87c2e279-0350-4f47-b7f3-68c0054c0100'} Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:33.676937 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. Oct 15 21:02:33.681361 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:33.685781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 87c2e279-0350-4f47-b7f3-68c0054c0100 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:33.686753 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-265fc4c5-0225-4623-9c26-d2d2ad550891 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100" status: 404 len: 111 microversion: 2.1 time: 0.062330 Oct 15 21:02:33.687624 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2014/3128] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:33 2022] GET /compute/v2.1/servers/87c2e279-0350-4f47-b7f3-68c0054c0100 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:33.703795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:33.704852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:33.713634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:33.714573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:33.715285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:33.716593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:33.717333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:33.717997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:33.940308 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-126ff2d6-a238-4811-8538-352ac9e3f74f tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d/os-interface" status: 200 len: 1832 microversion: 2.42 time: 0.251724 Oct 15 21:02:33.941763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2015/3129] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 21:02:33 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d/os-interface => generated 1832 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:33.957151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:33.958304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:33.972246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:33.973074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:33.973771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:33.974732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:33.975410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:33.976084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.405933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:34.422093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:34.428702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:34.429734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b45c7548-5f98-47b7-bf61-e4afaf96cca3 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.486728 Oct 15 21:02:34.433357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2016/3130] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:33 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:34.449802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "675ba1c4-0b55-4936-9005-97673bfc6b22"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "97f99e08-5287-4631-83bc-e4e52565d3e4"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "f425e84b-b112-4afe-9505-a001d8f8ce16"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:34.470906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.471926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.472597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.515796 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Creating event network-vif-plugged:675ba1c4-0b55-4936-9005-97673bfc6b22 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:34.519693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Creating event network-vif-plugged:97f99e08-5287-4631-83bc-e4e52565d3e4 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:34.520856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] Creating event network-vif-plugged:f425e84b-b112-4afe-9505-a001d8f8ce16 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:34.537154 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d08b18da-a1a9-41b7-8f52-65328307a111 req-6b24ea5d-e131-4dc3-81ff-30589097bd52 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.099930 Oct 15 21:02:34.538298 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2017/3131] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:34 2022] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:34.555550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:34.557425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:34.576137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.577022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.581600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.581600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.582881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.583576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.626822 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 HTTP/1.1 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8712cdbc-97cd-492a-865d-81a3d542b7a3'} Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:34.630594 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.636128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.639907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:34.643137 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-855e26d8-f673-4774-901a-227104b13574 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 404 len: 111 microversion: 2.1 time: 0.101851 Oct 15 21:02:34.645203 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2018/3132] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:34 2022] DELETE /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 111 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:34.652812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:34.658371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:34.676979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.685789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.686567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.687524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.688595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.689447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.813832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:34.828336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:34.830161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:34.836625 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b2158e6-715f-4466-8497-dc1fb0200a64 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.189691 Oct 15 21:02:34.837482 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2019/3133] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:34 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:34.855404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:34.856341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 8712cdbc-97cd-492a-865d-81a3d542b7a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:34.866851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.870871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.871543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.872502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.873189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.873813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.905667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:34.908957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 HTTP/1.1 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8712cdbc-97cd-492a-865d-81a3d542b7a3'} Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:34.910610 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 8712cdbc-97cd-492a-865d-81a3d542b7a3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-865b7bfe-f33e-4735-a08f-2ecc38c4a072 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3" status: 404 len: 111 microversion: 2.1 time: 0.073719 Oct 15 21:02:34.915164 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2020/3134] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:34 2022] GET /compute/v2.1/servers/8712cdbc-97cd-492a-865d-81a3d542b7a3 => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:34.920500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:34.921351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:34.931968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.932495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.933224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:34.934125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:34.934608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:34.935053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:35.551846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:35.564617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:35.566121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:35.567338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e90adec-09ae-4118-9661-ee5cf4913f63 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.653131 Oct 15 21:02:35.567907 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2021/3135] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:34 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:35.579075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:35.583158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:35.598565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:35.605856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:35.606593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:35.607528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:35.608174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:35.608765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:35.650675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 HTTP/1.1 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd38ead8c-d325-4fda-8df9-acff8948f988'} Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:35.653298 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:35.657551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:35.659567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:35.660806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ef4cc66-39d8-4698-bfa9-59eabaceee7e tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 404 len: 111 microversion: 2.1 time: 0.091807 Oct 15 21:02:35.662324 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2022/3136] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:35 2022] DELETE /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:35.669392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:35.670305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:35.677617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:35.678336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:35.678985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:35.681181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:35.682726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:35.683393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.419043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:36.430757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:36.433309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:36.442559 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-292e6b13-e7aa-4a1d-a4be-ee8c025bd968 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.778981 Oct 15 21:02:36.443316 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2023/3137] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:35 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 780 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:36.462650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:36.463424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: d38ead8c-d325-4fda-8df9-acff8948f988] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:36.474683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.475337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.475903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.476831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.480715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.481461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.512260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 HTTP/1.1 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd38ead8c-d325-4fda-8df9-acff8948f988'} Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:36.515944 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. Oct 15 21:02:36.520837 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.526226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance d38ead8c-d325-4fda-8df9-acff8948f988 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:36.527406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6923065-501a-40d8-8805-9e35c5306552 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988" status: 404 len: 111 microversion: 2.1 time: 0.082930 Oct 15 21:02:36.528275 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2024/3138] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:36 2022] GET /compute/v2.1/servers/d38ead8c-d325-4fda-8df9-acff8948f988 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:36.535606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:36.541893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:36.556056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.556802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.557543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.560694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.561448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.573182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.780053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:36.792001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:36.796060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:36.804089 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b2ec08e-50d5-4225-af14-4a62e406071d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.274502 Oct 15 21:02:36.817476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2025/3139] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:36 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:36.834252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:36.845856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:36.857013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.857872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.858536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.859501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.860198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.860846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.878895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 HTTP/1.1 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '352695e9-705f-447b-b213-141c9512ce65'} Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:36.881947 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:36.885825 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:36.888726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:36.889874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa84d2d7-b819-4eb9-a969-541c3e3c5e7c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 404 len: 111 microversion: 2.1 time: 0.070938 Oct 15 21:02:36.890975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2026/3140] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:36 2022] DELETE /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:36.896560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "5621f2f9-6408-416e-9e9b-6029170044f4"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "2a2ab61a-72e6-4577-a7e1-d1b4987a2389"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-plugged", "status": "completed", "tag": "c874ec72-d847-4302-a82c-1a15a1e24a5f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:36.907390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.908055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.908692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.924080 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Creating event network-vif-plugged:5621f2f9-6408-416e-9e9b-6029170044f4 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:36.924718 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Creating event network-vif-plugged:2a2ab61a-72e6-4577-a7e1-d1b4987a2389 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:36.925679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] Creating event network-vif-plugged:c874ec72-d847-4302-a82c-1a15a1e24a5f for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:36.932783 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cf982a86-bfe2-43b9-a115-ddf83a856228 req-bd029e2d-cb7a-4613-9f38-3181d1daa004 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.040397 Oct 15 21:02:36.933581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2027/3141] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:36 2022] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:36.945588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:36.946364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:36.970237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.971754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.972552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:36.973111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:36.973658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:36.974185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:37.290568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:37.304940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:37.306621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:37.307913 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5f2892b-9f01-4998-98f8-e58a52ab8068 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1741 microversion: 2.20 time: 0.372687 Oct 15 21:02:37.325682 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2028/3142] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:36 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1741 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:37.337380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:37.338015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 352695e9-705f-447b-b213-141c9512ce65] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:37.345666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:37.346176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:37.346626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:37.347359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:37.347828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:37.348265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:37.381955 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 HTTP/1.1 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '352695e9-705f-447b-b213-141c9512ce65'} Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:37.383732 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 352695e9-705f-447b-b213-141c9512ce65 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:37.389556 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b062fe07-2d21-4c34-898e-39ea0a0cf16c tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65" status: 404 len: 111 microversion: 2.1 time: 0.058339 Oct 15 21:02:37.397459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2029/3143] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:37 2022] GET /compute/v2.1/servers/352695e9-705f-447b-b213-141c9512ce65 => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:37.403119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:37.403634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:37.426772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:37.427553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:37.428168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:37.429100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:37.429916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:37.430542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:37.489276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:02:37.702340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-626286fe-9d3a-4b33-8d9e-617deeb3d532 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "DELETE /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 204 len: 0 microversion: 2.1 time: 0.303396 Oct 15 21:02:37.706096 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2030/3144] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:02:37 2022] DELETE /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 0 bytes in 305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:02:37.717626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:37.718361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:37.732956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:37.733542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:37.733994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:37.734801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:37.735318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:37.738320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:38.525787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:38.544585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:38.546401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:38.553431 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-045ac85e-f9ad-4460-bc9e-2325863494da tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1874 microversion: 2.32 time: 0.843472 Oct 15 21:02:38.575785 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2031/3145] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:37 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1874 bytes in 869 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:38.583128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:38.583762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:38.603631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:38.604430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:38.605266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:38.606506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:38.607011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:38.609499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:40.436176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6adaa12c2ca9469299994300bc856ac5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44a01230c0fc49a6b60d553d085f3808', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:40.451725 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09d079c1-e0e2-4c67-9342-796e80acdf13 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 200 len: 1480 microversion: 2.1 time: 1.872772 Oct 15 21:02:40.454813 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2032/3146] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:38 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 1480 bytes in 1878 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:40.473956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:40.482851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:40.501528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:40.502356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:40.503526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:40.504964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:40.506130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:40.511436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:40.763707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:40.781114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:40.785370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:40.786910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08c3914f-0587-4e5e-a854-a4c4720eab68 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.328975 Oct 15 21:02:40.788052 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2033/3147] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:40 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:40.802870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:40.810736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:40.831982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:40.832531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:40.832971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:40.843066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:40.844786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:40.846760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:41.422229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:41.448611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:41.453529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:41.459309 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f63906ee-18ef-4b47-aa4e-4be4136c2d94 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1723 microversion: 2.20 time: 0.669277 Oct 15 21:02:41.461527 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2034/3148] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:40 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1723 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:41.479217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d1c39fe2-5ce6-4f09-8e3d-27125a1ebb1f req-a036baf4-1203-4e8a-91c9-e76f8fc5cd92 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "tag": "142ac24e-4393-42c7-a3ab-d8d47ad61698"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:41.498156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d1c39fe2-5ce6-4f09-8e3d-27125a1ebb1f req-a036baf4-1203-4e8a-91c9-e76f8fc5cd92 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:41.498835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d1c39fe2-5ce6-4f09-8e3d-27125a1ebb1f req-a036baf4-1203-4e8a-91c9-e76f8fc5cd92 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:41.499292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d1c39fe2-5ce6-4f09-8e3d-27125a1ebb1f req-a036baf4-1203-4e8a-91c9-e76f8fc5cd92 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:41.531468 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d1c39fe2-5ce6-4f09-8e3d-27125a1ebb1f req-a036baf4-1203-4e8a-91c9-e76f8fc5cd92 service nova] Creating event network-changed:142ac24e-4393-42c7-a3ab-d8d47ad61698 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:02:41.540839 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d1c39fe2-5ce6-4f09-8e3d-27125a1ebb1f req-a036baf4-1203-4e8a-91c9-e76f8fc5cd92 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075563 Oct 15 21:02:41.542137 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2035/3149] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:41.549890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:41.552536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:41.563110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:41.564587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:41.565800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:41.567318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:41.568078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:41.570171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:42.681944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:42.692795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:42.698563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:42.710359 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3cf18b6-d4df-4f0f-83e2-3d242b818042 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2828 microversion: 2.32 time: 1.166698 Oct 15 21:02:42.719103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2036/3150] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:41 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2828 bytes in 1176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:42.725813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7613949e-ee27-41f2-afa8-1986fecf274a req-4940d218-d18d-43b9-a2c5-ae39ec3f4858 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "836a0d11-3560-4b69-bc0c-a3cec35c04fe", "name": "network-vif-deleted", "tag": "a0e4b881-c9f1-4d0d-8f2f-355cc9bbdd08"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:42.758069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7613949e-ee27-41f2-afa8-1986fecf274a req-4940d218-d18d-43b9-a2c5-ae39ec3f4858 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:42.762029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7613949e-ee27-41f2-afa8-1986fecf274a req-4940d218-d18d-43b9-a2c5-ae39ec3f4858 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:42.762750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7613949e-ee27-41f2-afa8-1986fecf274a req-4940d218-d18d-43b9-a2c5-ae39ec3f4858 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:42.807845 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7613949e-ee27-41f2-afa8-1986fecf274a req-4940d218-d18d-43b9-a2c5-ae39ec3f4858 service nova] Creating event network-vif-deleted:a0e4b881-c9f1-4d0d-8f2f-355cc9bbdd08 for instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe on nch2-master-1 Oct 15 21:02:42.822639 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7613949e-ee27-41f2-afa8-1986fecf274a req-4940d218-d18d-43b9-a2c5-ae39ec3f4858 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096672 Oct 15 21:02:42.824912 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2037/3151] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:42.835416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:42.840956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] [instance: 836a0d11-3560-4b69-bc0c-a3cec35c04fe] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:42.879359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:42.883234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:42.887553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:42.888544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:42.889312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:42.889979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:42.932834 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] HTTP exception thrown: Instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe could not be found. Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Request method failure captured: Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe HTTP/1.1 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx56Vma64q6xgphnbb0_N1eudeK2Md9ldPLf3Q3m6XKefE2akNuU7t4dI6h4SkZog6F2AruH4_TMibYv4MkIWf0OSR4uPkYe6LShfXuXcb3NbMz6AUcJu3YSVgBQDcePJI6Hh6IuNTBaAvN2v8FUO960MxCFgmngdOm-DSTb2ro9j-k Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/44a01230c0fc49a6b60d553d085f3808"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestJSON-1301631072 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 44a01230c0fc49a6b60d553d085f3808 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestJSON-1301631072 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6adaa12c2ca9469299994300bc856ac5 Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestJSON-1301631072-project Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe could not be found. Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '836a0d11-3560-4b69-bc0c-a3cec35c04fe'} Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe could not be found. Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe could not be found. Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:02:42.935296 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe could not be found. Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] Returning 404 to user: Instance 836a0d11-3560-4b69-bc0c-a3cec35c04fe could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:02:42.941653 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a9e525a-b3c1-40d2-8b24-2c5fa904dc73 tempest-ServersTestJSON-1301631072 tempest-ServersTestJSON-1301631072-project] 10.222.0.16 "GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe" status: 404 len: 111 microversion: 2.1 time: 0.112256 Oct 15 21:02:42.946234 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2038/3152] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:42 2022] GET /compute/v2.1/servers/836a0d11-3560-4b69-bc0c-a3cec35c04fe => generated 111 bytes in 120 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:02:42.958671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:42.961383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:42.990543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:43.009999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:43.018395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:43.030836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:43.035218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:43.037586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:43.284804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:43.300820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:43.307618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:43.317618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b662f0ba-b57e-4d05-8bb9-997dd974fe6d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.370096 Oct 15 21:02:43.318464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2039/3153] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:42 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:43.327678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-afd5e941-6123-4144-8ac3-6824e6a423ca req-4dfa9981-b667-4c05-9698-9af2630d2ac8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "name": "network-vif-unplugged", "status": "completed", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:43.349288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-afd5e941-6123-4144-8ac3-6824e6a423ca req-4dfa9981-b667-4c05-9698-9af2630d2ac8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:43.349996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-afd5e941-6123-4144-8ac3-6824e6a423ca req-4dfa9981-b667-4c05-9698-9af2630d2ac8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:43.368613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-afd5e941-6123-4144-8ac3-6824e6a423ca req-4dfa9981-b667-4c05-9698-9af2630d2ac8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:43.415891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-afd5e941-6123-4144-8ac3-6824e6a423ca req-4dfa9981-b667-4c05-9698-9af2630d2ac8 service nova] Creating event network-vif-unplugged:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:02:43.432028 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-afd5e941-6123-4144-8ac3-6824e6a423ca req-4dfa9981-b667-4c05-9698-9af2630d2ac8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.109469 Oct 15 21:02:43.433199 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2040/3154] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:43 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:43.769993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-79512112", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "17f4161c-0a61-4650-a009-d339764cbfd5"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:43.863161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] validate_networks() for [('17f4161c-0a61-4650-a009-d339764cbfd5', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:02:45.339344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:02:45.340800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:02:45.341735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:02:45.342442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:02:45.343091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:02:45.361471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:45.364123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:45.365785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:45.368085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:45.369433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:45.371213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:45.436113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:02:45.447782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for user a8af9544fe68497c8557248575f184ed and project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:02:45.481368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:02:45.488810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:02:45.603613 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a5f612e-920d-4674-9ef2-10b2bd6c0846 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.169137 Oct 15 21:02:45.604392 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2041/3155] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:02:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 2171 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:02:45.619512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:45.620128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:45.634495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:45.634991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:45.635432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:45.636145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:45.636606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:45.637179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:45.772461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:45.933537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:45.942952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:45.944530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:45.946492 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc8485a7-f25f-4201-91e5-8055405e8038 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1666 microversion: 2.20 time: 0.341112 Oct 15 21:02:45.964167 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2042/3156] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:45 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1666 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:45.979896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:45.980432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:45.986573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:45.987071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:45.987512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:45.988215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:45.988682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:45.989133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:46.207843 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccfa1fab-9560-4181-9017-1ab1e832e5ff tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d/os-interface" status: 200 len: 1832 microversion: 2.32 time: 0.241941 Oct 15 21:02:46.209056 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2043/3157] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 21:02:45 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d/os-interface => generated 1832 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:46.227402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:46.229238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:46.244127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:46.245532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:46.247940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:46.249260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:46.249983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:46.252062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:46.414903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:46.428725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:46.433725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:46.438688 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f96a1eb-2056-49da-beee-6c366c5fa76f tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.228361 Oct 15 21:02:46.443661 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2044/3158] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:46 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:46.452836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22260fae-03b5-47f9-b635-d4b9513fec99 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:46.457027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22260fae-03b5-47f9-b635-d4b9513fec99 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:46.606709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22260fae-03b5-47f9-b635-d4b9513fec99 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:46.622271 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22260fae-03b5-47f9-b635-d4b9513fec99 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1223 microversion: 2.1 time: 0.177139 Oct 15 21:02:46.636310 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2045/3159] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:46 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1223 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:46.770787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsTestJSON-server-213459499", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "74676419-a7cd-45de-a49c-04aa17d14498"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:46.856514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] validate_networks() for [('74676419-a7cd-45de-a49c-04aa17d14498', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:02:48.136849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:02:48.137541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:02:48.138342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:02:48.138800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:02:48.139510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:02:48.169467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:48.170852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:48.172044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:48.173379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:48.174396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:48.175437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:48.195449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:48.205450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Getting quotas for project da46bd0ec0384f9398a890dcf12deb7a. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:02:48.216286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Getting quotas for user 6efa496c103449e486364cdef764054d and project da46bd0ec0384f9398a890dcf12deb7a. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:02:48.244661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:02:48.248628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:02:48.376578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.738557 Oct 15 21:02:48.379490 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2046/3160] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:02:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 1742 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:02:48.396003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e7f34b49-9f58-47cb-8421-8d029d1523f7"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:48.402297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:48.479802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:48.482653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:48.483158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:48.483939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:48.484439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:48.489388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:49.238263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7a5963a-04db-4b0a-bd3a-888a910d0aed tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.856353 Oct 15 21:02:49.239792 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2047/3161] 10.222.0.16 () {62 vars in 1364 bytes} [Sat Oct 15 21:02:48 2022] POST /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments => generated 188 bytes in 859 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:02:49.404669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationTestJSON-server-493277341", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "3a03a03d-6802-41ab-b356-830701974cbd"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:49.535500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] validate_networks() for [('3a03a03d-6802-41ab-b356-830701974cbd', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:02:51.513750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:02:51.514877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:02:51.515565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:02:51.516083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:02:51.516555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:02:51.532321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:51.534315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:51.534315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:51.539480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:51.540215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:51.540866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:51.583778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Getting quotas for project f278bb578eca4b5895634257d5ac832e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:02:51.600169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Getting quotas for user f2295a9d9b034667957b8e1d3c4901ab and project f278bb578eca4b5895634257d5ac832e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:02:51.658889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:02:51.666185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:02:51.841098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50fb8936-8578-49ad-9918-f0e8cf65fd55 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.599894 Oct 15 21:02:51.842008 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2048/3162] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:02:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 2601 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:02:51.859051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:51.862280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:51.870933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:51.871415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:51.874803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:51.877692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:51.878649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:51.881451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:52.308254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:52.326445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:52.328134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:52.329573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3b78aa4-5dcd-452c-b553-7e170fce3e1e tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.484551 Oct 15 21:02:52.331460 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2049/3163] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:51 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:52.336743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:52.337580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:52.353988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:52.355212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:52.356011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:52.357193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:52.358006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:52.358569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:52.911532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:52.925945 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e430020-e8ff-46e7-9ced-f7762e4e734d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1323 microversion: 2.1 time: 0.591730 Oct 15 21:02:52.944952 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2050/3164] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:52 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1323 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:52.949925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:52.950762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:52.969409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:52.970243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:52.970871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:52.972245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:52.972947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:52.973702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:53.165243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:53.178309 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a635e08-59f0-4b12-89d1-929fda858e70 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1274 microversion: 2.1 time: 0.230637 Oct 15 21:02:53.193904 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2051/3165] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:52 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1274 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:53.219268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:53.221059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:53.232041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:53.232824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:53.235138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:53.236301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:53.236959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:53.238270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:53.375475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=138688) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 21:02:53.616816 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f9c5a96-3d56-4f57-85a7-c816505c7435 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/action" status: 202 len: 0 microversion: 2.20 time: 0.411851 Oct 15 21:02:53.618627 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2052/3166] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 21:02:53 2022] POST /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/action => generated 0 bytes in 416 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:02:53.624536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6afd166c-e7a4-478e-9c08-dd0d491bd490 req-2799c2e4-82c1-46a1-8080-8f91b41c4bdf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "tag": "675ba1c4-0b55-4936-9005-97673bfc6b22"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:53.638715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6afd166c-e7a4-478e-9c08-dd0d491bd490 req-2799c2e4-82c1-46a1-8080-8f91b41c4bdf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:53.639467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6afd166c-e7a4-478e-9c08-dd0d491bd490 req-2799c2e4-82c1-46a1-8080-8f91b41c4bdf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:53.640570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6afd166c-e7a4-478e-9c08-dd0d491bd490 req-2799c2e4-82c1-46a1-8080-8f91b41c4bdf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:53.664999 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6afd166c-e7a4-478e-9c08-dd0d491bd490 req-2799c2e4-82c1-46a1-8080-8f91b41c4bdf service nova] Creating event network-changed:675ba1c4-0b55-4936-9005-97673bfc6b22 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:02:53.674620 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6afd166c-e7a4-478e-9c08-dd0d491bd490 req-2799c2e4-82c1-46a1-8080-8f91b41c4bdf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053581 Oct 15 21:02:53.677397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2053/3167] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:53.749134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:53.756954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:53.780827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:53.782667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:53.783151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:53.785786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:53.786298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:53.786746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:54.255755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:54.267454 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80f23677-5326-45db-9f7d-4ee0c27abb81 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1275 microversion: 2.1 time: 0.588480 Oct 15 21:02:54.274076 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2054/3168] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:53 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1275 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:54.294158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f18d648b-e3e2-40cb-b3ce-5280c73dd002 req-7e4d8116-701b-48e0-b259-15187ff6a38e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b72dcc6-e2fb-40ad-860d-c65f66845edc", "tag": "ac46c3b1-490d-4d03-9d0d-44a3b6128cbd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:54.322138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f18d648b-e3e2-40cb-b3ce-5280c73dd002 req-7e4d8116-701b-48e0-b259-15187ff6a38e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:54.322640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f18d648b-e3e2-40cb-b3ce-5280c73dd002 req-7e4d8116-701b-48e0-b259-15187ff6a38e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:54.323086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f18d648b-e3e2-40cb-b3ce-5280c73dd002 req-7e4d8116-701b-48e0-b259-15187ff6a38e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:54.376983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f18d648b-e3e2-40cb-b3ce-5280c73dd002 req-7e4d8116-701b-48e0-b259-15187ff6a38e service nova] Creating event network-changed:ac46c3b1-490d-4d03-9d0d-44a3b6128cbd for instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc on nch2-master-1 Oct 15 21:02:54.395476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f18d648b-e3e2-40cb-b3ce-5280c73dd002 req-7e4d8116-701b-48e0-b259-15187ff6a38e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110491 Oct 15 21:02:54.398377 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2055/3169] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:54.418451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:54.421623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:54.439663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:54.440928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:54.441522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:54.442323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:54.442859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:54.443637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:54.943754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:54.958345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:54.964706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:54.969249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90e3aa02-922e-4b85-b6c4-226e7ddd0268 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.567398 Oct 15 21:02:54.971674 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2056/3170] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:54 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:54.984837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:54.986270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:55.000424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:55.003224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:55.004153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:55.005171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:55.011666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:55.012481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:56.398491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:56.414886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:56.418645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:56.420175 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b29dc45f-ff2a-4891-bf17-8eeb12b97c76 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1752 microversion: 2.20 time: 1.445797 Oct 15 21:02:56.420962 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2057/3171] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:54 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1752 bytes in 1449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:56.427169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:56.428292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:56.435722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:56.436488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:56.437273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:56.438658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:56.440126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:56.442684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.034582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:57.043510 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c64278d4-96f8-42ff-aed9-3ecd46668b3d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1323 microversion: 2.1 time: 0.620233 Oct 15 21:02:57.044184 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2058/3172] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:56 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1323 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:57.050981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:57.051876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:57.060326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.061451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.062317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.063330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.063997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.064509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.087903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:02:57.461223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:57.472045 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-153f9c1a-e134-4c08-8fc5-82b574c88e08 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.424957 Oct 15 21:02:57.473744 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2059/3173] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:57 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:57.479281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:57.480695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:57.487212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.488053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.488692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.489969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.490439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.491212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.588965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:57.598091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-962bb33a-61b1-4f99-bd16-7b5c49c5fc0b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1281 microversion: 2.1 time: 0.122991 Oct 15 21:02:57.599828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2060/3174] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:57 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1281 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:57.764608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV254Test-server-857177689", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:57.836050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:02:57.836729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:02:57.837204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:02:57.837624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:02:57.838044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:02:57.848858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.849361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.849803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.850588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.851662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.852417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.874089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Getting quotas for project 642b98a4621b4d84b3e1e97137a68d33. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:02:57.880175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Getting quotas for user c7ceb51003c24728b665e7f641e3634b and project 642b98a4621b4d84b3e1e97137a68d33. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:02:57.894464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:02:57.896445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:02:57.951178 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f04c2c9-2dd0-47f2-961d-765b9921ba21 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.54 time: 0.350152 Oct 15 21:02:57.951988 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2061/3175] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:02:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 352 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:02:57.957119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:57.957977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:57.966007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.966763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.967320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:57.968299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:57.968791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:57.969252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.049168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.058019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.059684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.061845 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7c89084-3f98-4e71-970f-eacd4a837331 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1365 microversion: 2.57 time: 0.107609 Oct 15 21:02:58.061845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2062/3176] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:57 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1365 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:58.066531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:58.067159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:58.074063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.074618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.075160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.075989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.076473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.076946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.232354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.242356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.244598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.245879 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c03e606-acea-4f7f-8559-1e04ac86bbd0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.183169 Oct 15 21:02:58.246451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2063/3177] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:58 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:58.251471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-af19212a-09e4-4a32-bddc-f355b98ad3d3 req-5eaa9af4-c12f-40ab-a779-33765170721e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a4f6576d-84bb-45c2-8151-db174cec168a", "tag": "e16ce893-1cb6-4224-8086-e6e24fb3252a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:02:58.262465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-af19212a-09e4-4a32-bddc-f355b98ad3d3 req-5eaa9af4-c12f-40ab-a779-33765170721e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.263024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-af19212a-09e4-4a32-bddc-f355b98ad3d3 req-5eaa9af4-c12f-40ab-a779-33765170721e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.263471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-af19212a-09e4-4a32-bddc-f355b98ad3d3 req-5eaa9af4-c12f-40ab-a779-33765170721e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.276493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-af19212a-09e4-4a32-bddc-f355b98ad3d3 req-5eaa9af4-c12f-40ab-a779-33765170721e service nova] Creating event network-changed:e16ce893-1cb6-4224-8086-e6e24fb3252a for instance a4f6576d-84bb-45c2-8151-db174cec168a on nch1-master-1 Oct 15 21:02:58.282060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-af19212a-09e4-4a32-bddc-f355b98ad3d3 req-5eaa9af4-c12f-40ab-a779-33765170721e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034147 Oct 15 21:02:58.282871 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2064/3178] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:02:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:02:58.288354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdc7ca6d-0941-4abb-a281-99c67b4fb1dd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:58.289471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdc7ca6d-0941-4abb-a281-99c67b4fb1dd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:58.352476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cdc7ca6d-0941-4abb-a281-99c67b4fb1dd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.360399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cdc7ca6d-0941-4abb-a281-99c67b4fb1dd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.361907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cdc7ca6d-0941-4abb-a281-99c67b4fb1dd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.363062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdc7ca6d-0941-4abb-a281-99c67b4fb1dd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1277 microversion: 2.54 time: 0.079037 Oct 15 21:02:58.363634 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2065/3179] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:02:58 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1277 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:02:58.369316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:58.369932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:58.376365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.376938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.377453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.378157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.378622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.379061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.537895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:58.547319 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0d782c8-c028-47cd-83d5-82f9842ec4bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1323 microversion: 2.1 time: 0.182563 Oct 15 21:02:58.548016 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2066/3180] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:58 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1323 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:58.553415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:58.554628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:58.563056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.563766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.564257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:58.565420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:58.565895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:58.566336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:59.217004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:02:59.229279 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fecefc15-f90e-464e-820b-d0c2a3826631 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.679905 Oct 15 21:02:59.237759 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2067/3181] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:58 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:02:59.247840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:02:59.248897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:02:59.259484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:59.261382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:59.262682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:02:59.264441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:02:59.265308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:02:59.265983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:00.023464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:00.035111 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7bf1d3fb-ad81-41a5-8a94-ee9c9e49c760 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.795222 Oct 15 21:03:00.052436 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2068/3182] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:02:59 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:00.060074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:00.060718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:00.070141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:00.072829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:00.073626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:00.083924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:00.086345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:00.088237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:00.416037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:00.425816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:00.427683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:00.429589 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48dd0bc6-a54c-430e-a495-d3f34da0e02b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1371 microversion: 2.57 time: 0.375180 Oct 15 21:03:00.430446 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2069/3183] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:00 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1371 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:00.435165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:00.435789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:00.444988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:00.445722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:00.446465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:00.447313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:00.447787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:00.448226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:01.116879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:01.126854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:01.128667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:01.129996 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-900771ef-f7a3-4072-a7fc-9846d74f6296 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.698499 Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2070/3184] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:00 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 700 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:01.149146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:01.283234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:01.293971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:01.295877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:01.297214 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ed173c1-9ffc-4389-a7df-c8b4f13e6cb0 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1347 microversion: 2.54 time: 0.165650 Oct 15 21:03:01.297903 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2071/3185] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:01 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1347 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:01.302516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:01.303133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:01.309949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:01.310454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:01.311101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:01.311816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:01.312901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:01.314269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:01.988501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:02.005946 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-327dc223-db1d-451c-80c8-acb04ca96d09 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1323 microversion: 2.1 time: 0.706368 Oct 15 21:03:02.006942 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2072/3186] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:01 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1323 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:02.013906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:02.014886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:02.023810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:02.024403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:02.024900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:02.025988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:02.026466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:02.026930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:02.531817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:02.546156 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df013ceb-90d7-4c17-a485-4845c93e61c1 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.537210 Oct 15 21:03:02.547663 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2073/3187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:02 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:02.567707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:02.568340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:02.572554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=9,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:02.578100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:02.578875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:02.579321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:02.580247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:02.580707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:02.581383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.136084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.144601 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb2bc9c9-9917-4e4f-9284-48b36d6fe7ec tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.585194 Oct 15 21:03:03.145594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2074/3188] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:02 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:03.167170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:03.167410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:03.173583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.174523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.174523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.175501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.175719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.176124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.292573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.300932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.302436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.304135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b1042d8-a69d-4374-9f79-432f19d17d67 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1356 microversion: 2.57 time: 0.141422 Oct 15 21:03:03.305424 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2075/3189] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:03 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1356 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:03.309712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c9a1383b-4b62-4543-96e2-3cc79825c569 req-ce638fef-e0fa-48c7-8c9e-179b3399e390 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:03.320665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c9a1383b-4b62-4543-96e2-3cc79825c569 req-ce638fef-e0fa-48c7-8c9e-179b3399e390 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.321165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c9a1383b-4b62-4543-96e2-3cc79825c569 req-ce638fef-e0fa-48c7-8c9e-179b3399e390 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.321601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c9a1383b-4b62-4543-96e2-3cc79825c569 req-ce638fef-e0fa-48c7-8c9e-179b3399e390 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.333590 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c9a1383b-4b62-4543-96e2-3cc79825c569 req-ce638fef-e0fa-48c7-8c9e-179b3399e390 service nova] Creating event network-changed:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:03:03.340057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c9a1383b-4b62-4543-96e2-3cc79825c569 req-ce638fef-e0fa-48c7-8c9e-179b3399e390 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033626 Oct 15 21:03:03.340933 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2076/3190] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:03.347222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f97c225c-7a32-44d2-8f75-4114204db972 req-8b1a5e50-2b36-4b3e-be84-d14aa84a1738 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b72dcc6-e2fb-40ad-860d-c65f66845edc", "name": "network-vif-plugged", "status": "completed", "tag": "ac46c3b1-490d-4d03-9d0d-44a3b6128cbd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:03.363039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f97c225c-7a32-44d2-8f75-4114204db972 req-8b1a5e50-2b36-4b3e-be84-d14aa84a1738 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.363586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f97c225c-7a32-44d2-8f75-4114204db972 req-8b1a5e50-2b36-4b3e-be84-d14aa84a1738 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.365575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f97c225c-7a32-44d2-8f75-4114204db972 req-8b1a5e50-2b36-4b3e-be84-d14aa84a1738 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.380418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f97c225c-7a32-44d2-8f75-4114204db972 req-8b1a5e50-2b36-4b3e-be84-d14aa84a1738 service nova] Creating event network-vif-plugged:ac46c3b1-490d-4d03-9d0d-44a3b6128cbd for instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc on nch2-master-1 Oct 15 21:03:03.390795 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f97c225c-7a32-44d2-8f75-4114204db972 req-8b1a5e50-2b36-4b3e-be84-d14aa84a1738 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048261 Oct 15 21:03:03.391577 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2077/3191] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:03.399094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:03.399877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:03.407589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.408141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.408610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.409429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.409920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.410576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.751413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.762269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.763815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.765162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-56e53201-f76a-4fcb-aa99-dd2573b16fd6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.371003 Oct 15 21:03:03.765780 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2078/3192] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:03 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:03.771367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:03.772034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:03.779989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.780554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.781269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.782252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.782902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.783549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.903644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.907666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:03.913679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.915158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:03.916415 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e9cdb26-ec47-406e-b3cc-10fa50b3aa5f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1335 microversion: 2.54 time: 0.149574 Oct 15 21:03:03.917288 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2079/3193] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:03 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1335 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:03.922351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:03.923315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:03.930527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.931229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.931879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:03.932598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:03.933100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:03.933587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.098638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.107955 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d5c6fdb6-9fc6-4a99-b2bb-27c9cf98fd00 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1323 microversion: 2.1 time: 0.189342 Oct 15 21:03:04.108633 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2080/3194] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:03 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1323 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:04.113401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:04.114100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:04.122197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.123137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.123628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.124444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.124972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.125443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.156909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:03:04.255411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b10a5b2-1a6d-4ce2-b66d-73e58795d85b tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "DELETE /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 204 len: 0 microversion: 2.57 time: 0.145239 Oct 15 21:03:04.262340 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2081/3195] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:03:04 2022] DELETE /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:03:04.270453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:04.271398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:04.279623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.279953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.280322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.281059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.281718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.282228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.456846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.465447 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb453706-03fb-436a-8310-d7d70d446a57 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.201513 Oct 15 21:03:04.466021 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2082/3196] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:04 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:04.470462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:04.471094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:04.476850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.477375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.477813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.478712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.479189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.479609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.613446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.624469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d6d6c22-f308-4ad7-85a4-4c3bcdfa349a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.157425 Oct 15 21:03:04.625306 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2083/3197] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:04 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:04.630119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:04.631080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:04.638007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.638695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.639354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.640206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.640698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.641363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.715944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.724113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.725613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.726780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a45548d1-a56a-41b1-9e62-11cb52164994 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1362 microversion: 2.57 time: 0.100781 Oct 15 21:03:04.727547 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2084/3198] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:04 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1362 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:04.796114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:04.796797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:04.803197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.804623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.805327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.806348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.806979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.807435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.941583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.950972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.952438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:04.953668 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31fd59f2-2512-4bfe-a2d6-9e9c4513bfc2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.161572 Oct 15 21:03:04.954227 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2085/3199] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:04 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:03:04.958851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:04.959406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:04.965283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.965779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.966230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:04.966937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:04.967407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:04.967844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:05.046349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:05.056343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:05.057869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:05.059155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ae41151-7486-47cb-8b6a-57bd48a30fc2 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1356 microversion: 2.54 time: 0.103990 Oct 15 21:03:05.060144 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2086/3200] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:04 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1356 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:05.089082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26d0298b-30ce-4b37-8da2-2e9c24423259 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Action: 'create', calling method: .version_select at 0x7fe758578670>, body: {"keypair": {"name": "tempest-ServerShowV254Test-keypair-1401276564", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNUQ7tU+p/IFcfNe/MoIznZ92q9NUWcZsOkjE92M97CinWu6pA/oH6awAvWvp21jXbKozFR/BXgvlPj+LhnOUuUubXBsy3ZI+1j8wWgUlu6v3m0knTMxeG6pNxKDsI1UXQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:05.110107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-26d0298b-30ce-4b37-8da2-2e9c24423259 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Getting quotas for project 642b98a4621b4d84b3e1e97137a68d33. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:05.117557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-26d0298b-30ce-4b37-8da2-2e9c24423259 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Getting quotas for user c7ceb51003c24728b665e7f641e3634b and project 642b98a4621b4d84b3e1e97137a68d33. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:05.153731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26d0298b-30ce-4b37-8da2-2e9c24423259 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 420 microversion: 2.54 time: 0.068304 Oct 15 21:03:05.159546 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2087/3201] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 21:03:05 2022] POST /compute/v2.1/os-keypairs => generated 420 bytes in 72 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 21:03:05.167442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:05.168385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:05.177299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:05.178257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:05.179020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:05.180246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:05.180953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:05.181641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:05.649634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:05.669633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-279b52f3-90fc-4f66-b815-82232af4f593 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1323 microversion: 2.1 time: 0.507880 Oct 15 21:03:05.678191 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2088/3202] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:05 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1323 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:05.690322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Action: 'action', calling method: >, body: {"rebuild": {"key_name": "tempest-ServerShowV254Test-keypair-1401276564", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:05.696777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:05.704878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:05.705654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:05.706328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:05.707788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:05.708139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:05.708635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:05.917128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:03:05.917834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:03:05.918067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:03:05.918487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:03:05.918910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:03:06.126435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:06.146845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:06.147404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:06.149581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:06.150354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:06.153760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:06.155489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:06.239159 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa465abd-b514-47ff-863d-245f63a19e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "POST /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e/action" status: 202 len: 1139 microversion: 2.54 time: 0.559066 Oct 15 21:03:06.240605 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2089/3203] 10.222.0.16 () {62 vars in 1320 bytes} [Sat Oct 15 21:03:05 2022] POST /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e/action => generated 1139 bytes in 562 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Oct 15 21:03:06.268394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:06.270294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:06.282297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:06.282820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:06.284337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:06.284895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:06.285608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:06.286173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:06.649902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:06.664919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b46a52f-f0fc-4a0d-b4c0-a52143ecacdb tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.422104 Oct 15 21:03:06.682215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2090/3204] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:06 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:06.699634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:06.700469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:06.712054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:06.719970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:06.720588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:06.721541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:06.722138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:06.722829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:06.989288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:06.999508 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6bf620f-904c-41ec-af09-090d9cbfeb19 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.316016 Oct 15 21:03:07.013644 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2091/3205] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:06 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:07.030902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:07.031571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:07.045469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:07.046005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:07.046537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:07.047313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:07.047643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:07.048060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:07.231292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:07.241848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:07.243537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:07.244856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37872108-0e29-400d-a2b8-a4c1da2dccfc tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1362 microversion: 2.57 time: 0.230073 Oct 15 21:03:07.245489 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2092/3206] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:07 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1362 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:07.250290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:07.250989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:07.258699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:07.259199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:07.259645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:07.260360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:07.260871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:07.261349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:07.305530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:07.860059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:07.918457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:07.927694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:07.947852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:07.957850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6cd2596-ae5c-418d-a6fa-120012b7ee37 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.709641 Oct 15 21:03:07.960461 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2093/3207] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:07 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:07.986191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:07.989940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:08.014724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:08.015891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:08.016502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:08.017801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:08.018453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:08.018925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:08.427348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:08.437800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:08.439328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:08.440668 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79b8d9c4-811b-4370-bb87-b8e9be4f79d8 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.477444 Oct 15 21:03:08.441684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2094/3208] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:07 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:08.457911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:08.458765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:08.480239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:08.480926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:08.481415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:08.482173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:08.482636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:08.483071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:08.748537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:08.759422 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90cf3af6-5773-445a-9044-3f45adc818d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.316351 Oct 15 21:03:08.760039 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2095/3209] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:08 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:08.764919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:08.765568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:08.772006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:08.772917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:08.773888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:08.774327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:08.774807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:08.775244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:08.996942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.005317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c834f32-b0a8-4064-abfe-f2358fc66506 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.243929 Oct 15 21:03:09.005896 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2096/3210] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:08 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:09.010982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:09.011813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:09.017960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.018625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.019253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.020114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.020751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.021486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.178173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.187206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b7cdc770-9c8b-4947-bf32-2193fd8552b4 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.180060 Oct 15 21:03:09.187989 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2097/3211] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:09 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:09.192792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e6d47e13-dd1c-4600-a997-ea9bc14f5921 req-e6cf2b4d-f80a-4bee-9336-746ff155b06c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4f6576d-84bb-45c2-8151-db174cec168a", "name": "network-vif-plugged", "status": "completed", "tag": "e16ce893-1cb6-4224-8086-e6e24fb3252a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:09.203233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e6d47e13-dd1c-4600-a997-ea9bc14f5921 req-e6cf2b4d-f80a-4bee-9336-746ff155b06c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.203721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e6d47e13-dd1c-4600-a997-ea9bc14f5921 req-e6cf2b4d-f80a-4bee-9336-746ff155b06c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.204159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e6d47e13-dd1c-4600-a997-ea9bc14f5921 req-e6cf2b4d-f80a-4bee-9336-746ff155b06c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.217838 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e6d47e13-dd1c-4600-a997-ea9bc14f5921 req-e6cf2b4d-f80a-4bee-9336-746ff155b06c service nova] Creating event network-vif-plugged:e16ce893-1cb6-4224-8086-e6e24fb3252a for instance a4f6576d-84bb-45c2-8151-db174cec168a on nch1-master-1 Oct 15 21:03:09.222683 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e6d47e13-dd1c-4600-a997-ea9bc14f5921 req-e6cf2b4d-f80a-4bee-9336-746ff155b06c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033606 Oct 15 21:03:09.223270 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2098/3212] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:09.230716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:09.231428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:09.237578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.238064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.238508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.239202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.239921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.240353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.313331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.321936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.323416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.324596 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07139b9b-c1c7-4eb0-9b78-e88d01e2d225 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1362 microversion: 2.57 time: 0.100250 Oct 15 21:03:09.325182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2099/3213] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:09 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1362 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:09.329737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:09.330316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:09.336898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.337535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.337979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.338670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.339184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.339575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.532631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.545560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec94dfbe-8a70-48af-bcfe-7d0d6d040ba8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.219285 Oct 15 21:03:09.559110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2100/3214] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:09 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:09.559110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:09.559110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:09.559593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.565147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.565147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.565147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.565147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.565147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.801994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.821756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.823479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.827035 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ace1034-0469-4649-9b77-9e7ba3d9e999 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.279413 Oct 15 21:03:09.828048 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2101/3215] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:09 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:09.832986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:09.834060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:09.844068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.844711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.845313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.847248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:09.847870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:09.848320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:09.956109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.973818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.976585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:09.978243 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e823f9e1-f2e4-41d1-a71e-aa85ff1a9e3a tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.149121 Oct 15 21:03:09.979633 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2102/3216] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:09 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:09.988143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e3e048d-f3ac-4a23-9543-9182eff732e2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1199458527", "metadata": {"": ""}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e3e048d-f3ac-4a23-9543-9182eff732e2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action HTTP/1.1 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 107 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1199458527", "metadata": {"": ""}}} Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-1199458527', 'metadata': {'': ''}}}} Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['createImage']['properties']['metadata']: Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['createImage']: Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'': ''} Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:03:09.994963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:10.001136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e3e048d-f3ac-4a23-9543-9182eff732e2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:10.003156 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e3e048d-f3ac-4a23-9543-9182eff732e2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action" status: 400 len: 170 microversion: 2.1 time: 0.020098 Oct 15 21:03:10.003975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2103/3217] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Oct 15 21:03:09 2022] POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action => generated 170 bytes in 24 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:03:10.015333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:10.016094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:10.026269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.026871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.027552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.028521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.029236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.029892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.243434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:10.255293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-29f52483-acd3-4f89-bdcf-35db252e745d tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.245774 Oct 15 21:03:10.258182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2104/3218] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:10 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:10.265811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:10.267005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:10.281615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.281831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.282320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.283051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.283562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.284351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.509881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:10.532162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1fdd266-29ef-4188-a1f6-a2f0cf32ca2d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.272097 Oct 15 21:03:10.537191 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2105/3219] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:10 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:10.542274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:10.542978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:10.550079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.550698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.551282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.552080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.552516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.552951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.742039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:10.752334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed05cf4b-7c46-44e0-89a2-690a42118433 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.217701 Oct 15 21:03:10.764942 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2106/3220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:10 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:10.778847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:10.779543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:10.787336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.788107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.788968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.790439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.790439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.790653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.965340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:10.974643 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75a4450e-bfe7-472c-b87d-90cb285af9bd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.208462 Oct 15 21:03:10.975255 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2107/3221] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:10 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:10.980362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:10.981174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:10.996851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:10.997363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:10.998003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:10.999424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.000015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.000494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.089474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.099031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.100750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.109728 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b653c518-7df3-41bf-b7ea-0d92535a8f05 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1362 microversion: 2.57 time: 0.125707 Oct 15 21:03:11.111045 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2108/3222] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:10 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1362 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:11.117817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:11.118750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:11.125984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.126758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.127260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.128286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.129062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.129834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.325995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.339636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.341781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.343418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e25a3b5-9b1a-4ee0-8edc-0a1d5fdfa4f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.230903 Oct 15 21:03:11.344874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2109/3223] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:11 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:11.350346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f34c0ae-b893-4bba-a3fb-f4e75e77a8e4 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-350691007", "metadata": {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f34c0ae-b893-4bba-a3fb-f4e75e77a8e4 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action HTTP/1.1 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 618 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-350691007", "metadata": {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}}} Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-350691007', 'metadata': {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}}}} Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:03:11.356130 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['createImage']['properties']['metadata']: Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': 'object'} Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['createImage']: Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'} Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Oct 15 21:03:11.359963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:11.364891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f34c0ae-b893-4bba-a3fb-f4e75e77a8e4 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:11.364891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5f34c0ae-b893-4bba-a3fb-f4e75e77a8e4 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action" status: 400 len: 938 microversion: 2.1 time: 0.011427 Oct 15 21:03:11.364891 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2110/3224] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Oct 15 21:03:11 2022] POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action => generated 938 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:03:11.369086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:11.369763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:11.378422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.379131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.379588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.389642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.390274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.390876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.486912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.496624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.498118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.499333 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86efec94-f32d-4c64-b87a-7374ff0edd82 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.135803 Oct 15 21:03:11.500284 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2111/3225] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:11 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:11.505787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f22f8e31-0659-4337-bffc-090aa1125251 req-080da220-fa2d-4e49-9b84-1e91ba50cf12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "name": "network-vif-plugged", "status": "completed", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:11.517434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f22f8e31-0659-4337-bffc-090aa1125251 req-080da220-fa2d-4e49-9b84-1e91ba50cf12 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.517925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f22f8e31-0659-4337-bffc-090aa1125251 req-080da220-fa2d-4e49-9b84-1e91ba50cf12 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.518366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f22f8e31-0659-4337-bffc-090aa1125251 req-080da220-fa2d-4e49-9b84-1e91ba50cf12 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.531367 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f22f8e31-0659-4337-bffc-090aa1125251 req-080da220-fa2d-4e49-9b84-1e91ba50cf12 service nova] Creating event network-vif-plugged:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:03:11.536760 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f22f8e31-0659-4337-bffc-090aa1125251 req-080da220-fa2d-4e49-9b84-1e91ba50cf12 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034440 Oct 15 21:03:11.540932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2112/3226] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:11.547128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:11.547950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:11.556071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.556601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.557101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.557865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.558338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.558830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.580616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:11.784386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:11.793602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec17601a-c208-4c1d-add1-d84680eb48bc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.251195 Oct 15 21:03:11.794462 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2113/3227] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:11 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:11.798750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:11.799360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:11.806319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.806898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.807431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.808224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:11.808881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:11.809577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:11.988338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.007553 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0e86b3d-f501-4c4b-a6ec-25cd9138c136 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.210806 Oct 15 21:03:12.007553 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2114/3228] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:11 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:12.014156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:12.014881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:12.016924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:12.033136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.033635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.034074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.034800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.035260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.035698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.223484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.237161 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7255886a-cbfa-41f9-a635-d2ef88fd034e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1323 microversion: 2.1 time: 0.225959 Oct 15 21:03:12.247815 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2115/3229] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:12 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1323 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:12.260520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:12.261467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:12.268500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.269286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.269730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.270550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.271184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.271701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.434477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.443886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a02f4d07-5999-41a6-a400-bdb6f9e9345f tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.194602 Oct 15 21:03:12.444534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2116/3230] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:12 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:12.449772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:12.450051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:12.456742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.457432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.457879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.458620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.459085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.459522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.540115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.551816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.554334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.555550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d86db73a-12cf-4da9-99e6-acda6f368010 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1362 microversion: 2.57 time: 0.110051 Oct 15 21:03:12.556114 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2117/3231] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:12 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1362 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:12.560747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:12.561390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:12.571864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.573603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.574376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.575504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.576240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.578480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.743306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.751082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.752729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.754209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e911d433-320f-4138-8f2e-6dc108bc18c6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.197089 Oct 15 21:03:12.773502 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2118/3232] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:12 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:12.776029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89d1ead4-fec0-4afd-a5f7-6b8584c2e21e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'action', calling method: >, body: {"createImage": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89d1ead4-fec0-4afd-a5f7-6b8584c2e21e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action HTTP/1.1 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 285 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: {"createImage": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc', 'body': {'createImage': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['createImage']['properties']['name']: Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['createImage']: Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.780365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89d1ead4-fec0-4afd-a5f7-6b8584c2e21e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-89d1ead4-fec0-4afd-a5f7-6b8584c2e21e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action" status: 400 len: 619 microversion: 2.1 time: 0.015799 Oct 15 21:03:12.785649 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2119/3233] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Oct 15 21:03:12 2022] POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action => generated 619 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:03:12.789075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:12.789597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:12.797475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.798037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.798486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.799336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.799814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.800318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.877064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.883515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.885504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:12.886788 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0ba26753-3262-4f94-9992-10a973efd9bd tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.102764 Oct 15 21:03:12.887892 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2120/3234] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:12 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:12.893153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:12.894201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:12.903360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.904046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.904883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:12.905740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:12.906341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:12.906779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.080049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.092401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d198328-3caa-4b5c-a98d-80182f4656cb tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.203253 Oct 15 21:03:13.105689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2121/3235] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:12 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:13.122383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:13.123018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:13.130483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.131161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.131677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.132403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.132863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.133325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.354620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.364134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c5b8a0c-6034-4431-a3b1-bc34dd2b8461 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.253878 Oct 15 21:03:13.377459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2122/3236] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:13 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:13.377459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:13.377459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:13.377459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.378373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.378373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.379282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.379429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.383126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.573009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.588404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78e40fa1-257e-42c2-8fe7-6b8f13306d4a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.222259 Oct 15 21:03:13.589502 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2123/3237] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:13 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:13.598717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:13.600044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:13.609313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.609958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.610486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.611435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.611962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.612490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.812277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.822143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0272b99-8156-42a2-b39c-ee52b47c2a0e tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1519 microversion: 2.1 time: 0.231691 Oct 15 21:03:13.837546 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2124/3238] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:13 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1519 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:13.849729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:13.850490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:13.858811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.859472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.860047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.860812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.861429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.861898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.944971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.956049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.959606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:13.961813 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b90e3ef4-52fa-4134-aca0-c63e165637a2 tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1362 microversion: 2.57 time: 0.120471 Oct 15 21:03:13.963023 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2125/3239] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:13 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1362 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:13.973883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1616211097"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:13.977517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:13.984809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.985357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.985766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:13.986805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:13.987449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:13.987987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:14.248779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e65ee6d9-bdd3-48b2-9c70-522db7e01479 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action" status: 202 len: 0 microversion: 2.1 time: 0.283888 Oct 15 21:03:14.249584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2126/3240] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:03:13 2022] POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action => generated 0 bytes in 286 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 21:03:14.255163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:14.255877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:14.270591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:14.270591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:14.270591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:14.270591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:14.270591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:14.270591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:14.439541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:14.450737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:14.452810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:14.454217 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef210930-5501-4564-9f39-192db7edcf57 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.203153 Oct 15 21:03:14.454817 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2127/3241] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:14 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:14.459462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c44dfe0-100f-4796-a0f7-755470e67688 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'create', calling method: .version_select at 0x7fe75443f8b0>, body: {"pool": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:15.126417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:16.175340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c44dfe0-100f-4796-a0f7-755470e67688 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 1.719326 Oct 15 21:03:16.176150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2128/3242] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 21:03:14 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 1721 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:16.182232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:16.183410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:16.191305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.192588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.193458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.202272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.203136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.203719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.284378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.296015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.297949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.299442 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82e18c79-6b9e-4303-8c67-b6fad2cb4ec7 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.122007 Oct 15 21:03:16.300038 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2129/3243] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:16 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:16.307051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1956754733"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:16.309666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:16.316919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.317518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.317954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.319566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.320218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.320797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.353953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:16.359345 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Cannot 'createImage' instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc while it is in task_state image_pending_upload Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action HTTP/1.1 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 85 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1956754733"}} Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'createImage' instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc while it is in task_state image_pending_upload Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-1956754733'}}} Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1349, in _action_create_image Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self.compute_api.snapshot(context, Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 155, in inner Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc in task_state image_pending_upload. Cannot snapshot while the instance is in this state. Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1354, in _action_create_image Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:03:16.361321 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'createImage' instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc while it is in task_state image_pending_upload Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'createImage' instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc while it is in task_state image_pending_upload Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 409 to user: Cannot 'createImage' instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc while it is in task_state image_pending_upload {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53a9ad65-dcf1-45aa-905c-f019cd905ca2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action" status: 409 len: 165 microversion: 2.1 time: 0.061084 Oct 15 21:03:16.366841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2130/3244] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:03:16 2022] POST /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc/action => generated 165 bytes in 62 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:03:16.371705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:16.372395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:16.379519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.380097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.380555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.381354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.381970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.382495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.541455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.549960 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a957f67d-5fba-4665-9921-049e0d052b09 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1316 microversion: 2.1 time: 0.185649 Oct 15 21:03:16.550643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2131/3245] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:16 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1316 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:16.565507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:16.565507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:16.565507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.565507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.565507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.565998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.566362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.566840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.607877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.608462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.608935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.609445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lazy-loading 'fault' on Instance uuid eb05f557-9385-4d0e-b5d6-0518c29c3a8d {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:03:16.655309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.664345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.665961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c50ab103943e4e3bacdc2777892c3bfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26a4431be325443c85b0ecef861d42ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.667209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c442371-e891-4a10-9e3c-5e6c331c53ee tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 200 len: 1367 microversion: 2.57 time: 0.115524 Oct 15 21:03:16.667774 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2132/3246] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:16 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 1367 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:16.672183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:16.672781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:16.678998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.679577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.680004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.680802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:16.681312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:16.681760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:16.826721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.836418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.837937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:16.839124 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3c76067-e2cd-4269-9815-e249edbf2c5f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.170425 Oct 15 21:03:16.839784 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2133/3247] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:16 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:16.844584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationTestJSON-server-2022017375", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "3a03a03d-6802-41ab-b356-830701974cbd"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:16.903525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] validate_networks() for [('3a03a03d-6802-41ab-b356-830701974cbd', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:03:17.328242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:03:17.328927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:03:17.329390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:03:17.329884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:03:17.330490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:03:17.342341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:17.342947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:17.343507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:17.344462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:17.345047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:17.345584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:17.368712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Getting quotas for project f278bb578eca4b5895634257d5ac832e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:17.375541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Getting quotas for user f2295a9d9b034667957b8e1d3c4901ab and project f278bb578eca4b5895634257d5ac832e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:17.391735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:03:17.394711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:03:17.451168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c71c4966-f93d-4043-b42e-079c86f818d9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.610215 Oct 15 21:03:17.451893 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2134/3248] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:03:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 612 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:03:17.462314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-996840ff-8b29-4de5-8fe1-dd372d46db65 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '.version_select at 0x7fe74fa98b80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:17.609602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-996840ff-8b29-4de5-8fe1-dd372d46db65 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/images/ccabcec1-70bb-47fc-952f-ece500d99a4c" status: 204 len: 0 microversion: 2.1 time: 0.156362 Oct 15 21:03:17.610479 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2135/3249] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:17 2022] DELETE /compute/v2.1/images/ccabcec1-70bb-47fc-952f-ece500d99a4c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:03:17.619005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:17.619758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:17.628041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:17.628749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:17.629381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:17.630164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:17.630785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:17.631337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:17.711090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:17.722910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:17.725806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:17.727218 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8bfb929-2835-4c19-8065-a6173654b73b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.112761 Oct 15 21:03:17.728022 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2136/3250] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:17 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:17.737515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3870271e-e26d-49bb-8425-8cf78f258ed8 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:17.738283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3870271e-e26d-49bb-8425-8cf78f258ed8 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:17.805361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3870271e-e26d-49bb-8425-8cf78f258ed8 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:17.813264 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3870271e-e26d-49bb-8425-8cf78f258ed8 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1224 microversion: 2.1 time: 0.084176 Oct 15 21:03:17.814082 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2137/3251] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:17 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:17.818391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:17.819035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:17.825692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:17.826265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:17.826796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:17.827518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:17.827990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:17.828478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.008529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:18.018671 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46d2c23b-6173-426d-96de-11e08a22b914 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1506 microversion: 2.1 time: 0.203705 Oct 15 21:03:18.019356 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2138/3252] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:17 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1506 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:18.024428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:18.025070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:18.031383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.031876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.032437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.033183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.034645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.034645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.212744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:18.224285 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-628e0a7f-b514-4ee9-bf98-2eed5fcbceb6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1538 microversion: 2.1 time: 0.203631 Oct 15 21:03:18.225825 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2139/3253] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:18 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1538 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:18.231841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:18.232501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] [instance: eb05f557-9385-4d0e-b5d6-0518c29c3a8d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:18.240186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.240685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.241814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.242772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.243445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.244187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.262514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] HTTP exception thrown: Instance eb05f557-9385-4d0e-b5d6-0518c29c3a8d could not be found. Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Request method failure captured: Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d HTTP/1.1 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_ZWxuBJnl_g0I_e0e2SV6tVs9waLNmR00mGSL39IJK534JnbeHSilEYyzaxWTvUlvRk60H5bYSkHBYH94kAYEeUKu8JGxO99w-EMJ4MH7zIP-KwZ8ZdszNnvqUIu5o6ZOtJHHGyLE10dhbUJh5-QaIQD3C4JHIytpPv-HT79s3BZ4 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.57 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 26a4431be325443c85b0ecef861d42ff Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerShowV257Test-14441886 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/26a4431be325443c85b0ecef861d42ff"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_26a4431be325443c85b0ecef861d42ff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/26a4431be325443c85b0ecef861d42ff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/26a4431be325443c85b0ecef861d42ff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerShowV257Test-14441886 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 26a4431be325443c85b0ecef861d42ff Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerShowV257Test-14441886 Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerShowV257Test-14441886-project Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c50ab103943e4e3bacdc2777892c3bfe Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerShowV257Test-14441886-project Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance eb05f557-9385-4d0e-b5d6-0518c29c3a8d could not be found. Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'eb05f557-9385-4d0e-b5d6-0518c29c3a8d'} Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eb05f557-9385-4d0e-b5d6-0518c29c3a8d could not be found. Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eb05f557-9385-4d0e-b5d6-0518c29c3a8d could not be found. Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:18.264131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eb05f557-9385-4d0e-b5d6-0518c29c3a8d could not be found. Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:18.269470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] Returning 404 to user: Instance eb05f557-9385-4d0e-b5d6-0518c29c3a8d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:18.270950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8081908-9934-4468-b019-cb8d13e5b12d tempest-ServerShowV257Test-14441886 tempest-ServerShowV257Test-14441886-project] 10.222.0.16 "GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d" status: 404 len: 111 microversion: 2.57 time: 0.043634 Oct 15 21:03:18.271826 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2140/3254] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:18 2022] GET /compute/v2.1/servers/eb05f557-9385-4d0e-b5d6-0518c29c3a8d => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:03:18.279370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:18.280136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:18.288790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.289369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.289901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.290938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.291410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.292722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.489624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:18.504899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:18.508955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:18.514141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab959ab7-3b1c-4e20-8b41-c7f1ec620235 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.239054 Oct 15 21:03:18.515072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2141/3255] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:18 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:18.535339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:18.535598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:18.544048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.544712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.545305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.546160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.546746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.547244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.589630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-instance-actions:events failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:18.590883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80278bf0-c72b-41d2-a2a3-128514962f9c tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a/os-instance-actions/req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8" status: 200 len: 313 microversion: 2.1 time: 0.068184 Oct 15 21:03:18.592192 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2142/3256] 10.222.0.16 () {58 vars in 1419 bytes} [Sat Oct 15 21:03:18 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a/os-instance-actions/req-19a57a3e-8892-4cf6-99d8-53e5c010b8c8 => generated 313 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:18.611064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:18.618407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:18.634298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.635714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.635714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.637177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.637731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.638161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.730964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3c39ba7-7b4e-43b3-93a3-759c9292896f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "POST /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a/action" status: 202 len: 0 microversion: 2.1 time: 0.127065 Oct 15 21:03:18.734831 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2143/3257] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:03:18 2022] POST /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a/action => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:03:18.750840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:18.751971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:18.761445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.763111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.764153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:18.765274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:18.765968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:18.767303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.031493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:19.044950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b34553b3-ff43-439e-b2bd-b45976570323 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1508 microversion: 2.1 time: 0.302294 Oct 15 21:03:19.049635 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2144/3258] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:18 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1508 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:19.064345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:19.065479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:19.072858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.073906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.074601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.075795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.076396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.076828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.233353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:19.251642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:19.253235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:19.255222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fa3427e3-f651-4d42-a54c-bb9dc868b2ab tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.204342 Oct 15 21:03:19.255956 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2145/3259] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:19 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:19.262401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:19.263060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:19.269764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.270612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.271253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.273294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.274617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.275116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.438342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:19.449572 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-837c15f4-943a-481e-b674-aaf88b81216b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1294 microversion: 2.1 time: 0.191905 Oct 15 21:03:19.450362 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2146/3260] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:19 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1294 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:19.455670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:19.456555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:19.464689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.465468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.466020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.466854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.467427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.468174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.816982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:19.837918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7d3cee1-21f8-4312-95b4-685e1672f08b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1538 microversion: 2.1 time: 0.383096 Oct 15 21:03:19.838137 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2147/3261] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:19 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1538 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:19.857376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:19.858717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:19.875667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.876136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.881014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:19.881014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:19.881014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:19.881014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.212220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.228294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.231105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.233712 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67954709-7506-4ebd-9809-e282f43fd19d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.394353 Oct 15 21:03:20.235810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2148/3262] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:19 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:20.265902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:20.268400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:20.280842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.281752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.282637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.283459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.284366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.285988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.323281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:20.586492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.595293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1a58a8e-1c72-451f-93b3-0d94ee63c741 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1508 microversion: 2.1 time: 0.356978 Oct 15 21:03:20.596121 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2149/3263] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:20 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1508 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:20.631056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:20.631801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:20.642654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.643179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.643741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.645151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.645635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.646079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.766899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.796712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.805001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:20.809540 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed8dcb13-096f-4ed0-84c5-63aa52fa7bc1 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.206799 Oct 15 21:03:20.825139 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2150/3264] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:20 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:20.825815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e9e6e7e1-b584-46c6-9289-9b663fd63e29 req-633ecf8a-5590-4627-bb90-a3dc9a7cf7aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b72dcc6-e2fb-40ad-860d-c65f66845edc", "name": "network-vif-plugged", "status": "completed", "tag": "ac46c3b1-490d-4d03-9d0d-44a3b6128cbd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:20.845717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e9e6e7e1-b584-46c6-9289-9b663fd63e29 req-633ecf8a-5590-4627-bb90-a3dc9a7cf7aa service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.848008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e9e6e7e1-b584-46c6-9289-9b663fd63e29 req-633ecf8a-5590-4627-bb90-a3dc9a7cf7aa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.849859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e9e6e7e1-b584-46c6-9289-9b663fd63e29 req-633ecf8a-5590-4627-bb90-a3dc9a7cf7aa service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.890494 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e9e6e7e1-b584-46c6-9289-9b663fd63e29 req-633ecf8a-5590-4627-bb90-a3dc9a7cf7aa service nova] Creating event network-vif-plugged:ac46c3b1-490d-4d03-9d0d-44a3b6128cbd for instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc on nch2-master-1 Oct 15 21:03:20.911446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e9e6e7e1-b584-46c6-9289-9b663fd63e29 req-633ecf8a-5590-4627-bb90-a3dc9a7cf7aa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097812 Oct 15 21:03:20.912286 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2151/3265] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:20.921079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:20.922109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:20.932587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.933157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.933580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:20.934311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:20.934830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:20.935349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.266991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:21.277176 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55bee2c0-a283-4ea0-a39c-4819bedcfa9a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.362379 Oct 15 21:03:21.278395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2152/3266] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:20 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:21.283420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:21.284281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:21.292064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:21.292819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:21.293684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.294632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:21.295285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:21.295939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.546825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:21.568901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a57a2eab-b8d2-4236-9cc8-47040dc3d7cc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1520 microversion: 2.1 time: 0.289075 Oct 15 21:03:21.570006 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2153/3267] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:21 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1520 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:21.575904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:21.576528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:21.583308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:21.583806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:21.584247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.585812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:21.586376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:21.586830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.873479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:21.888805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:21.891354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:21.893500 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87f9a39f-918b-40c5-a90e-68d93fa71214 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.321635 Oct 15 21:03:21.895396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2154/3268] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:21 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:21.916162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:21.917275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:21.924610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:21.925279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:21.926095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.927189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:21.927673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:21.928933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:21.964371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:03:22.117723 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eec041fe-51aa-45b8-bb7c-0637cc77f029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 204 len: 0 microversion: 2.1 time: 0.220477 Oct 15 21:03:22.118975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2155/3269] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:03:21 2022] DELETE /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:03:22.126423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:22.127284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:22.134530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:22.136868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.137614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.138095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.139043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.139517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.139959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.329811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:22.358275 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f1e8c99-cc14-47f6-a5a5-47d4f3348c00 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1506 microversion: 2.1 time: 0.232055 Oct 15 21:03:22.359506 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2156/3270] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:22 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1506 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:22.365350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:22.366972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:22.374607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.375134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.375581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.376583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.377530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.378237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.467030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:22.481394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:22.484654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:22.486322 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ca20269-5397-4945-8ac2-0e9ba90746ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.125380 Oct 15 21:03:22.486935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2157/3271] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:22 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:22.494526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:22.495158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:22.502045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.502558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.503391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.504116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.504582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.505015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.677069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:22.686895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c74b0f2d-4c87-4c94-81ac-1e165b25842b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1526 microversion: 2.1 time: 0.198904 Oct 15 21:03:22.703110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2158/3272] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:22 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1526 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:22.714427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-eb11728a-51b8-4b1b-830d-77cc478d7863 req-99794566-0787-4596-9db9-a8465bd50d27 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "085f9f3b-ac01-40f4-b859-c0b4a41f5d1d", "tag": "ec9d8bc4-aca9-4037-991f-3318ae367cbe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:22.724888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb11728a-51b8-4b1b-830d-77cc478d7863 req-99794566-0787-4596-9db9-a8465bd50d27 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.726239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb11728a-51b8-4b1b-830d-77cc478d7863 req-99794566-0787-4596-9db9-a8465bd50d27 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.726525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb11728a-51b8-4b1b-830d-77cc478d7863 req-99794566-0787-4596-9db9-a8465bd50d27 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.741153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-eb11728a-51b8-4b1b-830d-77cc478d7863 req-99794566-0787-4596-9db9-a8465bd50d27 service nova] Creating event network-changed:ec9d8bc4-aca9-4037-991f-3318ae367cbe for instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d on nch2-master-1 Oct 15 21:03:22.748472 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-eb11728a-51b8-4b1b-830d-77cc478d7863 req-99794566-0787-4596-9db9-a8465bd50d27 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043802 Oct 15 21:03:22.749589 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2159/3273] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:22.763334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:22.764058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:22.772614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.773285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.773729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.774738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:22.775141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:22.775575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:22.956453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:22.976395 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b374856-bcd0-4007-ad44-5414425dcd22 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.218796 Oct 15 21:03:22.977464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2160/3274] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:22 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:22.991821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '.version_select at 0x7fe74f9ad8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:22.992697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:23.005133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.005881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.007165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.007908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.008362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.008974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.049667 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20dff908-7601-4a37-a616-abd891e3e930 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a/os-instance-actions" status: 200 len: 611 microversion: 2.1 time: 0.070423 Oct 15 21:03:23.050273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2161/3275] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Oct 15 21:03:22 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a/os-instance-actions => generated 611 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:23.055826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:23.056446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:23.073935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.074437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.074887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.075607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.076067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.076620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.251066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.261188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.262990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.264339 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7499f5c2-a9c9-417b-bed0-149442bb9f6e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.213063 Oct 15 21:03:23.264913 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2162/3276] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:23 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:23.269756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-048664f9-8302-46e0-a5e7-040ee197048e req-22036938-bc48-44fd-a608-25485e18abe5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "name": "network-vif-plugged", "status": "completed", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:23.281680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-048664f9-8302-46e0-a5e7-040ee197048e req-22036938-bc48-44fd-a608-25485e18abe5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.282166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-048664f9-8302-46e0-a5e7-040ee197048e req-22036938-bc48-44fd-a608-25485e18abe5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.282825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-048664f9-8302-46e0-a5e7-040ee197048e req-22036938-bc48-44fd-a608-25485e18abe5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.296334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-048664f9-8302-46e0-a5e7-040ee197048e req-22036938-bc48-44fd-a608-25485e18abe5 service nova] Creating event network-vif-plugged:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:03:23.301594 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-048664f9-8302-46e0-a5e7-040ee197048e req-22036938-bc48-44fd-a608-25485e18abe5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035259 Oct 15 21:03:23.302214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2163/3277] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:23.308632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:23.309325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:23.316092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.316587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.317277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.318190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.318886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.319633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.349808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:03:23.420006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:23.462923 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-85fec674-0f4a-4e67-946a-271abd0a067d tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 204 len: 0 microversion: 2.1 time: 0.158317 Oct 15 21:03:23.464705 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2164/3278] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:03:23 2022] DELETE /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:03:23.484175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:23.488730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:23.501897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.506015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.506590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.507735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.508291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.509005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.763358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.775377 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-11393ff8-eb64-42ad-8248-81e80ef8c56f tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1512 microversion: 2.1 time: 0.304358 Oct 15 21:03:23.776215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2165/3279] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:23 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1512 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:23.785875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:23.785875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:23.796972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.797520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.797972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.798701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.799627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.800080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.910151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.922833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.924776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:23.926349 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2710edb-07bf-4725-968a-ab4dae1e0844 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.149420 Oct 15 21:03:23.941868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2166/3280] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:23 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:23.953455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:23.954589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:23.965250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.965888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.966300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:23.967285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:23.967861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:23.968424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:24.267532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:24.279792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d307acd1-32d1-4161-be2a-ecd9811a2969 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1526 microversion: 2.1 time: 0.336926 Oct 15 21:03:24.297069 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2167/3281] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:23 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1526 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:24.297069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:24.297069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:24.315928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:24.316433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:24.319514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:24.327076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:24.327585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:24.328178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:25.760044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:25.771263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95c25764-9fa4-4907-b6a5-95f314af1e99 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 1.483640 Oct 15 21:03:25.778207 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2168/3282] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:24 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 1492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:25.786683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:25.788903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:25.800488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:25.801328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:25.802088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:25.803589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:25.804293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:25.805663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:26.367477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:26.403580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:26.418461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:26.422056 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6b87b60-8b46-4514-b4fa-9e120089b796 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.639760 Oct 15 21:03:26.423242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2169/3283] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:25 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:26.431123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:26.435547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:26.470036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:26.473898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:26.474697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:26.478689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:26.480675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:26.481553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:26.938384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efa496c103449e486364cdef764054d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da46bd0ec0384f9398a890dcf12deb7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:26.951385 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3ba929f-9618-4e3e-aac9-528441990f79 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 200 len: 1512 microversion: 2.1 time: 0.526743 Oct 15 21:03:26.951766 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2170/3284] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:26 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 1512 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:26.967446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:26.969471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:26.982034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:26.982537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:26.982979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:26.998087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:26.998858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:26.999562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:27.354289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:27.386066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:27.414602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:27.419455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:27.420661 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-173b4d85-14cb-4a17-9b7e-2941b39b3e86 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.457118 Oct 15 21:03:27.451016 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2171/3285] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:26 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:27.466640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:27.467849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:27.486816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:27.487689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:27.488392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:27.489589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:27.491619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:27.492454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:27.625422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:27.658095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b418bf7-7d19-462e-b820-9ae896b4f5da tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1484 microversion: 2.1 time: 0.203062 Oct 15 21:03:27.660828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2172/3286] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:27 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1484 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:27.671479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f5d936e2-9560-43ad-9e3a-4d3ee49c25a3 req-40329d40-735a-4f31-a105-e7317fbfb132 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b72dcc6-e2fb-40ad-860d-c65f66845edc", "name": "network-vif-unplugged", "status": "completed", "tag": "ac46c3b1-490d-4d03-9d0d-44a3b6128cbd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:27.686761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f5d936e2-9560-43ad-9e3a-4d3ee49c25a3 req-40329d40-735a-4f31-a105-e7317fbfb132 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:27.687821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f5d936e2-9560-43ad-9e3a-4d3ee49c25a3 req-40329d40-735a-4f31-a105-e7317fbfb132 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:27.688893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f5d936e2-9560-43ad-9e3a-4d3ee49c25a3 req-40329d40-735a-4f31-a105-e7317fbfb132 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:27.705686 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f5d936e2-9560-43ad-9e3a-4d3ee49c25a3 req-40329d40-735a-4f31-a105-e7317fbfb132 service nova] Creating event network-vif-unplugged:ac46c3b1-490d-4d03-9d0d-44a3b6128cbd for instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc on nch2-master-1 Oct 15 21:03:27.716712 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f5d936e2-9560-43ad-9e3a-4d3ee49c25a3 req-40329d40-735a-4f31-a105-e7317fbfb132 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.051838 Oct 15 21:03:27.717755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2173/3287] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:27 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:27.725434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:27.728650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:27.741255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:27.742252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:27.742916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:27.743922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:27.744665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:27.745474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:27.813696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:27.954908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:27.964192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-418b362d-0d3a-4489-831e-25945a150fca tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.244827 Oct 15 21:03:27.964883 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2174/3288] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:27 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:27.971081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0a54f7cd-1f2d-4063-830e-cd880a18c031 req-339983fa-01bb-4665-b24f-aba470e83bd6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b72dcc6-e2fb-40ad-860d-c65f66845edc", "name": "network-vif-deleted", "tag": "ac46c3b1-490d-4d03-9d0d-44a3b6128cbd"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:27.981689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a54f7cd-1f2d-4063-830e-cd880a18c031 req-339983fa-01bb-4665-b24f-aba470e83bd6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:27.982251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a54f7cd-1f2d-4063-830e-cd880a18c031 req-339983fa-01bb-4665-b24f-aba470e83bd6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:27.982928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a54f7cd-1f2d-4063-830e-cd880a18c031 req-339983fa-01bb-4665-b24f-aba470e83bd6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.004017 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0a54f7cd-1f2d-4063-830e-cd880a18c031 req-339983fa-01bb-4665-b24f-aba470e83bd6 service nova] Creating event network-vif-deleted:ac46c3b1-490d-4d03-9d0d-44a3b6128cbd for instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc on nch2-master-1 Oct 15 21:03:28.013732 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0a54f7cd-1f2d-4063-830e-cd880a18c031 req-339983fa-01bb-4665-b24f-aba470e83bd6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046960 Oct 15 21:03:28.015074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2175/3289] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:28.027379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:28.028538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:28.036659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.037433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.039630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.040753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.041283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.041914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.360576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.373372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.374950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.376252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3b7e715-23cc-4d63-bae4-bb4f54214a69 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1810 microversion: 2.20 time: 0.359800 Oct 15 21:03:28.376878 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2176/3290] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:28 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1810 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:28.385229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:28.385827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] [instance: a4f6576d-84bb-45c2-8151-db174cec168a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:28.393863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.394612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.395234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.396204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.397367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.398296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.416290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] HTTP exception thrown: Instance a4f6576d-84bb-45c2-8151-db174cec168a could not be found. Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Request method failure captured: Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a HTTP/1.1 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_1XFc-MA87N9oe6aAuO0AzME2vFGnsbh7hUzYF_zHxePUC574GnH86vZGMp95-QaK9Ynpfud73hoi7Lcq99-wi--8h8rGTAQNcfg19Zp16ockyju2KfjV46U-zBwFJR6lVQIuaaeGpAakAe1tj5888p9Wx13xq7j9JEQN60R2UZws Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: da46bd0ec0384f9398a890dcf12deb7a Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-InstanceActionsTestJSON-1365767919 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da46bd0ec0384f9398a890dcf12deb7a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da46bd0ec0384f9398a890dcf12deb7a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da46bd0ec0384f9398a890dcf12deb7a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da46bd0ec0384f9398a890dcf12deb7a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-InstanceActionsTestJSON-1365767919 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: da46bd0ec0384f9398a890dcf12deb7a Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-InstanceActionsTestJSON-1365767919 Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-InstanceActionsTestJSON-1365767919-project Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6efa496c103449e486364cdef764054d Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-InstanceActionsTestJSON-1365767919-project Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance a4f6576d-84bb-45c2-8151-db174cec168a could not be found. Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'a4f6576d-84bb-45c2-8151-db174cec168a'} Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4f6576d-84bb-45c2-8151-db174cec168a could not be found. Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4f6576d-84bb-45c2-8151-db174cec168a could not be found. Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:28.418403 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4f6576d-84bb-45c2-8151-db174cec168a could not be found. Oct 15 21:03:28.423686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:28.425666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] Returning 404 to user: Instance a4f6576d-84bb-45c2-8151-db174cec168a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:28.426546 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc4a8294-729a-4075-8201-dbc3becbe8d5 tempest-InstanceActionsTestJSON-1365767919 tempest-InstanceActionsTestJSON-1365767919-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a" status: 404 len: 111 microversion: 2.1 time: 0.045978 Oct 15 21:03:28.427115 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2177/3291] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:28 2022] GET /compute/v2.1/servers/a4f6576d-84bb-45c2-8151-db174cec168a => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:03:28.459776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:28.460770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:28.478477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.479810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.480489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.482186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.483210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.484269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.600468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.622685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.624642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.626408 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3638b2d7-732c-4a6a-9eb3-3dda537c65ea tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.171829 Oct 15 21:03:28.630066 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2178/3292] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:28 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:03:28.698541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:28.699475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:28.710418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.710966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.711416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.712495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.713205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.713702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.782009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:28.782896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:28.784772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:28.785567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lazy-loading 'fault' on Instance uuid 0b72dcc6-e2fb-40ad-860d-c65f66845edc {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:03:28.888364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:28.909119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1bd64f8a-2c97-4ad9-a90a-8c32a4975b7a tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 200 len: 1313 microversion: 2.1 time: 0.217451 Oct 15 21:03:28.909119 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2179/3293] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:28 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 1313 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:29.000884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:29.002150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:29.029453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:29.031008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:29.031548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:29.032682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:29.033214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:29.033681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:29.329009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:29.343250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-96a890c9-ee50-4129-b676-c9c423b657f1 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.356750 Oct 15 21:03:29.344014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2180/3294] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:28 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:03:29.414086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:29.414732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:29.428624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:29.429460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:29.431192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:29.432026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:29.432506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:29.432975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:29.791151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:29.811187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:29.815049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:29.816589 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46c028dd-16aa-426a-a261-45000219f457 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1797 microversion: 2.20 time: 0.408280 Oct 15 21:03:29.817740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2181/3295] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:29 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1797 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:03:29.832227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:29.832227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:29.847104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:29.847461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:29.847868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:29.852234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:29.856192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:29.857007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:30.007824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:30.025921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:30.028527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:30.030322 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddc82f88-f6c6-4827-968f-026dd3fe8c7b tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.211559 Oct 15 21:03:30.030949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2182/3296] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:29 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:30.036431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:30.037539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:30.045244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:30.046107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:30.046786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:30.047745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:30.048766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:30.049665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:30.055437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:30.070201 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc HTTP/1.1 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc'} Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:30.072207 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:03:30.076608 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:30.081577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:30.085534 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba171352-78ec-4d39-9c25-4c192c2d602e tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 404 len: 111 microversion: 2.1 time: 0.051902 Oct 15 21:03:30.086797 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2183/3297] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:30 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:03:30.092221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4023f894-7945-4da6-9948-560aa18e31aa req-3e14f332-f259-4a1e-b709-783c587d605e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4f6576d-84bb-45c2-8151-db174cec168a", "name": "network-vif-deleted", "tag": "e16ce893-1cb6-4224-8086-e6e24fb3252a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:30.112254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4023f894-7945-4da6-9948-560aa18e31aa req-3e14f332-f259-4a1e-b709-783c587d605e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:30.113751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4023f894-7945-4da6-9948-560aa18e31aa req-3e14f332-f259-4a1e-b709-783c587d605e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:30.114474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4023f894-7945-4da6-9948-560aa18e31aa req-3e14f332-f259-4a1e-b709-783c587d605e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:30.136065 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4023f894-7945-4da6-9948-560aa18e31aa req-3e14f332-f259-4a1e-b709-783c587d605e service nova] Creating event network-vif-deleted:e16ce893-1cb6-4224-8086-e6e24fb3252a for instance a4f6576d-84bb-45c2-8151-db174cec168a on nch1-master-1 Oct 15 21:03:30.142476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4023f894-7945-4da6-9948-560aa18e31aa req-3e14f332-f259-4a1e-b709-783c587d605e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053540 Oct 15 21:03:30.143910 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2184/3298] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:30.154169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3faf1b36-b719-4238-95ed-f79ac2b54491 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'create', calling method: .version_select at 0x7fe74fa9f3a0>, body: {"keypair": {"name": "tempest-keypair-1070717387", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMSskIqW+O8pXAKMER52A8LexCoxQq8A+QUqHbo1AjKkCt64BHIFm/35/g7IHeIJkoexdC/iFTwNjSV6NQQwHQTDMsKJj13VNbmf9FBzhQPOjDJkb6WkXXvwXw9PHIvpag=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:30.175499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3faf1b36-b719-4238-95ed-f79ac2b54491 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:30.183330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3faf1b36-b719-4238-95ed-f79ac2b54491 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for user a8af9544fe68497c8557248575f184ed and project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:30.228983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3faf1b36-b719-4238-95ed-f79ac2b54491 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.083316 Oct 15 21:03:30.229636 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2185/3299] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:03:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:30.369291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:30.370604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:30.391393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:30.391925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:30.392386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:30.397438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:30.398300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:30.399910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:31.009773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:31.028290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aaf65ada-7dfe-4dd1-8235-4353c197a68b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.663971 Oct 15 21:03:31.045720 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2186/3300] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:30 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:03:31.076813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:31.077510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:31.091944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:31.093456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:31.097764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:31.098622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:31.099165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:31.099686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:31.328394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:31.342813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:31.344761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:31.351301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d4e7a41-3c43-4aaa-b71e-7ddb1450fb8f tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.291734 Oct 15 21:03:31.368433 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2187/3301] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:31 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:03:32.056307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:32.056307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:32.070319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:32.076926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:32.077816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:32.080584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:32.080584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:32.081581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:32.676811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:32.692668 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7eb7a903-eb1b-457c-ba94-e1929d0a47b2 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.638233 Oct 15 21:03:32.704466 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2188/3302] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:32 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:32.722577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:32.723366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:32.735594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:32.742279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:32.747697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:32.748732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:32.750709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:32.762007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:33.444942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:33.462425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:33.466280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:33.467960 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d426092a-5e13-456a-9ee2-6f2d08444c16 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1408 microversion: 2.54 time: 0.762025 Oct 15 21:03:33.469299 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2189/3303] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:32 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1408 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:33.722791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:33.723420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:33.741246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:33.741858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:33.742516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:33.743274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:33.743737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:33.744187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:34.328832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:34.354124 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7604454d-e0fa-4450-94ac-ece6921fe0fd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.635834 Oct 15 21:03:34.377235 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2190/3304] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:33 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:03:34.863543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-30dedf67-50f6-4d7c-bbe0-edf56c1fad5c req-28b28dd6-97a2-48aa-8015-34a5cebfc86f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "085f9f3b-ac01-40f4-b859-c0b4a41f5d1d", "name": "network-vif-plugged", "status": "completed", "tag": "ec9d8bc4-aca9-4037-991f-3318ae367cbe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:34.876925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30dedf67-50f6-4d7c-bbe0-edf56c1fad5c req-28b28dd6-97a2-48aa-8015-34a5cebfc86f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:34.878063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30dedf67-50f6-4d7c-bbe0-edf56c1fad5c req-28b28dd6-97a2-48aa-8015-34a5cebfc86f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:34.878761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30dedf67-50f6-4d7c-bbe0-edf56c1fad5c req-28b28dd6-97a2-48aa-8015-34a5cebfc86f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:34.893229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-30dedf67-50f6-4d7c-bbe0-edf56c1fad5c req-28b28dd6-97a2-48aa-8015-34a5cebfc86f service nova] Creating event network-vif-plugged:ec9d8bc4-aca9-4037-991f-3318ae367cbe for instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d on nch2-master-1 Oct 15 21:03:34.900382 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-30dedf67-50f6-4d7c-bbe0-edf56c1fad5c req-28b28dd6-97a2-48aa-8015-34a5cebfc86f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.521056 Oct 15 21:03:34.902077 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2191/3305] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:34.909283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:34.910403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:34.918667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:34.919514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:34.920210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:34.921417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:34.922426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:34.923142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:35.042583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:35.071905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:35.078081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:35.081060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07c49260-9073-4eef-bad4-92e5fedbae37 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1414 microversion: 2.54 time: 0.176732 Oct 15 21:03:35.094970 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2192/3306] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:34 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1414 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:35.109384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-400263298", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:35.230525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] validate_networks() for [('17f4161c-0a61-4650-a009-d339764cbfd5', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:03:36.456481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:03:36.457382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:03:36.457877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:03:36.458302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:03:36.459711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:03:36.487744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:36.488238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:36.491536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:36.493879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:36.495343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:36.496018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:36.551068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:36.561489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for user a8af9544fe68497c8557248575f184ed and project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:36.569959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:36.603124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:03:36.610512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:03:36.760736 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-097c2589-f4b5-463b-b504-d6c3e88db9c5 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.653488 Oct 15 21:03:36.763734 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2193/3307] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:03:35 2022] POST /compute/v2.1/servers => generated 383 bytes in 1667 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:03:36.772931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:36.783995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:36.804176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:36.804956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:36.805713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:36.806991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:36.808214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:36.809288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:37.266901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:37.284820 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-85b05537-ef62-47d1-8f27-27a3c36cd74b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1324 microversion: 2.1 time: 0.519085 Oct 15 21:03:37.295689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2194/3308] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:36 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1324 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:37.312780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:37.313378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:37.336416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:37.347950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:37.349980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:37.351706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:37.355258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:37.356193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:37.590532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:37.618162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:37.623055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:37.625327 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73778eb8-4950-4bfd-aec1-bd3f90c50a6c tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1399 microversion: 2.54 time: 0.328017 Oct 15 21:03:37.639398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2195/3309] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:37 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1399 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:37.660408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-330b4cbe-2d31-4980-8150-99c5aceb3612 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:37.661306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-330b4cbe-2d31-4980-8150-99c5aceb3612 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:37.859385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-330b4cbe-2d31-4980-8150-99c5aceb3612 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:37.880174 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-330b4cbe-2d31-4980-8150-99c5aceb3612 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1224 microversion: 2.1 time: 0.231503 Oct 15 21:03:37.880824 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2196/3310] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:37 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1224 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:37.911231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bf7cdf46-1ad1-4a92-876f-13ac873edd11 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '.version_select at 0x7fe74fec6ca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:37.955699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bf7cdf46-1ad1-4a92-876f-13ac873edd11 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-ServerShowV254Test-keypair-1401276564" status: 204 len: 0 microversion: 2.54 time: 0.070161 Oct 15 21:03:37.961170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2197/3311] 10.222.0.16 () {60 vars in 1319 bytes} [Sat Oct 15 21:03:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-ServerShowV254Test-keypair-1401276564 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:03:37.983795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:37.985247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:38.013243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:38.014043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:38.017195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:38.018492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:38.020588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:38.021092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:38.081723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:03:38.286311 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4b27211-e5d3-449a-9069-d42f4b3a7917 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 204 len: 0 microversion: 2.54 time: 0.318497 Oct 15 21:03:38.291109 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2198/3312] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:03:37 2022] DELETE /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:03:38.301247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:38.303283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:38.341008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:38.341629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:38.345554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:38.348379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:38.355988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:38.363661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:38.509580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:38.525439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:38.527018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c7ceb51003c24728b665e7f641e3634b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '642b98a4621b4d84b3e1e97137a68d33', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:38.528353 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37f9347b-212f-49e0-ab21-5adb250df0bc tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 200 len: 1405 microversion: 2.54 time: 0.234665 Oct 15 21:03:38.549805 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2199/3313] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:38 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 1405 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:03:38.566118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:38.567224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:38.579315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:38.580059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:38.580760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:38.581699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:38.582326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:38.582918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:38.784695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:38.796740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-904d9fdd-0ba9-4b03-bbbe-384341b621e5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 200 len: 1521 microversion: 2.1 time: 0.245574 Oct 15 21:03:38.798966 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2200/3314] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:38 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 1521 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:38.972136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestMultiTenantJSON-server-818527203", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "18ab4509-6fb9-4c0a-b27e-c19743138dfe"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:39.044497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] validate_networks() for [('18ab4509-6fb9-4c0a-b27e-c19743138dfe', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:03:39.580416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:03:39.581295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:03:39.582042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:03:39.582592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:03:39.583027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:03:39.596375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:39.596909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:39.598139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:39.598575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:39.599063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:39.600123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:39.619148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Getting quotas for project c6d5585dc2254ed2aed773305d209049. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:39.625323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Getting quotas for user 14c805d8b3e944e8ad9f6150322b96ce and project c6d5585dc2254ed2aed773305d209049. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:39.641015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:03:39.643580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:03:39.709748 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5dc4ac6d-08d5-4414-bf5b-9d0b34083910 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.908812 Oct 15 21:03:39.710970 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2201/3315] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:03:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 911 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:03:39.716493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.4.132"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:39.720164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:39.730296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:39.730782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:39.731234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:39.732124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:39.732594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:39.733050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:39.768828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:03:39.769519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquired lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:03:43.032578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:03:44.860063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Updating instance_info_cache with network_info: [{"id": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d", "address": "fa:16:3e:80:ea:d2", "network": {"id": "3a03a03d-6802-41ab-b356-830701974cbd", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-1335856832-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.4.132", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f278bb578eca4b5895634257d5ac832e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap0bf5cf03-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:03:44.878600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Releasing lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:03:44.879602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6149dc6-3b2d-4cb2-ba7a-4e50f8a92383 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action" status: 202 len: 0 microversion: 2.1 time: 5.167132 Oct 15 21:03:44.880368 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2202/3316] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:03:39 2022] POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action => generated 0 bytes in 5169 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:03:44.888291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:44.889004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:44.895953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:44.896973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:44.897478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:44.898207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:44.899163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:44.899163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:45.215598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:45.218946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:45.230550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1aa4c9ab-a8ef-48da-b907-589ba639b900 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1324 microversion: 2.1 time: 0.347881 Oct 15 21:03:45.231841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2203/3317] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:44 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1324 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:45.331631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-1942351304", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "ab239bdb-1532-47da-b608-6197271e259a"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:45.417074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] validate_networks() for [('ab239bdb-1532-47da-b608-6197271e259a', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:03:46.431221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:03:46.432743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:03:46.433654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:03:46.434192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:03:46.434708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:03:46.450787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:46.451322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:46.452036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:46.452946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:46.453624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:46.454186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:46.488190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Getting quotas for project f20ae5c8217d412289d006ab9e01e7f4. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:46.495020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Getting quotas for user 124511010d044f8fb07dc09e7d754303 and project f20ae5c8217d412289d006ab9e01e7f4. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:46.510440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:03:46.512774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:03:46.563516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47b0b5a6-a527-40f0-ac54-02ca10546f3d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.330233 Oct 15 21:03:46.564252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2204/3318] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:03:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 1332 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:03:46.589603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:46.590310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] [instance: 5fe03c0d-436f-4aac-b0b6-c6929fde891e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:46.600757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:46.601519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:46.602068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:46.602952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:46.603377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:46.604056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:46.624614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] HTTP exception thrown: Instance 5fe03c0d-436f-4aac-b0b6-c6929fde891e could not be found. Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Request method failure captured: Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e HTTP/1.1 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx__rVJqOTXiF2SPWJZCMpGKIXjmpLEBnqLzGso9qHBkgvMGQLB3e5xyohjP6VD8OkpLYr5UOJcysb01wovmqxKxWVnUXvOz6IIQcsAi-uWKgEVh0QYDbkscMWU3GZH4z7PaGNyh1r3ByYNNDNcUZe-k61VKlddvPOGNMOt6G6WD2Vg Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.54 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 642b98a4621b4d84b3e1e97137a68d33 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerShowV254Test-74559440 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/642b98a4621b4d84b3e1e97137a68d33"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_642b98a4621b4d84b3e1e97137a68d33"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/642b98a4621b4d84b3e1e97137a68d33"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/642b98a4621b4d84b3e1e97137a68d33"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerShowV254Test-74559440 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 642b98a4621b4d84b3e1e97137a68d33 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerShowV254Test-74559440 Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerShowV254Test-74559440-project Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c7ceb51003c24728b665e7f641e3634b Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerShowV254Test-74559440-project Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 5fe03c0d-436f-4aac-b0b6-c6929fde891e could not be found. Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5fe03c0d-436f-4aac-b0b6-c6929fde891e'} Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5fe03c0d-436f-4aac-b0b6-c6929fde891e could not be found. Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5fe03c0d-436f-4aac-b0b6-c6929fde891e could not be found. Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:46.626813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5fe03c0d-436f-4aac-b0b6-c6929fde891e could not be found. Oct 15 21:03:46.633012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:46.634016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] Returning 404 to user: Instance 5fe03c0d-436f-4aac-b0b6-c6929fde891e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:46.635115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-89f142be-99db-42ca-8d39-efbf3cc9d940 tempest-ServerShowV254Test-74559440 tempest-ServerShowV254Test-74559440-project] 10.222.0.16 "GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e" status: 404 len: 111 microversion: 2.54 time: 0.069435 Oct 15 21:03:46.639096 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2205/3319] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:46 2022] GET /compute/v2.1/servers/5fe03c0d-436f-4aac-b0b6-c6929fde891e => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:03:46.647276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:46.648160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:46.658458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:46.659227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:46.660165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:46.660900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:46.661415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:46.662773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:46.918691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:46.932973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36407a87-8d13-44f9-8953-2c30fb9b4eda tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1327 microversion: 2.1 time: 0.295971 Oct 15 21:03:46.946654 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2206/3320] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:46 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1327 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:47.184651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2ab58253-6234-44db-9ce7-8a2a71ffd215 req-d435817e-7618-47f2-b5fd-b1554d7f6ead service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:47.204071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2ab58253-6234-44db-9ce7-8a2a71ffd215 req-d435817e-7618-47f2-b5fd-b1554d7f6ead service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:47.205217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2ab58253-6234-44db-9ce7-8a2a71ffd215 req-d435817e-7618-47f2-b5fd-b1554d7f6ead service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:47.205954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2ab58253-6234-44db-9ce7-8a2a71ffd215 req-d435817e-7618-47f2-b5fd-b1554d7f6ead service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:47.230630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2ab58253-6234-44db-9ce7-8a2a71ffd215 req-d435817e-7618-47f2-b5fd-b1554d7f6ead service nova] Creating event network-changed:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:03:47.237160 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2ab58253-6234-44db-9ce7-8a2a71ffd215 req-d435817e-7618-47f2-b5fd-b1554d7f6ead service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.300047 Oct 15 21:03:47.237736 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2207/3321] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:47.244506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.4.132"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:47.248764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:47.256612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:47.257389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:47.258179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:47.258907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:47.259428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:47.260081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:47.292309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "refresh_cache-085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:03:47.292739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquired lock "refresh_cache-085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:03:51.527053 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.network.neutron [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] re-assign floating IP 172.35.4.132 from instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 Oct 15 21:03:51.612494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:03:52.816468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Updating instance_info_cache with network_info: [{"id": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d", "address": "fa:16:3e:80:ea:d2", "network": {"id": "3a03a03d-6802-41ab-b356-830701974cbd", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-1335856832-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f278bb578eca4b5895634257d5ac832e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap0bf5cf03-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:03:52.826340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:03:53.797630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Updating instance_info_cache with network_info: [{"id": "ec9d8bc4-aca9-4037-991f-3318ae367cbe", "address": "fa:16:3e:fe:bb:a8", "network": {"id": "3a03a03d-6802-41ab-b356-830701974cbd", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-1335856832-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.4.132", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f278bb578eca4b5895634257d5ac832e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapec9d8bc4-ac", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:03:53.823202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Releasing lock "refresh_cache-085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:03:53.828799 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f28f2110-ec91-45d3-8b33-19dd19028b40 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d/action" status: 202 len: 0 microversion: 2.1 time: 6.586600 Oct 15 21:03:53.832237 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2208/3322] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:03:47 2022] POST /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d/action => generated 0 bytes in 6593 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:03:53.845406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:53.846255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:53.854411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:53.855697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:53.856324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:53.857241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:53.857916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:53.858513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:54.054903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:54.083936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86204424-fe13-4b81-a610-7ca8a1f4e016 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1324 microversion: 2.1 time: 0.245222 Oct 15 21:03:54.101547 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2209/3323] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:53 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1324 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:54.108826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-113777070", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "ab239bdb-1532-47da-b608-6197271e259a"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:54.220296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] validate_networks() for [('ab239bdb-1532-47da-b608-6197271e259a', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:03:55.640766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:03:55.641690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:03:55.642237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:03:55.643873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:03:55.644371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:03:55.654973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:55.655459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:55.655895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:55.656788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:55.657274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:55.657733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:55.675344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Getting quotas for project f20ae5c8217d412289d006ab9e01e7f4. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:03:55.681342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Getting quotas for user 124511010d044f8fb07dc09e7d754303 and project f20ae5c8217d412289d006ab9e01e7f4. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:03:55.695001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:03:55.697373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:03:55.750144 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-038cc362-0cf8-4e49-bf6c-cb230ca4582a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.645975 Oct 15 21:03:55.751398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2210/3324] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:03:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 1648 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:03:55.756774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:55.757915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:55.768839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:55.769433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:55.769950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:55.770586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:55.773415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:55.774111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:56.016520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:56.037751 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e78078a-fc16-488a-800a-d15ec594dc62 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1327 microversion: 2.1 time: 0.284801 Oct 15 21:03:56.051730 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2211/3325] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:55 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1327 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:56.073360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:56.075310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:56.086041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:56.086981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:56.087743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:56.089241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:56.090588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:56.091242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:56.357906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:56.376423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:56.378318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:56.379830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b5d1606-d5a7-4958-82b5-e2cc231cff31 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2956 microversion: 2.42 time: 0.321708 Oct 15 21:03:56.381099 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2212/3326] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:03:56 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2956 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:03:56.386361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "227d10b5-22c4-44a9-b65e-de1ed6d8680c", "tag": "6ca218e1-1310-4faf-9e51-a986e94f59d7"}, {"name": "network-changed", "server_uuid": "39bc88ae-5f1d-4b2e-b803-251ca3efac5c", "tag": "2c0abf87-a1c2-4694-9a7c-3560674a208d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:56.397832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:56.400047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:56.400846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:56.421698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] Creating event network-changed:6ca218e1-1310-4faf-9e51-a986e94f59d7 for instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c on nch1-master-1 Oct 15 21:03:56.422519 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] Creating event network-changed:2c0abf87-a1c2-4694-9a7c-3560674a208d for instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c on nch2-master-1 Oct 15 21:03:56.434044 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6c25f937-d2d0-4eb1-a556-ac9e77677866 req-78733cd0-490d-4ee4-b78b-ae13e8a2a468 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.051227 Oct 15 21:03:56.435565 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2213/3327] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:56 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:56.443193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-671fa5ee-e767-4631-8f71-b1ac6b663e79 req-12d8cbc5-db67-4c58-a097-06ca33881fdc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "227d10b5-22c4-44a9-b65e-de1ed6d8680c", "name": "network-vif-plugged", "status": "completed", "tag": "6ca218e1-1310-4faf-9e51-a986e94f59d7"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:56.448976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=12,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:03:56.456667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-671fa5ee-e767-4631-8f71-b1ac6b663e79 req-12d8cbc5-db67-4c58-a097-06ca33881fdc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:56.457252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-671fa5ee-e767-4631-8f71-b1ac6b663e79 req-12d8cbc5-db67-4c58-a097-06ca33881fdc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:56.457832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-671fa5ee-e767-4631-8f71-b1ac6b663e79 req-12d8cbc5-db67-4c58-a097-06ca33881fdc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:56.478128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-671fa5ee-e767-4631-8f71-b1ac6b663e79 req-12d8cbc5-db67-4c58-a097-06ca33881fdc service nova] Creating event network-vif-plugged:6ca218e1-1310-4faf-9e51-a986e94f59d7 for instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c on nch1-master-1 Oct 15 21:03:56.485702 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-671fa5ee-e767-4631-8f71-b1ac6b663e79 req-12d8cbc5-db67-4c58-a097-06ca33881fdc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047093 Oct 15 21:03:56.486594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2214/3328] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:03:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:03:56.510666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.4.132"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:57.044278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:57.054668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:57.055908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:57.056625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:57.060563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:57.061178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:57.061777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:57.745422 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] HTTP exception thrown: Floating IP 172.35.4.132 is not associated with instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3. Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Request method failure captured: Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action HTTP/1.1 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 49 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_2U0wpi9iLFol51425g7xlLgJZYv44qoqxJRdbBxz4YPFPBsxPKv03xFKBNW2j7nPvjTCKZ6yrS0YXfJwzQHSc5oH5nUDBioVusEka57vt3SlAQpUJ_M9mPyJ1PHcTih2cxI9UASJHph_vYYfIrHewGVsrJOqVsxtD0fQRIL22RbQ Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f278bb578eca4b5895634257d5ac832e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f2295a9d9b034667957b8e1d3c4901ab Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: {"removeFloatingIp": {"address": "172.35.4.132"}} Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: exception: Floating IP 172.35.4.132 is not associated with instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3. Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd75b558a-f4a1-47f7-977c-d9fb11cfefa3', 'body': {'removeFloatingIp': {'address': '172.35.4.132'}}} Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 300, in _remove_floating_ip Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Floating IP 172.35.4.132 is not associated with instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3. Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP 172.35.4.132 is not associated with instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3. Oct 15 21:03:57.747232 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:03:57.752086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Returning 409 to user: Floating IP 172.35.4.132 is not associated with instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:03:57.752086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3405217-7fb6-411b-9a1c-11ad827ef177 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action" status: 409 len: 146 microversion: 2.1 time: 1.260672 Oct 15 21:03:57.752086 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2215/3329] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:03:56 2022] POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action => generated 146 bytes in 1262 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:03:57.755667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:03:57.756627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:03:57.763592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:57.764343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:57.773188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:57.774569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:03:57.776024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:03:57.776780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:03:58.345153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:03:58.362501 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae292e67-b695-49b1-aa93-99780737a8c2 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1324 microversion: 2.1 time: 0.611955 Oct 15 21:03:58.363070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2216/3330] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:03:57 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1324 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:03:58.542501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1316586092", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:03:58.615389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] validate_networks() for [('e3c96722-97ae-4b06-ab5c-8a523bae70f3', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:04:00.005492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:04:00.006870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:04:00.007622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:04:00.008113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:04:00.008634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:04:00.027880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=31 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:00.032066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:00.033603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:00.036294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:00.042134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:00.042955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:00.043652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:00.055896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=9 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:00.073281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Getting quotas for project 172114b9a46e4df8ae8076ed582f0b5f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:00.100023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Getting quotas for user 90e8754a100240a498e422de07c76bbf and project 172114b9a46e4df8ae8076ed582f0b5f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:00.136859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:04:00.141470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:04:00.215965 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-214100d7-46a3-446b-85f2-d97f92dfe119 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.851735 Oct 15 21:04:00.216631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2217/3331] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:03:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 1853 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:04:00.227686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-395471983", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "451", "networks": [{"uuid": "ab239bdb-1532-47da-b608-6197271e259a"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:00.304266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] validate_networks() for [('ab239bdb-1532-47da-b608-6197271e259a', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:04:01.313774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:04:01.314420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:04:01.315012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:04:01.315505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:04:01.316187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:04:01.327004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:01.327954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:01.328448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:01.329610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:01.330364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:01.330810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:01.356437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Getting quotas for project f20ae5c8217d412289d006ab9e01e7f4. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:01.363257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Getting quotas for user 124511010d044f8fb07dc09e7d754303 and project f20ae5c8217d412289d006ab9e01e7f4. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:01.378082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:04:01.380641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:04:01.434719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08459cdc-7cf3-4ccc-a4c4-8528428d6a0c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.212190 Oct 15 21:04:01.436446 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2218/3332] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 1215 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:04:01.441791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:01.442910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:01.450924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:01.452033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:01.452693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:01.453687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:01.454361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:01.455126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:02.051477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cfd953fb-e3cd-4a1e-9a63-8c913335528b tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d/os-interface" status: 200 len: 1832 microversion: 2.42 time: 0.613514 Oct 15 21:04:02.054390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2219/3333] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 21:04:01 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d/os-interface => generated 1832 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:02.061061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:02.061929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:02.070121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:02.070918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:02.071600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:02.072894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:02.073759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:02.074372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:02.534237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:02.545978 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1df60d12-3d95-48e1-8659-1d56529a9e24 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1327 microversion: 2.1 time: 0.488654 Oct 15 21:04:02.547124 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2220/3334] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:02 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1327 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:02.552899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.4.132"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:02.836836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:02.851790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:02.852832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:02.853337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:02.854888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:02.855279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:02.856125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:03.789977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "refresh_cache-085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:04:03.790608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquired lock "refresh_cache-085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:04:06.246763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:04:07.563013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Updating instance_info_cache with network_info: [{"id": "ec9d8bc4-aca9-4037-991f-3318ae367cbe", "address": "fa:16:3e:fe:bb:a8", "network": {"id": "3a03a03d-6802-41ab-b356-830701974cbd", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-1335856832-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f278bb578eca4b5895634257d5ac832e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapec9d8bc4-ac", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:04:07.578612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Releasing lock "refresh_cache-085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:04:07.579679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c372098f-2a03-496a-9a08-70bce8fa78e6 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d/action" status: 202 len: 0 microversion: 2.1 time: 5.031290 Oct 15 21:04:07.580398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2221/3335] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:04:02 2022] POST /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d/action => generated 0 bytes in 5033 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:04:07.589548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}, {"name": "network-changed", "server_uuid": "085f9f3b-ac01-40f4-b859-c0b4a41f5d1d", "tag": "ec9d8bc4-aca9-4037-991f-3318ae367cbe"}, {"name": "network-changed", "server_uuid": "ecdc3ab5-4d71-4604-9dba-bc9338bfafe2", "tag": "3fb330a2-3034-4b0f-82fa-330b45824727"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:07.602774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:07.603403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:07.604103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:07.619118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Creating event network-changed:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:04:07.619725 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Creating event network-changed:ec9d8bc4-aca9-4037-991f-3318ae367cbe for instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d on nch2-master-1 Oct 15 21:04:07.620337 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] Creating event network-changed:3fb330a2-3034-4b0f-82fa-330b45824727 for instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 on nch1-master-1 Oct 15 21:04:07.632704 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-127ca49f-401f-465e-95e0-3cfc1f8445fb req-5f2edf4b-9b7e-4c40-bf9d-879977a1f22c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.046612 Oct 15 21:04:07.633690 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2222/3336] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:07 2022] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:07.640888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0ff45c02-2dfd-4bf5-8a2e-a85745bf3802 req-e135bd03-f3e2-4fdc-af52-04099704325d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39bc88ae-5f1d-4b2e-b803-251ca3efac5c", "name": "network-vif-plugged", "status": "completed", "tag": "2c0abf87-a1c2-4694-9a7c-3560674a208d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:07.663486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ff45c02-2dfd-4bf5-8a2e-a85745bf3802 req-e135bd03-f3e2-4fdc-af52-04099704325d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:07.664093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ff45c02-2dfd-4bf5-8a2e-a85745bf3802 req-e135bd03-f3e2-4fdc-af52-04099704325d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:07.664640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ff45c02-2dfd-4bf5-8a2e-a85745bf3802 req-e135bd03-f3e2-4fdc-af52-04099704325d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:07.679058 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0ff45c02-2dfd-4bf5-8a2e-a85745bf3802 req-e135bd03-f3e2-4fdc-af52-04099704325d service nova] Creating event network-vif-plugged:2c0abf87-a1c2-4694-9a7c-3560674a208d for instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c on nch2-master-1 Oct 15 21:04:07.684260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0ff45c02-2dfd-4bf5-8a2e-a85745bf3802 req-e135bd03-f3e2-4fdc-af52-04099704325d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048858 Oct 15 21:04:07.687101 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2223/3337] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:07.693911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:07.694730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:07.716390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:07.717003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:07.717565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:07.718892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:07.719766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:07.720440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:08.326267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:08.328638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=92,nova_cell1:SAVEPOINT=3,nova_cell1:RELEASE=3,nova_cell1:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:08.336968 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a46a352-d071-444e-add3-df802b776f27 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1521 microversion: 2.1 time: 0.648586 Oct 15 21:04:08.337587 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2224/3338] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:07 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1521 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:08.342578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:08.343199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:08.349698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:08.350199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:08.350644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:08.351355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:08.351823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:08.352262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:08.754408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:08.762892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce52ae41-4475-44e4-86cc-af3895bd9fca tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1317 microversion: 2.1 time: 0.424275 Oct 15 21:04:08.763480 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2225/3339] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:08 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1317 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:08.768233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:08.768905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:08.774943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:08.775433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:08.775882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:08.776585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:08.777073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:08.777506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:09.152773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:09.160592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e83c22eb-b1a5-453a-b20f-314054f98b46 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1320 microversion: 2.1 time: 0.396053 Oct 15 21:04:09.161232 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2226/3340] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:08 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1320 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:09.165829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:09.166490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:09.172243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:09.172930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:09.173580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:09.174298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:09.175170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:09.175650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.000562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:10.011810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:10.013712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:10.014577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75a4e4e7-0161-471b-9738-840b7cf2af51 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2945 microversion: 2.32 time: 0.852407 Oct 15 21:04:10.015199 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2227/3341] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:09 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2945 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:10.019919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:10.020932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:10.028576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.029090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.029768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.030673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.031291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.031876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.069221 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a538beb9-00a8-436f-80a8-326389c510a9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments/e7f34b49-9f58-47cb-8421-8d029d1523f7" status: 200 len: 194 microversion: 2.20 time: 0.052902 Oct 15 21:04:10.069985 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2228/3342] 10.222.0.16 () {60 vars in 1454 bytes} [Sat Oct 15 21:04:10 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments/e7f34b49-9f58-47cb-8421-8d029d1523f7 => generated 194 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:04:10.074589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:10.075204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:10.081544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.082045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.082487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.084239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.085956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.086404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.477321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:10.493114 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed80d0d1-2143-4688-89d1-535277112349 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1528 microversion: 2.1 time: 0.421869 Oct 15 21:04:10.493711 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2229/3343] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:10 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1528 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:10.510742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:10.512240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:10.518821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.519328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.519777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.520823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.521874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.522936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.550668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:10.643626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-588c77ea-6f9f-4383-9b1b-6c23aa0691d5 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "DELETE /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 204 len: 0 microversion: 2.1 time: 0.137020 Oct 15 21:04:10.644232 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2230/3344] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:10 2022] DELETE /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:10.651436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:10.652899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:10.658869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.659332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.659775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.660520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.661787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.662239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.817069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:10.825483 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d23fbc01-b172-43ec-afb4-2f144b3a2f80 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1521 microversion: 2.1 time: 0.180051 Oct 15 21:04:10.826058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2231/3345] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:10 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1521 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:10.830519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ecdc3ab5-4d71-4604-9dba-bc9338bfafe2", "name": "network-vif-plugged", "status": "completed", "tag": "3fb330a2-3034-4b0f-82fa-330b45824727"}, {"server_uuid": "29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820", "name": "network-vif-plugged", "status": "completed", "tag": "8cf88a8d-33e1-47f1-bf30-24878a31214c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:10.840697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.841239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.841670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.854558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] Creating event network-vif-plugged:3fb330a2-3034-4b0f-82fa-330b45824727 for instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 on nch1-master-1 Oct 15 21:04:10.855141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] Creating event network-vif-plugged:8cf88a8d-33e1-47f1-bf30-24878a31214c for instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 on nch2-master-1 Oct 15 21:04:10.864637 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-82176e31-7855-417c-b6fc-8335cabbcbd3 req-fcb16359-8680-41fe-8ae4-d93e1ebf5319 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.036739 Oct 15 21:04:10.865696 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2232/3346] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:10 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:10.870823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820", "tag": "8cf88a8d-33e1-47f1-bf30-24878a31214c"}, {"name": "network-changed", "server_uuid": "085f9f3b-ac01-40f4-b859-c0b4a41f5d1d", "tag": "ec9d8bc4-aca9-4037-991f-3318ae367cbe"}, {"name": "network-changed", "server_uuid": "5f318670-01e5-4565-8e32-ac7ef73ba33c", "tag": "b779e022-9b5e-4fc4-8c92-ae2e834e6c81"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:10.882634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.883115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.883548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.898231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Creating event network-changed:8cf88a8d-33e1-47f1-bf30-24878a31214c for instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 on nch2-master-1 Oct 15 21:04:10.898746 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Creating event network-changed:ec9d8bc4-aca9-4037-991f-3318ae367cbe for instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d on nch2-master-1 Oct 15 21:04:10.899251 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] Creating event network-changed:b779e022-9b5e-4fc4-8c92-ae2e834e6c81 for instance 5f318670-01e5-4565-8e32-ac7ef73ba33c on nch1-master-1 Oct 15 21:04:10.908780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e096ed3e-2392-4571-be99-e0f3ab6f28f5 req-56326fc8-76ab-43d4-859c-6f541ac988d1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.041676 Oct 15 21:04:10.909417 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2233/3347] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:10 2022] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:10.914691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:10.915385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:10.921591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.922079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.922516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:10.923212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:10.923665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:10.924089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.216943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:11.224926 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d27beb7-eae1-40c1-84f8-40e993515c6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1317 microversion: 2.1 time: 0.314298 Oct 15 21:04:11.225582 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2234/3348] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:10 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1317 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:11.230518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:11.231015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:11.237165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.237659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.238105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.238808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.239278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.239713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.344718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:11.355478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c8d779a-a5a3-4b7d-912b-10daf328ab8c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d/os-interface" status: 200 len: 1832 microversion: 2.32 time: 0.128704 Oct 15 21:04:11.356071 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2235/3349] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 21:04:11 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d/os-interface => generated 1832 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:11.360850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:11.361425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:11.368188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.368696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.369165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.369873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.370441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.370767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.371205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=11 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:11.458827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lazy-loading 'services' on Instance uuid 8dc308ad-212c-4df0-baa6-f446dac1d979 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:04:11.532866 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c907b3fa-a16a-4d6a-8bea-2684da3e171e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments/e7f34b49-9f58-47cb-8421-8d029d1523f7" status: 202 len: 0 microversion: 2.20 time: 0.175439 Oct 15 21:04:11.533430 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2236/3350] 10.222.0.16 () {60 vars in 1457 bytes} [Sat Oct 15 21:04:11 2022] DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments/e7f34b49-9f58-47cb-8421-8d029d1523f7 => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:11.539595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:11.540090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:11.547039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.547428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.547880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.548597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.549112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.549550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.779201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:11.787121 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1628a884-ca17-4703-94a6-b8f84e0e4cf3 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1320 microversion: 2.1 time: 0.252293 Oct 15 21:04:11.787706 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2237/3351] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:11 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1320 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:11.792113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:11.792766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:11.798476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.799003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.799453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.800156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.800621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.801089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.933567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:11.941308 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b77e61b3-b66d-4138-a748-2c2d8c919fcb tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1528 microversion: 2.1 time: 0.152621 Oct 15 21:04:11.941877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2238/3352] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:11 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1528 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:11.948648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.4.132"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:11.951211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:11.957424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.957951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.958398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.959112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:11.959578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:11.960012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:11.985893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:04:11.986303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquired lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:04:13.549932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:04:14.310309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Updating instance_info_cache with network_info: [{"id": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d", "address": "fa:16:3e:80:ea:d2", "network": {"id": "3a03a03d-6802-41ab-b356-830701974cbd", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-1335856832-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.4.132", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f278bb578eca4b5895634257d5ac832e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap0bf5cf03-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:04:14.318406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=2,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:14.324579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Releasing lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:04:14.325714 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e25913f8-4c29-4228-9fd3-c048212dc8bc tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action" status: 202 len: 0 microversion: 2.1 time: 2.381586 Oct 15 21:04:14.326407 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2239/3353] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:04:11 2022] POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action => generated 0 bytes in 2384 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:04:14.331893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1601007180"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:14.334551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:14.342038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.342537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.342983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.343705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.344170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.344607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.568347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bf262a7c-2fa6-4f1f-8b56-c5afddf15a17 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c/action" status: 202 len: 0 microversion: 2.1 time: 0.240416 Oct 15 21:04:14.570204 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2240/3354] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:04:14 2022] POST /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c/action => generated 0 bytes in 243 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 21:04:14.575840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:14.578555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:14.587301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.587485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.588244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.589169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.589802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.590409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.626736 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1d1f8366-ad58-4e4b-976e-5ac85ea614e0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.055096 Oct 15 21:04:14.627939 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2241/3355] 10.222.0.16 () {60 vars in 1343 bytes} [Sat Oct 15 21:04:14 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:14.803892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:14.804407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:14.811092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.811599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.812190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.812911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.813547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.813980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.829763 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] HTTP exception thrown: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Request method failure captured: Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c HTTP/1.1 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyAqR5zADoe_7c6kbvoinnhYFf-B_58rUW6vbcopDq8ZBBXz0-nnSAAe1AuU1zJkhgXfchSXf4ON-yCzCxAXkuPs_8L0MGf9ydTXphzBstG5W5S7U2OtEEj988Jt_Z1C86SxT_xjZTq7U4Ot1wlbVGPfo-BDvEikaNTKqgl77CkjZME Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 42cfcf1dcb864013ac16376c2419d05f Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-442748238 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-442748238 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 42cfcf1dcb864013ac16376c2419d05f Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-442748238 Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestMultiTenantJSON-442748238-project Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 7d02831cb7e24fc69671dd999f4746fb Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-442748238-project Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '39bc88ae-5f1d-4b2e-b803-251ca3efac5c'} Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:14.831588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Returning 404 to user: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:14.836770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd359f01-a634-4a7a-9e57-774ca62c0252 tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] 10.222.0.16 "DELETE /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 404 len: 111 microversion: 2.1 time: 0.203855 Oct 15 21:04:14.838669 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2242/3356] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:14 2022] DELETE /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 111 bytes in 209 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:14.842433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:14.843059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:14.848987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.849524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.849967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:14.850674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:14.851141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:14.851594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:15.050833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:15.059203 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8fce9b03-ff20-411b-b2e9-3040789fb0f4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1317 microversion: 2.1 time: 0.220416 Oct 15 21:04:15.059920 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2243/3357] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:14 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1317 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:15.064586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:15.065254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:15.072050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:15.072770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:15.073405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:15.074267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:15.074887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:15.075321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:15.456914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:15.466465 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b2aa7f8-f537-4399-8945-48ee7124ae17 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1320 microversion: 2.1 time: 0.405403 Oct 15 21:04:15.467044 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2244/3358] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:15 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1320 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:15.474579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-107af6b3-4613-472a-a303-0bf711db0ef5 req-290ea973-5e13-4a3d-8340-41cf8270b6b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f318670-01e5-4565-8e32-ac7ef73ba33c", "name": "network-vif-plugged", "status": "completed", "tag": "b779e022-9b5e-4fc4-8c92-ae2e834e6c81"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:15.483504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-107af6b3-4613-472a-a303-0bf711db0ef5 req-290ea973-5e13-4a3d-8340-41cf8270b6b0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:15.484035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-107af6b3-4613-472a-a303-0bf711db0ef5 req-290ea973-5e13-4a3d-8340-41cf8270b6b0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:15.484633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-107af6b3-4613-472a-a303-0bf711db0ef5 req-290ea973-5e13-4a3d-8340-41cf8270b6b0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:15.497444 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-107af6b3-4613-472a-a303-0bf711db0ef5 req-290ea973-5e13-4a3d-8340-41cf8270b6b0 service nova] Creating event network-vif-plugged:b779e022-9b5e-4fc4-8c92-ae2e834e6c81 for instance 5f318670-01e5-4565-8e32-ac7ef73ba33c on nch1-master-1 Oct 15 21:04:15.502275 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-107af6b3-4613-472a-a303-0bf711db0ef5 req-290ea973-5e13-4a3d-8340-41cf8270b6b0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033526 Oct 15 21:04:15.503370 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2245/3359] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:15.511106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bb0bcc38-3db3-4c3f-a38e-01b3aa7a5dd6 req-75281227-0f8c-4042-bdb9-9a412adb8048 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35c7b923-29cf-4679-ae98-f8f8b14141aa", "tag": "b54ef53b-2708-4479-8161-64ddb65c9afb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:15.522001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bb0bcc38-3db3-4c3f-a38e-01b3aa7a5dd6 req-75281227-0f8c-4042-bdb9-9a412adb8048 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:15.522489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bb0bcc38-3db3-4c3f-a38e-01b3aa7a5dd6 req-75281227-0f8c-4042-bdb9-9a412adb8048 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:15.522923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bb0bcc38-3db3-4c3f-a38e-01b3aa7a5dd6 req-75281227-0f8c-4042-bdb9-9a412adb8048 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:15.552603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-bb0bcc38-3db3-4c3f-a38e-01b3aa7a5dd6 req-75281227-0f8c-4042-bdb9-9a412adb8048 service nova] Creating event network-changed:b54ef53b-2708-4479-8161-64ddb65c9afb for instance 35c7b923-29cf-4679-ae98-f8f8b14141aa on nch2-master-1 Oct 15 21:04:15.559777 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bb0bcc38-3db3-4c3f-a38e-01b3aa7a5dd6 req-75281227-0f8c-4042-bdb9-9a412adb8048 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052289 Oct 15 21:04:15.560707 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2246/3360] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:15.566080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f28c83d3-e870-4f98-ae0b-786dd76b7f69 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '.version_select at 0x7fe7583524c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:16.476794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f28c83d3-e870-4f98-ae0b-786dd76b7f69 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/f81806a8-9334-4147-a0b9-bbec40dbf77b" status: 200 len: 184 microversion: 2.1 time: 0.914412 Oct 15 21:04:16.478387 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2247/3361] 10.222.0.16 () {58 vars in 1260 bytes} [Sat Oct 15 21:04:15 2022] GET /compute/v2.1/os-floating-ips/f81806a8-9334-4147-a0b9-bbec40dbf77b => generated 184 bytes in 917 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:16.483397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83c2179a-e9d4-4def-afda-90e3a55e2dbe tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '.version_select at 0x7fe74fa46280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:16.624473 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83c2179a-e9d4-4def-afda-90e3a55e2dbe tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd" status: 204 len: 0 microversion: 2.1 time: 0.144712 Oct 15 21:04:16.625241 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2248/3362] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:16 2022] DELETE /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:16.630473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:16.631194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:16.633500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:16.640375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:16.640875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:16.641357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:16.642062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:16.642527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:16.642961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:16.670821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:16.756982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-42371b49-22aa-4077-82ee-936bedfcdea7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 204 len: 0 microversion: 2.20 time: 0.130421 Oct 15 21:04:16.763828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2249/3363] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:04:16 2022] DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:04:16.771393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:16.772214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:16.778347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:16.778839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:16.779279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:16.779982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:16.780443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:16.780904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:16.967279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:16.975892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-298b9a0a-c53a-4c61-ac10-49c8babbad2f tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1528 microversion: 2.1 time: 0.210781 Oct 15 21:04:16.976448 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2250/3364] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:16 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1528 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:16.981017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:16.981730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:16.987570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:16.988059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:16.988497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:16.989287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:16.989836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:16.991246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:17.126163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:17.134787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-537e3623-3f94-4320-b0d9-4243a7a79afe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1317 microversion: 2.1 time: 0.157155 Oct 15 21:04:17.135230 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2251/3365] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:16 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1317 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:17.140590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:17.141317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:17.147397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:17.147887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:17.148326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:17.149021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:17.149502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:17.149929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:17.280171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:17.288563 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2363b44-956f-4b6b-916a-197765114d9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1320 microversion: 2.1 time: 0.152256 Oct 15 21:04:17.289159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2252/3366] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:17 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1320 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:17.293694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.4.132"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:17.422290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:17.429229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:17.429711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:17.430197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:17.430922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:17.431381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:17.431805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:17.640087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:04:17.640645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquired lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:04:18.272893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:04:18.813240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Updating instance_info_cache with network_info: [{"id": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d", "address": "fa:16:3e:80:ea:d2", "network": {"id": "3a03a03d-6802-41ab-b356-830701974cbd", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-1335856832-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f278bb578eca4b5895634257d5ac832e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap0bf5cf03-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:04:18.827589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Releasing lock "refresh_cache-d75b558a-f4a1-47f7-977c-d9fb11cfefa3" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:04:18.828574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-386409b2-65f9-43d1-a28c-4eea27c50ca9 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action" status: 202 len: 0 microversion: 2.1 time: 1.538463 Oct 15 21:04:18.829314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2253/3367] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:04:17 2022] POST /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3/action => generated 0 bytes in 1540 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:04:18.835029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-91b3db05-0f80-4e06-b072-23623649fb85 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '.version_select at 0x7fe74fa3cdc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:18.856086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-91b3db05-0f80-4e06-b072-23623649fb85 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Image not found. Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-91b3db05-0f80-4e06-b072-23623649fb85 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd HTTP/1.1 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd'} Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Oct 15 21:04:18.858337 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get(image_id) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd could not be found. Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-91b3db05-0f80-4e06-b072-23623649fb85 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-91b3db05-0f80-4e06-b072-23623649fb85 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd" status: 404 len: 62 microversion: 2.1 time: 0.030186 Oct 15 21:04:18.863975 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2254/3368] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Oct 15 21:04:18 2022] GET /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd => generated 62 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:04:18.868048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:18.868687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:18.877815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:18.879054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:18.880190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:18.881209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:18.882058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:18.882593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.122586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:19.132615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:19.134139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:19.135678 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-972d1bdf-a1c6-41af-8d6c-97b70bd9fbe9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 200 len: 1725 microversion: 2.20 time: 0.272802 Oct 15 21:04:19.136218 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2255/3369] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:18 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 1725 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:19.141711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Action: 'update', calling method: >, body: {"server": {"name": "39bc88ae-5f1d-4b2e-b803-251ca3efac5c_new"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:19.144195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:19.150401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.151005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.151334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.152063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.152520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.152997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.169286 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] HTTP exception thrown: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Request method failure captured: Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c HTTP/1.1 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 64 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyAqR5zADoe_7c6kbvoinnhYFf-B_58rUW6vbcopDq8ZBBXz0-nnSAAe1AuU1zJkhgXfchSXf4ON-yCzCxAXkuPs_8L0MGf9ydTXphzBstG5W5S7U2OtEEj988Jt_Z1C86SxT_xjZTq7U4Ot1wlbVGPfo-BDvEikaNTKqgl77CkjZME Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 42cfcf1dcb864013ac16376c2419d05f Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-442748238 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/42cfcf1dcb864013ac16376c2419d05f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-442748238 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 42cfcf1dcb864013ac16376c2419d05f Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-442748238 Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestMultiTenantJSON-442748238-project Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 7d02831cb7e24fc69671dd999f4746fb Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-442748238-project Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "39bc88ae-5f1d-4b2e-b803-251ca3efac5c_new"}} Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '39bc88ae-5f1d-4b2e-b803-251ca3efac5c', 'body': {'server': {'name': '39bc88ae-5f1d-4b2e-b803-251ca3efac5c_new'}}} Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:19.171041 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 915, in update Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(ctxt, req, id, is_detail=True) Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] Returning 404 to user: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ec39c38-5f95-4df7-8102-4ca4fda9a90e tempest-ServersNegativeTestMultiTenantJSON-442748238 tempest-ServersNegativeTestMultiTenantJSON-442748238-project] 10.222.0.16 "PUT /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 404 len: 111 microversion: 2.1 time: 0.035007 Oct 15 21:04:19.175631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2256/3370] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:04:19 2022] PUT /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:19.178687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}, {"server_uuid": "085f9f3b-ac01-40f4-b859-c0b4a41f5d1d", "name": "network-vif-deleted", "tag": "ec9d8bc4-aca9-4037-991f-3318ae367cbe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:19.189239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.189843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.190290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.205137 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] Creating event network-changed:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:04:19.205551 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] Creating event network-vif-deleted:ec9d8bc4-aca9-4037-991f-3318ae367cbe for instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d on nch2-master-1 Oct 15 21:04:19.216116 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1bf4c989-4f9c-4cb5-bedb-546b4c86e4f9 req-c7f8a2e3-5361-4336-b03f-03168418d89d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.041546 Oct 15 21:04:19.216841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2257/3371] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:19 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:19.222571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-28ec2263-f6eb-4416-b23d-a0325d6a7f1c req-55e0a049-5a77-48f3-aeff-3dec6c5b08d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35c7b923-29cf-4679-ae98-f8f8b14141aa", "name": "network-vif-plugged", "status": "completed", "tag": "b54ef53b-2708-4479-8161-64ddb65c9afb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:19.232272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-28ec2263-f6eb-4416-b23d-a0325d6a7f1c req-55e0a049-5a77-48f3-aeff-3dec6c5b08d8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.232752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-28ec2263-f6eb-4416-b23d-a0325d6a7f1c req-55e0a049-5a77-48f3-aeff-3dec6c5b08d8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.233246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-28ec2263-f6eb-4416-b23d-a0325d6a7f1c req-55e0a049-5a77-48f3-aeff-3dec6c5b08d8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.245806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-28ec2263-f6eb-4416-b23d-a0325d6a7f1c req-55e0a049-5a77-48f3-aeff-3dec6c5b08d8 service nova] Creating event network-vif-plugged:b54ef53b-2708-4479-8161-64ddb65c9afb for instance 35c7b923-29cf-4679-ae98-f8f8b14141aa on nch2-master-1 Oct 15 21:04:19.251206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-28ec2263-f6eb-4416-b23d-a0325d6a7f1c req-55e0a049-5a77-48f3-aeff-3dec6c5b08d8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032885 Oct 15 21:04:19.252137 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2258/3372] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:19.259247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:19.259962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:19.268535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.269030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.269487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.270346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.270901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.271333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.705151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:19.708425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=3,nova_cell1:INSERT=2,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:19.715685 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-936c98b2-f384-4d5b-a24e-62aae6baa31e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.462358 Oct 15 21:04:19.716273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2259/3373] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:19 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:19.721729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:19.722335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:19.728866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.729451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.730039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.732302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:19.732768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:19.733240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:19.970676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:19.980566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6db5518-1d15-4cff-b4b2-c18282839ad9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1320 microversion: 2.1 time: 0.263211 Oct 15 21:04:19.981229 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2260/3374] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:19 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1320 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:20.003811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:20.006060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:20.013899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:20.014811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:20.015930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:20.016855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:20.017547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:20.017991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:20.036675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] HTTP exception thrown: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Request method failure captured: Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d HTTP/1.1 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_2U0wpi9iLFol51425g7xlLgJZYv44qoqxJRdbBxz4YPFPBsxPKv03xFKBNW2j7nPvjTCKZ6yrS0YXfJwzQHSc5oH5nUDBioVusEka57vt3SlAQpUJ_M9mPyJ1PHcTih2cxI9UASJHph_vYYfIrHewGVsrJOqVsxtD0fQRIL22RbQ Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f278bb578eca4b5895634257d5ac832e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f2295a9d9b034667957b8e1d3c4901ab Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '085f9f3b-ac01-40f4-b859-c0b4a41f5d1d'} Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:20.038473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Returning 404 to user: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e8bf529-2a52-4e6d-b5cd-40b1c24a9879 tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "DELETE /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 404 len: 111 microversion: 2.1 time: 0.044538 Oct 15 21:04:20.044190 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2261/3375] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:19 2022] DELETE /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:20.046660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:20.046847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:20.054162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:20.054665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:20.055108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:20.055832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:20.056689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:20.057013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:20.575116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:20.584624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-317cc968-2ef3-4be8-9158-a42a89661dfc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1521 microversion: 2.1 time: 0.542856 Oct 15 21:04:20.597311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2262/3376] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:20 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1521 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:20.597311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:20.597311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:20.621345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:20.621345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:20.621345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:20.625558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:20.629271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:20.629271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:20.684445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:20.817195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-999c0745-f111-49e8-aefd-1252bfe7ba31 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "DELETE /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 204 len: 0 microversion: 2.1 time: 0.229425 Oct 15 21:04:20.821701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2263/3377] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:20 2022] DELETE /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:20.831043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-602983429", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:20.906872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] validate_networks() for [('e3c96722-97ae-4b06-ab5c-8a523bae70f3', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:04:21.316058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:04:21.316677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:04:21.317133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:04:21.317556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:04:21.318181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:04:21.327875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.328372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.328818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.329726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.330219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.330690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.352494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Getting quotas for project 172114b9a46e4df8ae8076ed582f0b5f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:21.358995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Getting quotas for user 90e8754a100240a498e422de07c76bbf and project 172114b9a46e4df8ae8076ed582f0b5f. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:21.374838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:04:21.377293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:04:21.428445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8fe4cbd6-5155-433f-9193-3b4675c78fc2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.602443 Oct 15 21:04:21.429614 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2264/3378] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 607 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:04:21.436369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:21.437021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:21.445365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.446001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.446609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.447330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.447879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.448386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.467554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] HTTP exception thrown: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Request method failure captured: Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d HTTP/1.1 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_2U0wpi9iLFol51425g7xlLgJZYv44qoqxJRdbBxz4YPFPBsxPKv03xFKBNW2j7nPvjTCKZ6yrS0YXfJwzQHSc5oH5nUDBioVusEka57vt3SlAQpUJ_M9mPyJ1PHcTih2cxI9UASJHph_vYYfIrHewGVsrJOqVsxtD0fQRIL22RbQ Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f278bb578eca4b5895634257d5ac832e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f2295a9d9b034667957b8e1d3c4901ab Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '085f9f3b-ac01-40f4-b859-c0b4a41f5d1d'} Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.470173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Returning 404 to user: Instance 085f9f3b-ac01-40f4-b859-c0b4a41f5d1d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d806917-4d2a-49ef-bfba-aa774a79d23a tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d" status: 404 len: 111 microversion: 2.1 time: 0.041129 Oct 15 21:04:21.475964 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2265/3379] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:21 2022] GET /compute/v2.1/servers/085f9f3b-ac01-40f4-b859-c0b4a41f5d1d => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:21.478302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:21.479193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:21.485508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.485999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.486437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.487135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.487593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.488019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.507162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] HTTP exception thrown: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Request method failure captured: Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 HTTP/1.1 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9tfj-IfeDlmhCVopaxKqpvSD6hcfJy661w2SylEc8Y4h2qBwVnqJG8EPnKb0eyLrD36V761YW1_ot578qNdTTF4wGF86vEaSHTq2fmlx2SGsXOHDZnkb0j7gd99vnTAGZaImphiObDk2NPNtACw5rhiHELEFOKd6GSfPiHcqDCZvU Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.20 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c04b2976577d42799a800d21545d1969"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c04b2976577d42799a800d21545d1969"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fb8dc371c65f4bbb92d559c7ed78964c Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8dc308ad-212c-4df0-baa6-f446dac1d979'} Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.508904 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Returning 404 to user: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99e9b494-cc6c-4b66-8578-232ed3cdb670 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 404 len: 111 microversion: 2.20 time: 0.035778 Oct 15 21:04:21.514560 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2266/3380] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:21 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:04:21.516159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:21.516546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:21.523394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.523876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.524311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.525024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.525505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.525926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.556791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:21.650151 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2bc2db0f-ecda-48c8-b06e-c9337bd1aed6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 204 len: 0 microversion: 2.1 time: 0.137590 Oct 15 21:04:21.650933 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2267/3381] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:21 2022] DELETE /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:21.657326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:21.658030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:21.664805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.665311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.665752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.666455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.666911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.667336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.820437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14c805d8b3e944e8ad9f6150322b96ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c6d5585dc2254ed2aed773305d209049', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:21.830536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10ffc96d-5824-4d67-8124-14a300d078c8 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 200 len: 1534 microversion: 2.1 time: 0.178201 Oct 15 21:04:21.831760 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2268/3382] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:21 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 1534 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:21.838296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:21.838954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:21.850631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.851134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.851814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:21.852705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:21.853430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:21.854066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:22.000075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:22.008574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4dcb6ee6-d241-494e-9b15-8018881f6164 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1320 microversion: 2.1 time: 0.174778 Oct 15 21:04:22.009302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2269/3383] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:21 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1320 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:22.014135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}, {"server_uuid": "8dc308ad-212c-4df0-baa6-f446dac1d979", "name": "network-vif-deleted", "tag": "2c0beae0-39a4-4ed3-a26b-9fb323c358d3"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:22.025151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:22.025882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:22.026327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:22.039351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] Creating event network-changed:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:04:22.039882 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] Creating event network-vif-deleted:2c0beae0-39a4-4ed3-a26b-9fb323c358d3 for instance 8dc308ad-212c-4df0-baa6-f446dac1d979 on nch1-master-1 Oct 15 21:04:22.048210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7c4de9e1-0f6c-4de5-a2e9-9c1ed483bcec req-be33d3be-4451-4c3b-9e29-753b2390145e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.037382 Oct 15 21:04:22.048918 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2270/3384] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:22 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:22.054522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:22.055230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:22.062094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:22.062591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:22.063041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:22.063747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:22.064228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:22.064652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:22.170484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:22.178765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3e012e4-c6a6-4ccd-a43e-ef48745676e8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1220 microversion: 2.1 time: 0.128681 Oct 15 21:04:22.179332 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2271/3385] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:22 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1220 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:22.183828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-278603bf-6f18-48b6-a6f4-847a3884808b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '.version_select at 0x7fe754418dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:24.217228 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-278603bf-6f18-48b6-a6f4-847a3884808b tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/f81806a8-9334-4147-a0b9-bbec40dbf77b" status: 202 len: 0 microversion: 2.1 time: 2.036779 Oct 15 21:04:24.218035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2272/3386] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:04:22 2022] DELETE /compute/v2.1/os-floating-ips/f81806a8-9334-4147-a0b9-bbec40dbf77b => generated 0 bytes in 2038 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:04:24.223475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9457c80-d7fa-4a8f-be8f-4818f75655cb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '.version_select at 0x7fe74f881dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:24.245864 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9457c80-d7fa-4a8f-be8f-4818f75655cb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1037964320" status: 204 len: 0 microversion: 2.20 time: 0.026600 Oct 15 21:04:24.246488 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2273/3387] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 21:04:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1037964320 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:04:24.267871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:24.268446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:24.279671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:24.281496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:24.282340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:24.283218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:24.283941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:24.284540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:24.617730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:24.625803 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71f91293-82b1-44db-bf1c-92d4eb81294b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 200 len: 1527 microversion: 2.1 time: 0.378104 Oct 15 21:04:24.633152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2274/3388] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:24 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 1527 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:24.633152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:24.633152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] [instance: 39bc88ae-5f1d-4b2e-b803-251ca3efac5c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:24.639041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:24.639536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:24.640187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:24.640993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:24.641544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:24.641979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:24.657678 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] HTTP exception thrown: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Request method failure captured: Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c HTTP/1.1 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyAe7F1oQpCBo6ll-tgMJE0cD6xMKoLpmS9C_msoawHjHOK2RcEU-a0OOSu8JgrywsHrdgS3ATSk3N5HNo_jdijMRHl6EbxgURi_xPvqpRx8HegzuKrwxIDXpJwjKMb1PUP-1saGf-0ldQlqiervXSZKfzO3CbQEAxz431qeGVUne1k Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c6d5585dc2254ed2aed773305d209049 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-257858712 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c6d5585dc2254ed2aed773305d209049"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c6d5585dc2254ed2aed773305d209049"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c6d5585dc2254ed2aed773305d209049"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c6d5585dc2254ed2aed773305d209049"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-257858712 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c6d5585dc2254ed2aed773305d209049 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-257858712 Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestMultiTenantJSON-257858712-project Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 14c805d8b3e944e8ad9f6150322b96ce Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-257858712-project Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '39bc88ae-5f1d-4b2e-b803-251ca3efac5c'} Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:24.659365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] Returning 404 to user: Instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-56c4950e-01c6-430d-9555-dd0fd67e2eb6 tempest-ServersNegativeTestMultiTenantJSON-257858712 tempest-ServersNegativeTestMultiTenantJSON-257858712-project] 10.222.0.16 "GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c" status: 404 len: 111 microversion: 2.1 time: 0.033132 Oct 15 21:04:24.664297 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2275/3389] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:24 2022] GET /compute/v2.1/servers/39bc88ae-5f1d-4b2e-b803-251ca3efac5c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:24.666509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:24.667129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:24.673011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:24.673541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:24.673985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:24.674693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:24.675159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:24.675632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.080137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:25.091526 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09e3983f-7b35-4bc4-abf1-c0cf2207c44a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1510 microversion: 2.1 time: 0.428479 Oct 15 21:04:25.092438 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2276/3390] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:24 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1510 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:25.097754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:25.099803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:25.111291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.111835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.112422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.113250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.114734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.116169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.227195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:25.235480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abf075fb-e13d-4b61-9375-6476a66f68c6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1274 microversion: 2.1 time: 0.141397 Oct 15 21:04:25.236021 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2277/3391] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:25 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1274 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:25.241842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-67e858e9-9416-4b94-b83d-77cba68f3e7d req-352b6346-5e49-44d7-8e95-1b6ebb5b9a71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39bc88ae-5f1d-4b2e-b803-251ca3efac5c", "name": "network-vif-deleted", "tag": "2c0abf87-a1c2-4694-9a7c-3560674a208d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:25.252301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-67e858e9-9416-4b94-b83d-77cba68f3e7d req-352b6346-5e49-44d7-8e95-1b6ebb5b9a71 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.252791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-67e858e9-9416-4b94-b83d-77cba68f3e7d req-352b6346-5e49-44d7-8e95-1b6ebb5b9a71 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.253349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-67e858e9-9416-4b94-b83d-77cba68f3e7d req-352b6346-5e49-44d7-8e95-1b6ebb5b9a71 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.266329 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-67e858e9-9416-4b94-b83d-77cba68f3e7d req-352b6346-5e49-44d7-8e95-1b6ebb5b9a71 service nova] Creating event network-vif-deleted:2c0abf87-a1c2-4694-9a7c-3560674a208d for instance 39bc88ae-5f1d-4b2e-b803-251ca3efac5c on nch2-master-1 Oct 15 21:04:25.273010 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-67e858e9-9416-4b94-b83d-77cba68f3e7d req-352b6346-5e49-44d7-8e95-1b6ebb5b9a71 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035546 Oct 15 21:04:25.274359 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2278/3392] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:25.281796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:25.282395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:25.290011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.290524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.291220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.293084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.293730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.294335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.323341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:25.326579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:25.415404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-acb73ed0-6c86-4816-83f4-a12c5808f2bf tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 204 len: 0 microversion: 2.1 time: 0.139040 Oct 15 21:04:25.418198 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2279/3393] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:25 2022] DELETE /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:25.423894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04303ea6-a2d7-480d-b67b-ac43478fb594 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'create', calling method: .version_select at 0x7fe74fc17160>, body: {"keypair": {"name": "tempest-keypair-593384543", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBV5JIM0B3CkQ9omDJCJHqHEfTc3nR5pJ/zLFHs4zUFG3v3DOXXH9wH4Hk9aJCm+lJ1WdsnDdyDuSTmntUfR2CrQVLnVZ//EwHaQnbpTRgWnTgNYXYjf2emrQE/pGWD98Q=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:25.435403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=4,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:25.446519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-04303ea6-a2d7-480d-b67b-ac43478fb594 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for project c04b2976577d42799a800d21545d1969. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:25.454645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-04303ea6-a2d7-480d-b67b-ac43478fb594 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for user fb8dc371c65f4bbb92d559c7ed78964c and project c04b2976577d42799a800d21545d1969. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:25.481756 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04303ea6-a2d7-480d-b67b-ac43478fb594 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.062020 Oct 15 21:04:25.482289 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2280/3394] 10.222.0.16 () {62 vars in 1200 bytes} [Sat Oct 15 21:04:25 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 63 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 15 21:04:25.486863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:25.487595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:25.495471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.495966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.496405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.497139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.497601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.498029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.807205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:25.818298 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e2183dd8-b791-449a-a75f-50fa74a13082 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1512 microversion: 2.1 time: 0.334952 Oct 15 21:04:25.818931 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2281/3395] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:25 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1512 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:25.833492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:25.833492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:25.855711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.856659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.857530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:25.858550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:25.859240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:25.859935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:26.213321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2295a9d9b034667957b8e1d3c4901ab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f278bb578eca4b5895634257d5ac832e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:26.222261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0ee3b17-e762-413d-9f7f-9b058cdbb4cd tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 200 len: 1525 microversion: 2.1 time: 0.396200 Oct 15 21:04:26.223753 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2282/3396] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:25 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 1525 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:26.228262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:26.229011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:26.236563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:26.237530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:26.238388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:26.240183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:26.240667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:26.241148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:26.258618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c HTTP/1.1 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '227d10b5-22c4-44a9-b65e-de1ed6d8680c'} Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:26.260681 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:26.265724 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:26.267206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:26.268446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7dcf31cf-7d7d-46f0-bb05-82feb2e5d6c1 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 404 len: 111 microversion: 2.1 time: 0.043823 Oct 15 21:04:26.278256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2283/3397] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:26 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:26.284932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:26.286859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:26.294527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:26.295313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:26.295995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:26.297132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:26.297787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:26.298701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:26.643457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:26.656271 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b58ab483-f1a4-4e26-a44f-1f5360757dc6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 200 len: 1511 microversion: 2.1 time: 0.374607 Oct 15 21:04:26.657015 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2284/3398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:26 2022] GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 1511 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:26.663156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:26.664144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:26.680702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:26.681529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:26.682160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:26.683246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:26.683999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:26.684682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:27.368384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:27.378314 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09f64fd8-237c-44bc-a532-cde78bf2e8bc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1316 microversion: 2.1 time: 0.719358 Oct 15 21:04:27.379244 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2285/3399] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:26 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1316 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:27.385322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-299708542", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:27.456755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] validate_networks() for [('17f4161c-0a61-4650-a009-d339764cbfd5', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:04:28.750308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:04:28.750907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:04:28.752201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:04:28.752839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:04:28.753483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:04:28.767411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:28.768225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:28.768872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:28.771293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:28.771937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:28.772601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:28.795882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:28.813534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Getting quotas for user a8af9544fe68497c8557248575f184ed and project 994259bbbe124b4e9ff0542ca402b9f7. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:28.831807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:04:28.834421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:04:28.915632 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a22e888c-c63a-4fdc-8274-9a3db270ae0c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.534636 Oct 15 21:04:28.918830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2286/3400] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:27 2022] POST /compute/v2.1/servers => generated 383 bytes in 1539 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:04:28.924528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:28.937874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:28.938851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'flavor': '451', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:28.954256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:28.954765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:28.955208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:28.956095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:28.956663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:28.957712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:29.074769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:29.173785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:29.187822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:29.188328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:29.188782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:29.491678 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-483d395a-dc3f-4b86-b756-e7dfe3b27c95 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?flavor=451" status: 200 len: 1513 microversion: 2.1 time: 0.571225 Oct 15 21:04:29.492569 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2287/3401] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 21:04:28 2022] GET /compute/v2.1/servers/detail?flavor=451 => generated 1513 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:29.499255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-347806892", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a8bfc216-af8d-45e1-9e10-7c244ff5c0ea"}], "security_groups": [{"name": "tempest-securitygroup--1312470462"}], "key_name": "tempest-keypair-593384543", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:29.681456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] validate_networks() for [('a8bfc216-af8d-45e1-9e10-7c244ff5c0ea', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:04:30.172823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:04:30.173464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:04:30.173867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:04:30.174308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:04:30.174934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:04:30.185320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.185804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.186244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:30.186968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.187419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.187935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:30.216288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for project c04b2976577d42799a800d21545d1969. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:30.221553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Getting quotas for user fb8dc371c65f4bbb92d559c7ed78964c and project c04b2976577d42799a800d21545d1969. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:30.235674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:04:30.237977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:04:30.287836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9cd1eb31-9244-4b2f-b258-73171d13f529 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.793797 Oct 15 21:04:30.288515 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2288/3402] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:04:29 2022] POST /compute/v2.1/servers => generated 402 bytes in 795 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:04:30.293152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:30.293774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] [instance: d75b558a-f4a1-47f7-977c-d9fb11cfefa3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:30.300488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.300951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.301452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:30.302308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.302766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.303197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:30.317719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] HTTP exception thrown: Instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 could not be found. Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Request method failure captured: Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 HTTP/1.1 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_2U0wpi9iLFol51425g7xlLgJZYv44qoqxJRdbBxz4YPFPBsxPKv03xFKBNW2j7nPvjTCKZ6yrS0YXfJwzQHSc5oH5nUDBioVusEka57vt3SlAQpUJ_M9mPyJ1PHcTih2cxI9UASJHph_vYYfIrHewGVsrJOqVsxtD0fQRIL22RbQ Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f278bb578eca4b5895634257d5ac832e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f278bb578eca4b5895634257d5ac832e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f278bb578eca4b5895634257d5ac832e Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-915114113 Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f2295a9d9b034667957b8e1d3c4901ab Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-915114113-project Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 could not be found. Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd75b558a-f4a1-47f7-977c-d9fb11cfefa3'} Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 could not be found. Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 could not be found. Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:30.319308 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 could not be found. Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] Returning 404 to user: Instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a87f5065-83da-4cbb-a80c-0c84b375cbac tempest-FloatingIPsAssociationTestJSON-915114113 tempest-FloatingIPsAssociationTestJSON-915114113-project] 10.222.0.16 "GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3" status: 404 len: 111 microversion: 2.1 time: 0.031265 Oct 15 21:04:30.325240 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2289/3403] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:30 2022] GET /compute/v2.1/servers/d75b558a-f4a1-47f7-977c-d9fb11cfefa3 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:30.329502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c1ea499d-ee10-48e2-9307-fbd48f4e1b00 req-b0ba8ff5-d57b-4781-848c-713ec5e2a33b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "227d10b5-22c4-44a9-b65e-de1ed6d8680c", "name": "network-vif-deleted", "tag": "6ca218e1-1310-4faf-9e51-a986e94f59d7"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:30.341508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c1ea499d-ee10-48e2-9307-fbd48f4e1b00 req-b0ba8ff5-d57b-4781-848c-713ec5e2a33b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.341913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c1ea499d-ee10-48e2-9307-fbd48f4e1b00 req-b0ba8ff5-d57b-4781-848c-713ec5e2a33b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.342356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c1ea499d-ee10-48e2-9307-fbd48f4e1b00 req-b0ba8ff5-d57b-4781-848c-713ec5e2a33b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:30.359154 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c1ea499d-ee10-48e2-9307-fbd48f4e1b00 req-b0ba8ff5-d57b-4781-848c-713ec5e2a33b service nova] Creating event network-vif-deleted:6ca218e1-1310-4faf-9e51-a986e94f59d7 for instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c on nch1-master-1 Oct 15 21:04:30.364922 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c1ea499d-ee10-48e2-9307-fbd48f4e1b00 req-b0ba8ff5-d57b-4781-848c-713ec5e2a33b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038310 Oct 15 21:04:30.366181 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2290/3404] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:30.375094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:30.376135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:30.385072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.385746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.386254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:30.387123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:30.387881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:30.388394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:31.291719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:31.305561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4b8a57f-ff75-4987-acc4-2bf9f9a1eefa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1316 microversion: 2.1 time: 0.937395 Oct 15 21:04:31.323646 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2291/3405] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:30 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1316 bytes in 956 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:31.333930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:31.334563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:31.340966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:31.341857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:31.342816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:31.343781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:31.344473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:31.345478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:31.487451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:31.496038 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5257a090-5943-40da-b7ef-63dd5f4dac13 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1282 microversion: 2.1 time: 0.167274 Oct 15 21:04:31.496623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2292/3406] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:31 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1282 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:31.501390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:31.508088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:31.509098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': 'tempest-ListServerFiltersTestJSON-instance-1942351304', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:31.515075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:31.515559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:31.515992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:31.516715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:31.517320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:31.517803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:31.621172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:31.689524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:31.695808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:31.696291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:31.696731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:32.198997 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6c3cfc3-ffcc-4966-b708-bc409fd6d601 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?name=tempest-ListServerFiltersTestJSON-instance-1942351304" status: 200 len: 1515 microversion: 2.1 time: 0.701023 Oct 15 21:04:32.209774 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2293/3407] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:04:31 2022] GET /compute/v2.1/servers/detail?name=tempest-ListServerFiltersTestJSON-instance-1942351304 => generated 1515 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:32.209774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:32.209774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:32.213546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:32.214212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:32.214822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:32.215524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:32.215980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:32.216980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:32.333130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:32.342082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:32.343819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:32.345243 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66f7af0b-d4dd-4011-903c-2216ff3c4606 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1338 microversion: 2.20 time: 0.144275 Oct 15 21:04:32.346106 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2294/3408] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:32 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1338 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:32.351198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:32.356372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:32.357207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'vm_state': ['active'], 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:32.362517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:32.362999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:32.363424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:32.364353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:32.365272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:32.365698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:32.467806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:32.540086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:32.547559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:32.548083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:32.548494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:33.393914 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-65a730ff-9939-4ca6-9678-9a27883afb9b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?status=active" status: 200 len: 4516 microversion: 2.1 time: 1.046353 Oct 15 21:04:33.394548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2295/3409] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Oct 15 21:04:32 2022] GET /compute/v2.1/servers/detail?status=active => generated 4516 bytes in 1048 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:33.399942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:33.401472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:33.407892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:33.408611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:33.409299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:33.410240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:33.411094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:33.411759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:33.888060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:33.896431 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3479016-2a1d-4ac8-ac03-2bf4e31e7d28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1316 microversion: 2.1 time: 0.500282 Oct 15 21:04:33.897109 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2296/3410] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:33 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1316 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:33.902904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d75b558a-f4a1-47f7-977c-d9fb11cfefa3", "name": "network-vif-deleted", "tag": "0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d"}, {"name": "network-changed", "server_uuid": "14e27a99-2936-474b-85fd-21419ba8f525", "tag": "b1631449-5e4d-4b87-98a4-9cdf67c296d6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:33.914740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:33.915233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:33.915645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:33.918607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:33.932193 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] Creating event network-vif-deleted:0bf5cf03-3ad1-46b9-ba24-cb51c2b3fb0d for instance d75b558a-f4a1-47f7-977c-d9fb11cfefa3 on nch2-master-1 Oct 15 21:04:33.933143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] Creating event network-changed:b1631449-5e4d-4b87-98a4-9cdf67c296d6 for instance 14e27a99-2936-474b-85fd-21419ba8f525 on nch1-master-1 Oct 15 21:04:33.942887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7cfa3089-753e-48b5-a693-f1b64f150b36 req-696e2371-1c6d-4643-b9c9-a52e87e57fbc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.044510 Oct 15 21:04:33.943503 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2297/3411] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:33 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:33.948865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:33.949622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:33.956100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:33.956596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:33.957065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:33.957864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:33.958339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:33.958767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:34.657605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:34.665964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d8b378f-e045-4bf6-8cfc-056f6f191a3c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.721278 Oct 15 21:04:34.667120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2298/3412] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:33 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 723 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:34.671862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:34.672468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:34.678601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:34.679087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:34.679738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:34.680441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:34.681089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:34.681761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:34.893377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:34.901703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:34.903153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:34.904259 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f4d2645-5cb1-4bb5-be3a-73b1b17df086 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1338 microversion: 2.20 time: 0.236027 Oct 15 21:04:34.904821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2299/3413] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:34 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1338 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:34.909140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:34.915430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:34.915896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.servers [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Removing options 'limit' from query {{(pid=138688) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Oct 15 21:04:34.917106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:34.922234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:34.922720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:34.923164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:34.924008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:34.924375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:34.924848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:34.959004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:34.964767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:34.965293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:34.965744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:35.890052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c98089bb-2044-4dd3-bbfd-5b8c7eb87c29 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?limit=1" status: 200 len: 1659 microversion: 2.1 time: 0.984085 Oct 15 21:04:35.891540 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2300/3414] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 21:04:34 2022] GET /compute/v2.1/servers/detail?limit=1 => generated 1659 bytes in 986 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:35.899098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-66dcc27b-ed39-46bf-a3f8-2ac42332a8a6 req-51c8a09e-33d3-4e78-9315-aeff0a56e39f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "14e27a99-2936-474b-85fd-21419ba8f525", "name": "network-vif-plugged", "status": "completed", "tag": "b1631449-5e4d-4b87-98a4-9cdf67c296d6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:35.916763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66dcc27b-ed39-46bf-a3f8-2ac42332a8a6 req-51c8a09e-33d3-4e78-9315-aeff0a56e39f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:35.918737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66dcc27b-ed39-46bf-a3f8-2ac42332a8a6 req-51c8a09e-33d3-4e78-9315-aeff0a56e39f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:35.919358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-66dcc27b-ed39-46bf-a3f8-2ac42332a8a6 req-51c8a09e-33d3-4e78-9315-aeff0a56e39f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:35.934081 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-66dcc27b-ed39-46bf-a3f8-2ac42332a8a6 req-51c8a09e-33d3-4e78-9315-aeff0a56e39f service nova] Creating event network-vif-plugged:b1631449-5e4d-4b87-98a4-9cdf67c296d6 for instance 14e27a99-2936-474b-85fd-21419ba8f525 on nch1-master-1 Oct 15 21:04:35.943844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-66dcc27b-ed39-46bf-a3f8-2ac42332a8a6 req-51c8a09e-33d3-4e78-9315-aeff0a56e39f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050156 Oct 15 21:04:35.944631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2301/3415] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:35.950883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:35.951598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:35.958827 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:04:35.959672 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:04:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:04:35.968434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:35.970328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:35.970793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:35.971570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:35.972039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:35.972501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:36.323777 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:04:36.324233 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:04:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:04:36.615220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:36.624944 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-652f3975-884f-4554-8198-bddd63d1d525 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1316 microversion: 2.1 time: 0.678838 Oct 15 21:04:36.638937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2302/3416] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:35 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1316 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:36.650430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:36.651138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:36.657854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:36.658071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:36.658486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:36.659393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:36.660631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:36.661371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:37.543632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:37.556486 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70f9943a-092e-4c5a-a9f6-33e3e05095e6 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.916246 Oct 15 21:04:37.557795 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2303/3417] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:36 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 918 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:37.563297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:37.570033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:37.571120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'vm_state': ['active'], 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:37.580743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:37.581330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:37.581814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:37.583293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:37.583750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:37.584252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:37.757686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:37.861350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3f25bbc-4c75-493e-b296-3ad0dc1dff6c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?status=active" status: 200 len: 1010 microversion: 2.1 time: 0.302116 Oct 15 21:04:37.874869 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2304/3418] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:04:37 2022] GET /compute/v2.1/servers?status=active => generated 1010 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:37.880127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:37.880734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:37.910283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:37.913596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:37.914177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:37.917560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:37.918179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:37.919619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:38.455975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:38.467505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:38.471817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:38.482887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-91f07bf1-8b76-4206-9ddd-cc9878c3cf86 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.606782 Oct 15 21:04:38.484280 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2305/3419] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:37 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:38.502290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:38.503392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:38.509390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:38.522659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:38.523443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:38.524078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:38.524989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:38.525644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:38.526245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:38.811719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:38.825990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5aa7f7c8-63a3-46c3-8a60-21f5808edbe2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1504 microversion: 2.1 time: 0.340398 Oct 15 21:04:38.829689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2306/3420] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:38 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1504 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:38.839647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-da8ed507-008d-41c6-b141-d8eb516627c9 req-c72641cc-4c1e-4305-a4fa-a7cf432929d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01646836-3e7c-424e-afc1-3d7bcade5a63", "tag": "5f813cb5-4d28-4378-b6c5-65ff57d8aee6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:38.867358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-da8ed507-008d-41c6-b141-d8eb516627c9 req-c72641cc-4c1e-4305-a4fa-a7cf432929d3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:38.868062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-da8ed507-008d-41c6-b141-d8eb516627c9 req-c72641cc-4c1e-4305-a4fa-a7cf432929d3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:38.870045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-da8ed507-008d-41c6-b141-d8eb516627c9 req-c72641cc-4c1e-4305-a4fa-a7cf432929d3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:38.896110 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-da8ed507-008d-41c6-b141-d8eb516627c9 req-c72641cc-4c1e-4305-a4fa-a7cf432929d3 service nova] Creating event network-changed:5f813cb5-4d28-4378-b6c5-65ff57d8aee6 for instance 01646836-3e7c-424e-afc1-3d7bcade5a63 on nch2-master-1 Oct 15 21:04:38.907505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-da8ed507-008d-41c6-b141-d8eb516627c9 req-c72641cc-4c1e-4305-a4fa-a7cf432929d3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074061 Oct 15 21:04:38.908925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2307/3421] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:38.916785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:38.923394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:38.924001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.servers [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Removing options 'limit' from query {{(pid=138688) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Oct 15 21:04:38.925197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:38.932754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:38.933400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:38.933957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:38.935074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:38.935623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:38.936711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.032944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:39.088458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-feda55c3-4360-421a-9919-5586ee95531e tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=100000" status: 200 len: 1010 microversion: 2.1 time: 0.177841 Oct 15 21:04:39.096170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2308/3422] 10.222.0.16 () {58 vars in 1150 bytes} [Sat Oct 15 21:04:38 2022] GET /compute/v2.1/servers?limit=100000 => generated 1010 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:39.102626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:39.103571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:39.110005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.110627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.111089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.112178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.112754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.114172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.283213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.292099 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07483f17-49b3-4c8e-b73a-e1fb0fc0a1c0 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.201408 Oct 15 21:04:39.305262 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2309/3423] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:39 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:39.305262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:39.305262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:39.307318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.307812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.308253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.308960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.310434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.310876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.342335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:39.426020 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43a18062-6491-4a13-8123-623fb22dcf30 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "DELETE /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 204 len: 0 microversion: 2.1 time: 0.130590 Oct 15 21:04:39.430224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2310/3424] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:39 2022] DELETE /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:39.433965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:39.438994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.439745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:39.447172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.447779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.448227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.448960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.449447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.449876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.479109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:39.510536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d9638e10-4b93-4d65-a26d-27461dba86ac tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1010 microversion: 2.1 time: 0.081209 Oct 15 21:04:39.511081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2311/3425] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:04:39 2022] GET /compute/v2.1/servers => generated 1010 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:39.516084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:39.516428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:39.523622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.524123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.524569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.525314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.525781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.526256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.698203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.711331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-34098eb0-d20e-476d-9e20-42e12b88b2ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1510 microversion: 2.1 time: 0.199245 Oct 15 21:04:39.711917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2312/3426] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:39 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1510 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:39.717349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:39.718186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:39.730675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.731294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.731875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.734167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:39.734931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:39.735611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:39.743362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:39.963707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.975886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.977856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.979297 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b75e5f3-d99e-4bdf-8970-27e4d4fd9cca tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.265992 Oct 15 21:04:39.980166 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2313/3427] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:39 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:39.984766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:39.990943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:39.991665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'flavor': '451', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:40.005436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.006260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.006858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.007608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.008082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.008521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.096697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:40.165513 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a1886a64-ef66-402b-a195-1f825555c215 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?flavor=451" status: 200 len: 345 microversion: 2.1 time: 0.184474 Oct 15 21:04:40.166127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2314/3428] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:04:39 2022] GET /compute/v2.1/servers?flavor=451 => generated 345 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:40.180210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:40.195837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:40.196658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.servers [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Removing options 'limit' from query {{(pid=138688) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Oct 15 21:04:40.197546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:40.205002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.205769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.206588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.208167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.209468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.209911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.250929 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d13670e4-5b66-4994-94ab-8164c3cc70cc tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=1" status: 200 len: 484 microversion: 2.1 time: 0.075573 Oct 15 21:04:40.251903 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2315/3429] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Oct 15 21:04:40 2022] GET /compute/v2.1/servers?limit=1 => generated 484 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:04:40.265534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:40.273222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:40.274114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': 'tempest-ListServerFiltersTestJSON-instance-1942351304', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:40.283079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.283565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.284068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.285220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.285688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.286120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.393307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:40.470342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-470cc52f-e665-4ad3-b6f8-b40f85274010 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?name=tempest-ListServerFiltersTestJSON-instance-1942351304" status: 200 len: 346 microversion: 2.1 time: 0.208944 Oct 15 21:04:40.479871 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2316/3430] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:04:40 2022] GET /compute/v2.1/servers?name=tempest-ListServerFiltersTestJSON-instance-1942351304 => generated 346 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:40.485308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:40.485584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:40.493419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.493580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.493926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.494628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.495145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.495573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.780162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:40.790467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d234e352-b9da-4e47-b161-14b467373d71 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.309381 Oct 15 21:04:40.807414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2317/3431] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:40 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:40.812542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28572598-395a-4872-9b95-ccb97bb9c76c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:40.819796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28572598-395a-4872-9b95-ccb97bb9c76c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.820490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28572598-395a-4872-9b95-ccb97bb9c76c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.821149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28572598-395a-4872-9b95-ccb97bb9c76c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.866359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-28572598-395a-4872-9b95-ccb97bb9c76c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:04:40.939450 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-28572598-395a-4872-9b95-ccb97bb9c76c tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "POST /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2/action" status: 202 len: 0 microversion: 2.1 time: 0.130127 Oct 15 21:04:40.940777 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2318/3432] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:04:40 2022] POST /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:04:40.947151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:40.952641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:40.961673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.962604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.967042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:40.970944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:40.971475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:40.972011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:41.614418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:41.623182 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5910aac2-869b-4976-852a-e7600bfeb8b7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1510 microversion: 2.1 time: 0.680707 Oct 15 21:04:41.623851 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2319/3433] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:40 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1510 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:41.648609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c392233b-5b72-4c72-9acd-246f15afdef8 req-c4937b5d-f8d3-4e44-858f-c2f14fc7ea45 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:41.659007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c392233b-5b72-4c72-9acd-246f15afdef8 req-c4937b5d-f8d3-4e44-858f-c2f14fc7ea45 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:41.659506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c392233b-5b72-4c72-9acd-246f15afdef8 req-c4937b5d-f8d3-4e44-858f-c2f14fc7ea45 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:41.659987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c392233b-5b72-4c72-9acd-246f15afdef8 req-c4937b5d-f8d3-4e44-858f-c2f14fc7ea45 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:41.674348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c392233b-5b72-4c72-9acd-246f15afdef8 req-c4937b5d-f8d3-4e44-858f-c2f14fc7ea45 service nova] Creating event network-changed:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:04:41.682681 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c392233b-5b72-4c72-9acd-246f15afdef8 req-c4937b5d-f8d3-4e44-858f-c2f14fc7ea45 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057006 Oct 15 21:04:41.683936 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2320/3434] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:41.690628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:41.692387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:41.699791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:41.700525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:41.701103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:41.702066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:41.702648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:41.703292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:42.421305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:42.436263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c07dbd24-5f6b-49f4-978c-9d3d68dbbf0d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1522 microversion: 2.1 time: 0.750352 Oct 15 21:04:42.439041 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2321/3435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:41 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1522 bytes in 754 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:42.447088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:42.448573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:42.459371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:42.460268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:42.461092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:42.462336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:42.464010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:42.466841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:42.928380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:42.944706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:42.955807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:42.957688 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f46c1b6-4bb8-45a9-9bfe-aaf739f594b5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.516254 Oct 15 21:04:42.971050 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2322/3436] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:42 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:42.972926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:42.979595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:42.995809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:42.998547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:42.999414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:42.999996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.001280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.003914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.332437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:43.343163 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-12631003-c593-4658-b601-3c1aff8aa48c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.383217 Oct 15 21:04:43.344327 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2323/3437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:42 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:43.350316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:43.351158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:43.362380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.363692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.365751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.369328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.371280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.371609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.413863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.414359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.415310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.415310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lazy-loading 'fault' on Instance uuid 14e27a99-2936-474b-85fd-21419ba8f525 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:04:43.492015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:43.505624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fe427cb-43c4-47f3-8ff6-fdd783b3b90c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 200 len: 1306 microversion: 2.1 time: 0.160225 Oct 15 21:04:43.506214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2324/3438] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:43 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 1306 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:43.512410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:43.513288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:43.520397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.521097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.521734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.522435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.523060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.523689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.698268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:43.708755 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a2d73f1-89c8-4a91-966f-7c2e4283f68a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1522 microversion: 2.1 time: 0.201293 Oct 15 21:04:43.709909 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2325/3439] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:43 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1522 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:43.988450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:43.989364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:43.996880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:43.997620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:43.998229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:43.999353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.000126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.000949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.173990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:44.207161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:44.208992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:44.211440 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19be2fc1-e372-4d26-80af-edbd03a0fb88 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.229001 Oct 15 21:04:44.212539 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2326/3440] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:43 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:44.217457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1a146e04-ef98-403f-bcef-b890b2a1d91c req-2e9dc09c-2893-4746-bc75-22bf290ce85a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "14e27a99-2936-474b-85fd-21419ba8f525", "name": "network-vif-deleted", "tag": "b1631449-5e4d-4b87-98a4-9cdf67c296d6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:44.230264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1a146e04-ef98-403f-bcef-b890b2a1d91c req-2e9dc09c-2893-4746-bc75-22bf290ce85a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.230926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1a146e04-ef98-403f-bcef-b890b2a1d91c req-2e9dc09c-2893-4746-bc75-22bf290ce85a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.231694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1a146e04-ef98-403f-bcef-b890b2a1d91c req-2e9dc09c-2893-4746-bc75-22bf290ce85a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.245222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1a146e04-ef98-403f-bcef-b890b2a1d91c req-2e9dc09c-2893-4746-bc75-22bf290ce85a service nova] Creating event network-vif-deleted:b1631449-5e4d-4b87-98a4-9cdf67c296d6 for instance 14e27a99-2936-474b-85fd-21419ba8f525 on nch1-master-1 Oct 15 21:04:44.255018 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1a146e04-ef98-403f-bcef-b890b2a1d91c req-2e9dc09c-2893-4746-bc75-22bf290ce85a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040606 Oct 15 21:04:44.256351 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2327/3441] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:44.303326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ac9e28c6-947a-49a3-a284-ce1d03cffa27 req-9f49c4e1-3f31-44ca-818d-73a24fc6797c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01646836-3e7c-424e-afc1-3d7bcade5a63", "name": "network-vif-plugged", "status": "completed", "tag": "5f813cb5-4d28-4378-b6c5-65ff57d8aee6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:44.325009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ac9e28c6-947a-49a3-a284-ce1d03cffa27 req-9f49c4e1-3f31-44ca-818d-73a24fc6797c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.325543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ac9e28c6-947a-49a3-a284-ce1d03cffa27 req-9f49c4e1-3f31-44ca-818d-73a24fc6797c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.326319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ac9e28c6-947a-49a3-a284-ce1d03cffa27 req-9f49c4e1-3f31-44ca-818d-73a24fc6797c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.341427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ac9e28c6-947a-49a3-a284-ce1d03cffa27 req-9f49c4e1-3f31-44ca-818d-73a24fc6797c service nova] Creating event network-vif-plugged:5f813cb5-4d28-4378-b6c5-65ff57d8aee6 for instance 01646836-3e7c-424e-afc1-3d7bcade5a63 on nch2-master-1 Oct 15 21:04:44.351340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ac9e28c6-947a-49a3-a284-ce1d03cffa27 req-9f49c4e1-3f31-44ca-818d-73a24fc6797c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054653 Oct 15 21:04:44.369288 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2328/3442] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:44.383103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.384652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:44.395895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.396699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.397923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.399420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.400084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.400740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.578646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:44.594022 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.221838 Oct 15 21:04:44.597300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2329/3443] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:44.600531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.601529 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c78832d8-9236-4285-8b62-ae6b27e945d8 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001282 Oct 15 21:04:44.602562 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2330/3444] 10.222.0.16 () {56 vars in 886 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/ => generated 377 bytes in 4 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Oct 15 21:04:44.607161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.608064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:44.616939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.617700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.618337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.619278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.619928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.620640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.640697 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 HTTP/1.1 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '14e27a99-2936-474b-85fd-21419ba8f525'} Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:44.642771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:04:44.649173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:44.650954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:44.652181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 404 len: 111 microversion: 2.1 time: 0.048567 Oct 15 21:04:44.653306 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2331/3445] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:44.654980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.656296 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3405dc8c-c821-43f1-8d9b-62fc67beacfc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2/" status: 200 len: 379 microversion: 2.1 time: 0.001894 Oct 15 21:04:44.657209 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2332/3446] 10.222.0.16 () {54 vars in 1035 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/v2/ => generated 379 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 221 bytes (1 switches on core 0) Oct 15 21:04:44.662368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.663603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:44.670345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.671161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.671798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.672708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.673385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.674007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.678129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:44.830819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:44.859016 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.200573 Oct 15 21:04:44.859613 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2333/3447] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:44.860901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.862102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-869b4d82-a0fd-4227-b8b5-df5e25e3da3d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001457 Oct 15 21:04:44.862496 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2334/3448] 10.222.0.16 () {54 vars in 1041 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/v2.1/ => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:44.873528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:44.874151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:44.881524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.882490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.883433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:44.884366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:44.884993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:44.885614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:45.189885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=30 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:45.486180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:45.495237 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1522 microversion: 2.1 time: 0.631546 Oct 15 21:04:45.495951 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2335/3449] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:44 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1522 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:45.497399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:45.498518 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ac923a5-cb66-4715-97f5-88728c11ed74 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001567 Oct 15 21:04:45.499058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2336/3450] 10.222.0.16 () {56 vars in 886 bytes} [Sat Oct 15 21:04:45 2022] GET /compute/ => generated 377 bytes in 2 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Oct 15 21:04:45.505224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eba45c8-a8f5-46dc-941a-d10c75f8be2a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": 12345, "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eba45c8-a8f5-46dc-941a-d10c75f8be2a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 163 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": 12345, "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 12345, 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 12345 is not of type 'string' Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['name']: Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 12345 Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:04:45.516949 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eba45c8-a8f5-46dc-941a-d10c75f8be2a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0eba45c8-a8f5-46dc-941a-d10c75f8be2a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 127 microversion: 2.1 time: 0.017987 Oct 15 21:04:45.521476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2337/3451] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:45 2022] POST /compute/v2.1/servers => generated 127 bytes in 19 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:04:45.526521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:45.527143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:45.536164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:45.536676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:45.537152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:45.538089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:45.538727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:45.539336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:45.797182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:45.810125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:45.812091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:45.814577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19ebe98e-8202-4d5a-a16b-d717f4573a42 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.293898 Oct 15 21:04:45.816509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2338/3452] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:45 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:45.822000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:45.822963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:45.832534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:45.833440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:45.834082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:45.834994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:45.835981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:45.836594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.078265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:46.087267 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32072f04-c4c0-4850-b26c-62f341379581 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.268993 Oct 15 21:04:46.088227 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2339/3453] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:45 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:46.108970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:46.109651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:46.120529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.121719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.122468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.125715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.126515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.127525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.281681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:46.294289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0527655-7662-45bf-bc57-66e445ef7820 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1324 microversion: 2.1 time: 0.204225 Oct 15 21:04:46.295462 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2340/3454] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:46 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1324 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:46.307236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:46.309738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:46.317300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.317983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.318572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.319878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.320697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.321305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.560849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:46.574021 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66aa9d4f-e7a4-4d80-a90b-339d844afc22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.275966 Oct 15 21:04:46.585068 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2341/3455] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:46 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:46.585068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:46.593683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:46.604461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.604964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.605448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.606325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.606623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.607059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.966601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:46.976366 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a701312b-bbe1-4d11-8d49-dbdf448a3231 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1522 microversion: 2.1 time: 0.396799 Oct 15 21:04:46.976986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2342/3456] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:46 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1522 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:46.982317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:46.983805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:46.992541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.993996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.994799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:46.995758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:46.996577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:46.997006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:47.211753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:47.221454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:47.223733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:47.225053 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4459095-f13a-427c-98b8-93bf413dd8f7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.246815 Oct 15 21:04:47.226209 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2343/3457] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:46 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:47.319456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:47.320487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:47.327765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:47.337083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:47.339234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:47.340475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:47.341230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:47.342724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:47.543137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:47.557632 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98412d24-da8d-4c56-90af-e3f399247c4b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1520 microversion: 2.1 time: 0.242620 Oct 15 21:04:47.559247 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2344/3458] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:47 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1520 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:47.589688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:47.590507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:47.600895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:47.601435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:47.601880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:47.604631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:47.605176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:47.605620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:47.655261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:04:47.809903 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50c22093-02e0-40de-983a-1541de487ad7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 204 len: 0 microversion: 2.1 time: 0.229414 Oct 15 21:04:47.813126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2345/3459] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:47 2022] DELETE /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:04:47.822341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:47.824405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:47.831162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:47.836436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:47.836436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:47.836436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:47.836436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:47.836436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:48.056173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:48.064476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c442b77-716f-47f6-b0d8-c2341af71bd3 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1526 microversion: 2.1 time: 0.247148 Oct 15 21:04:48.078231 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2346/3460] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:47 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1526 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:04:48.090156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:48.090941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:48.097945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:48.098553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:48.099014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:48.100025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:48.100513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:48.101085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:48.281303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:48.300071 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ad2ff02-f859-4da8-b665-aa36e05a2497 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1522 microversion: 2.1 time: 0.220639 Oct 15 21:04:48.301513 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2347/3461] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:48 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1522 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:48.306286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:48.307463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:48.315860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:48.317148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:48.318016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:48.318911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:48.319548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:48.320156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:48.507323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:48.518813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:48.521246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:48.522738 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35be3be4-9688-4b82-97af-7081a63621bb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.220282 Oct 15 21:04:48.523954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2348/3462] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:48 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:48.611580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13e5a863-f509-4780-bbab-c41eaa430b52 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-2035798583", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "7dd5d8e5-812c-4c10-a979-341594357aa3", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:48.689650 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-13e5a863-f509-4780-bbab-c41eaa430b52 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Block Device 7dd5d8e5-812c-4c10-a979-341594357aa3 is not bootable. Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13e5a863-f509-4780-bbab-c41eaa430b52 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 358 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-2035798583", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "7dd5d8e5-812c-4c10-a979-341594357aa3", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: exception: Block Device 7dd5d8e5-812c-4c10-a979-341594357aa3 is not bootable. Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-2035798583', 'imageRef': '', 'flavorRef': '452', 'block_device_mapping_v2': [{'boot_index': 0, 'source_type': 'volume', 'destination_type': 'volume', 'delete_on_termination': False, 'volume_id': '7dd5d8e5-812c-4c10-a979-341594357aa3'}], 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1672, in _create_instance Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi boot_meta = block_device.get_bdm_image_metadata( Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/block_device.py", line 664, in get_bdm_image_metadata Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidBDMVolumeNotBootable(id=volume_id) Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidBDMVolumeNotBootable: Block Device 7dd5d8e5-812c-4c10-a979-341594357aa3 is not bootable. Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:48.697326 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Block Device 7dd5d8e5-812c-4c10-a979-341594357aa3 is not bootable. Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Block Device 7dd5d8e5-812c-4c10-a979-341594357aa3 is not bootable. Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13e5a863-f509-4780-bbab-c41eaa430b52 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Block Device 7dd5d8e5-812c-4c10-a979-341594357aa3 is not bootable. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13e5a863-f509-4780-bbab-c41eaa430b52 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 110 microversion: 2.1 time: 0.091243 Oct 15 21:04:48.702043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2349/3463] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:48 2022] POST /compute/v2.1/servers => generated 110 bytes in 92 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:04:48.711075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:48.711689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:48.730558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:48.731066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:48.731512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:48.732231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:48.732696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:48.733157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:49.025007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:49.035497 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f16ae5e1-0bcb-4d97-85c3-f10b166840c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.328107 Oct 15 21:04:49.035779 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2350/3464] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:48 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:49.062255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:49.062948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:49.070619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:49.071746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:49.072290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:49.073394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:49.073655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:49.074373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:49.289949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:49.301799 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-85980377-eff1-443c-bc65-157b23c2988f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.243043 Oct 15 21:04:49.307322 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2351/3465] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:49 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:04:49.313188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:49.314066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:49.322487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:49.323312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:49.324146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:49.325146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:49.326175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:49.327953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:49.373184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:49.736061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8af9544fe68497c8557248575f184ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '994259bbbe124b4e9ff0542ca402b9f7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:49.745277 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c531d6b3-cd59-40e4-a15b-9432eb1b9f79 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 200 len: 1526 microversion: 2.1 time: 0.436165 Oct 15 21:04:49.745874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2352/3466] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:49 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 1526 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:50.007584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ddc2924-f84d-4863-9aa4-1c5128a9d4b3 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74eda71f0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-515427653", "description": "tempest-description-61222256"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:50.320174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=26 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:50.527748 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ddc2924-f84d-4863-9aa4-1c5128a9d4b3 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 245 microversion: 2.1 time: 0.769459 Oct 15 21:04:50.532388 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2353/3467] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:04:49 2022] POST /compute/v2.1/os-security-groups => generated 245 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:50.538522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:50.539446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:50.543907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:04:50.554120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:50.555214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:50.556376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:50.565670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:50.567144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:50.567701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:50.850363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:50.877895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cccd79ff-0205-4854-a190-16eebad27c01 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1522 microversion: 2.1 time: 0.344080 Oct 15 21:04:50.879253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2354/3468] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:50 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1522 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:50.890141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:50.891360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:50.904945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:50.905514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:50.906983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:50.908450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:50.908937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:50.909454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:51.114461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:51.134098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:51.141418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:51.147415 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c48332c-6d50-4c7d-88d3-bd65618ff469 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.263788 Oct 15 21:04:51.162653 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2355/3469] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:50 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:51.175859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8e34a1d3-a68c-4083-a96b-b47e8b872b14 req-db9ff64b-0d46-4365-b4fd-f83f993cfb36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01646836-3e7c-424e-afc1-3d7bcade5a63", "name": "network-vif-deleted", "tag": "5f813cb5-4d28-4378-b6c5-65ff57d8aee6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:51.228910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8e34a1d3-a68c-4083-a96b-b47e8b872b14 req-db9ff64b-0d46-4365-b4fd-f83f993cfb36 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:51.229621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8e34a1d3-a68c-4083-a96b-b47e8b872b14 req-db9ff64b-0d46-4365-b4fd-f83f993cfb36 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:51.231682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8e34a1d3-a68c-4083-a96b-b47e8b872b14 req-db9ff64b-0d46-4365-b4fd-f83f993cfb36 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:51.285236 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8e34a1d3-a68c-4083-a96b-b47e8b872b14 req-db9ff64b-0d46-4365-b4fd-f83f993cfb36 service nova] Creating event network-vif-deleted:5f813cb5-4d28-4378-b6c5-65ff57d8aee6 for instance 01646836-3e7c-424e-afc1-3d7bcade5a63 on nch2-master-1 Oct 15 21:04:51.302668 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8e34a1d3-a68c-4083-a96b-b47e8b872b14 req-db9ff64b-0d46-4365-b4fd-f83f993cfb36 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.135397 Oct 15 21:04:51.303257 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2356/3470] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:51.321143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-711f054e-4cbc-4c01-abd5-9e8da588d37d tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74eda7b80>, body: {"security_group_rule": {"parent_group_id": "4f923023-c258-449e-b560-40ca0d79954f", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:51.717318 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-711f054e-4cbc-4c01-abd5-9e8da588d37d tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.407892 Oct 15 21:04:51.718714 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2357/3471] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:04:51 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:51.742743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2360cf85-83d1-4a49-a764-e9dc47b1accd req-a25aa21a-9d66-4354-81c7-0e86741dba30 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "name": "network-vif-plugged", "status": "completed", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:51.779139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2360cf85-83d1-4a49-a764-e9dc47b1accd req-a25aa21a-9d66-4354-81c7-0e86741dba30 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:51.780230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2360cf85-83d1-4a49-a764-e9dc47b1accd req-a25aa21a-9d66-4354-81c7-0e86741dba30 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:51.780980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2360cf85-83d1-4a49-a764-e9dc47b1accd req-a25aa21a-9d66-4354-81c7-0e86741dba30 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:51.818854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2360cf85-83d1-4a49-a764-e9dc47b1accd req-a25aa21a-9d66-4354-81c7-0e86741dba30 service nova] Creating event network-vif-plugged:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:04:51.828701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2360cf85-83d1-4a49-a764-e9dc47b1accd req-a25aa21a-9d66-4354-81c7-0e86741dba30 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107001 Oct 15 21:04:51.830372 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2358/3472] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:04:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:51.851361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:51.855204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 01646836-3e7c-424e-afc1-3d7bcade5a63] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:51.880549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:51.885690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:51.886603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:51.887660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:51.888315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:51.888970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:51.913087 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 01646836-3e7c-424e-afc1-3d7bcade5a63 could not be found. Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 HTTP/1.1 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 01646836-3e7c-424e-afc1-3d7bcade5a63 could not be found. Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '01646836-3e7c-424e-afc1-3d7bcade5a63'} Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01646836-3e7c-424e-afc1-3d7bcade5a63 could not be found. Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01646836-3e7c-424e-afc1-3d7bcade5a63 could not be found. Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.915178 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01646836-3e7c-424e-afc1-3d7bcade5a63 could not be found. Oct 15 21:04:51.920868 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.923305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 01646836-3e7c-424e-afc1-3d7bcade5a63 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:51.924460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bcafba0-e98f-476c-9636-201e5e23173b tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63" status: 404 len: 111 microversion: 2.1 time: 0.090837 Oct 15 21:04:51.925606 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2359/3473] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:51 2022] GET /compute/v2.1/servers/01646836-3e7c-424e-afc1-3d7bcade5a63 => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:51.937757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b4bea47-7659-4e9d-ac31-460136f44d73 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1307744174", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "destination_type": "local"}, {"source_type": "volume", "uuid": "bb8f8bd6-a6d7-4af9-b412-9283178245ae", "destination_type": "invalid"}], "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b4bea47-7659-4e9d-ac31-460136f44d73 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 495 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1307744174", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "destination_type": "local"}, {"source_type": "volume", "uuid": "bb8f8bd6-a6d7-4af9-b412-9283178245ae", "destination_type": "invalid"}], "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1307744174', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'block_device_mapping_v2': [{'source_type': 'image', 'delete_on_termination': True, 'boot_index': 0, 'uuid': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'destination_type': 'local'}, {'source_type': 'volume', 'uuid': 'bb8f8bd6-a6d7-4af9-b412-9283178245ae', 'destination_type': 'invalid'}], 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['local', 'volume'] Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['server']['properties']['block_device_mapping_v2']['items']['properties']['destination_type']: Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'enum': ['local', 'volume'], 'type': 'string'} Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']['block_device_mapping_v2'][1]: Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'invalid' Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:51.947343 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Oct 15 21:04:51.952717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:51.959645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b4bea47-7659-4e9d-ac31-460136f44d73 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:51.960732 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b4bea47-7659-4e9d-ac31-460136f44d73 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 155 microversion: 2.1 time: 0.030517 Oct 15 21:04:51.968015 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2360/3474] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:51 2022] POST /compute/v2.1/servers => generated 155 bytes in 38 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:04:51.981562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f02a512-2e31-4713-9c59-bd6f0bdfbc52 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74e836670>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1840798857", "description": "tempest-description-1234969033"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:52.267770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f02a512-2e31-4713-9c59-bd6f0bdfbc52 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.297562 Oct 15 21:04:52.268907 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2361/3475] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:04:51 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:52.279319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:52.284480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:52.297570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:52.298272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:52.300779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:52.303247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:52.304232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:52.305174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:52.506364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:52.518266 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-007ada65-c994-4709-9a3e-1292397e515f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1499 microversion: 2.1 time: 0.246317 Oct 15 21:04:52.520610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2362/3476] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:52 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1499 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:52.527648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0349944-19f7-4c85-bf70-d3b6752ea4cd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '.version_select at 0x7fe74e8368b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:52.563195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e0349944-19f7-4c85-bf70-d3b6752ea4cd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Image not found. Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0349944-19f7-4c85-bf70-d3b6752ea4cd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd HTTP/1.1 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd'} Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd to delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd could not be found. Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:52.566583 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:04:52.571751 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:52.580533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0349944-19f7-4c85-bf70-d3b6752ea4cd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:52.582093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0349944-19f7-4c85-bf70-d3b6752ea4cd tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd" status: 404 len: 62 microversion: 2.1 time: 0.058290 Oct 15 21:04:52.586217 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2363/3477] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:52 2022] DELETE /compute/v2.1/images/0bff7746-32e7-4f99-8e8a-5fcbc10ad8dd => generated 62 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:04:52.592480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:52.593475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:52.607655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:52.608570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:52.609323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:52.611093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:52.611995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:52.613297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:53.246325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:53.254958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c0a4b4d-edc8-4953-9d52-93bc93712325 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.667298 Oct 15 21:04:53.269698 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2364/3478] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:52 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:53.281569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:53.282528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:53.289744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:53.290507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:53.291145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:53.292050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:53.292733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:53.293395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:53.522871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:53.535009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:53.540204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:53.549552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2581e845-bde1-495a-8329-08cd34be9c61 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1406 microversion: 2.20 time: 0.271645 Oct 15 21:04:53.551810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2365/3479] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:53 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1406 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:53.557771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10a6e18b-e677-42a6-8109-43d14857f305 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74e836940>, body: {"security_group_rule": {"parent_group_id": "923d82a5-4c1f-4a65-8a70-05de3d61a060", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "cidr": "10.2.3.124/24"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:54.006103 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10a6e18b-e677-42a6-8109-43d14857f305 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 236 microversion: 2.1 time: 0.452469 Oct 15 21:04:54.010219 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2366/3480] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:04:53 2022] POST /compute/v2.1/os-security-group-rules => generated 236 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:54.023538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:54.037623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:54.039433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'vm_state': ['stopped'], 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:04:54.054170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:54.056305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:54.057596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:54.057981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:54.058265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:54.058694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:54.108962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:04:54.187353 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09a1b17d-f397-4876-8b80-8b25094167a6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?status=shutoff" status: 200 len: 346 microversion: 2.1 time: 0.174027 Oct 15 21:04:54.188540 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2367/3481] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Oct 15 21:04:54 2022] GET /compute/v2.1/servers?status=shutoff => generated 346 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:54.196433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:54.197431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:54.207434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:54.208495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:54.209189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:54.210214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:54.210916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:54.211539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:54.243649 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c HTTP/1.1 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '227d10b5-22c4-44a9-b65e-de1ed6d8680c'} Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:54.245052 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-db4e38ce-b9da-4e21-ac4b-b7e931fbb9fc tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 404 len: 111 microversion: 2.1 time: 0.058737 Oct 15 21:04:54.251574 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2368/3482] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:04:54 2022] DELETE /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:54.257493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:54.258345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:54.265538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:54.266050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:54.271633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:54.272811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:54.274027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:54.274697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:54.521474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:54.531848 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d060ddb-e6ef-48c2-9b43-4b8a612b9ef2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.280534 Oct 15 21:04:54.533432 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2369/3483] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:54 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:54.577195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6b3b9ae-c3c7-49e4-a173-d17f3cf305ff tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74ef9e5e0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-930323537", "description": "tempest-description-422941434"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:55.075541 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6b3b9ae-c3c7-49e4-a173-d17f3cf305ff tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 246 microversion: 2.1 time: 0.538361 Oct 15 21:04:55.076146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2370/3484] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:04:54 2022] POST /compute/v2.1/os-security-groups => generated 246 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:55.081984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-88c08c86-8366-4ef9-8de7-33a21ed4c55f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:55.095320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88c08c86-8366-4ef9-8de7-33a21ed4c55f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:55.095824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88c08c86-8366-4ef9-8de7-33a21ed4c55f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:55.096270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88c08c86-8366-4ef9-8de7-33a21ed4c55f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:55.149130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-88c08c86-8366-4ef9-8de7-33a21ed4c55f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Going to try to start instance {{(pid=138688) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 21:04:55.238475 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-88c08c86-8366-4ef9-8de7-33a21ed4c55f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "POST /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2/action" status: 202 len: 0 microversion: 2.1 time: 0.159780 Oct 15 21:04:55.239230 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2371/3485] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:04:55 2022] POST /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2/action => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:04:55.249930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:55.251582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 227d10b5-22c4-44a9-b65e-de1ed6d8680c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:55.263112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:55.263897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:55.264678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:55.265855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:55.266581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:55.268116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:55.296229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c HTTP/1.1 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '227d10b5-22c4-44a9-b65e-de1ed6d8680c'} Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.298564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.303720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 227d10b5-22c4-44a9-b65e-de1ed6d8680c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:55.305628 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7cfcd483-4e22-4737-8635-3e7c15e4d029 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c" status: 404 len: 111 microversion: 2.1 time: 0.063071 Oct 15 21:04:55.306738 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2372/3486] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:55 2022] GET /compute/v2.1/servers/227d10b5-22c4-44a9-b65e-de1ed6d8680c => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:04:55.332763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:55.335052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:55.354571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:55.355373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:55.356039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:55.357068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:55.357754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:55.358372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:55.738177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:55.764889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:55.776758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:55.780027 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae97be94-e7b5-43b9-807d-d26bc59e9a39 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1597 microversion: 2.20 time: 0.470023 Oct 15 21:04:55.783192 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2373/3487] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:04:55 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1597 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:04:55.827355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8d68300-1714-4148-afb9-374da76c0cdb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1891418843", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"a": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8d68300-1714-4148-afb9-374da76c0cdb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 492 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1891418843", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"a": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1891418843', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'a': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}, 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['metadata']['patternProperties']['^[a-zA-Z0-9-_:. ]{1,255}$']: Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'maxLength': 255, 'type': 'string'} Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.854842 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']['metadata']: Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8d68300-1714-4148-afb9-374da76c0cdb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:04:55.859536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e8d68300-1714-4148-afb9-374da76c0cdb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 624 microversion: 2.1 time: 0.058575 Oct 15 21:04:55.866655 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2374/3488] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:04:55 2022] POST /compute/v2.1/servers => generated 624 bytes in 68 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:04:55.879076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b456858-4412-44b6-a52d-81a03666a112 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74ef9ed30>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-297304311", "description": "tempest-description-451408472"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:56.791724 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b456858-4412-44b6-a52d-81a03666a112 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 246 microversion: 2.1 time: 0.923599 Oct 15 21:04:56.793605 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2375/3489] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:04:55 2022] POST /compute/v2.1/os-security-groups => generated 246 bytes in 926 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:56.799440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:56.802897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:56.827325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:56.829118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:56.831818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:56.833291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:56.834018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:56.834690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:57.150781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:57.162189 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81d60d5b-4ea6-45f3-ae1f-81acc8f2cb00 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1508 microversion: 2.1 time: 0.366899 Oct 15 21:04:57.163066 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2376/3490] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:56 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1508 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:57.168615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-244bccf8-2209-4fb6-9344-5a9862e0711c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '.version_select at 0x7fe74e8365e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:57.208332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-244bccf8-2209-4fb6-9344-5a9862e0711c tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1070717387" status: 202 len: 0 microversion: 2.1 time: 0.043386 Oct 15 21:04:57.214246 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2377/3491] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:04:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1070717387 => generated 0 bytes in 50 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:04:57.220722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:57.221470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:57.230014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:57.230736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:57.231553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:57.233636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:57.234299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:57.235468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:57.434676 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c69c54f2-5992-49d3-aa9c-2f87280f4bba tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.218773 Oct 15 21:04:57.435476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2378/3492] 10.222.0.16 () {60 vars in 1316 bytes} [Sat Oct 15 21:04:57 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-interface => generated 284 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:04:57.441924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:57.442798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:57.494475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:57.495334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:57.495466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:57.496180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:57.496677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:57.497261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:57.920868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:57.933349 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d18bec7-a81a-472a-a286-97ebb5cd66b2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.496541 Oct 15 21:04:57.938283 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2379/3493] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:57 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:57.951817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e010bb9f-0ca7-4a87-98ca-f8600316427e tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74ed50430>, body: {"security_group_rule": {"parent_group_id": "43cdcc72-02ab-497a-ad0f-54d292dac97e", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "group_id": "b43f4877-9abf-4809-8cad-71fabfe79114"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:59.251906 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e010bb9f-0ca7-4a87-98ca-f8600316427e tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 330 microversion: 2.1 time: 1.311599 Oct 15 21:04:59.262201 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2380/3494] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:04:57 2022] POST /compute/v2.1/os-security-group-rules => generated 330 bytes in 1313 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:59.330449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-145a19ef-d5f2-455b-b063-0c94d2056a05 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: .version_select at 0x7fe74fa60af0>, body: {"keypair": {"name": "tempest-keypair-808352224", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOjlAtH4h+2Y/D1Oe0RlrosJ1bewKPxQtQayBoN1H8yt4qWmmSzdEoLvfQ9R5G6E8rFKh4uBd3L6EdaBKqEFRKiomypiAkj6Lcsj+nj11jJTl8cqKGunpxTV01/teB1t9A=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:04:59.353083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-145a19ef-d5f2-455b-b063-0c94d2056a05 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:04:59.359789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-145a19ef-d5f2-455b-b063-0c94d2056a05 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for user 2f4e4fc2f97f4c1a832c6348a97fd155 and project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:04:59.394205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-145a19ef-d5f2-455b-b063-0c94d2056a05 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.139733 Oct 15 21:04:59.395661 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2381/3495] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:04:59 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:04:59.406525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:59.407599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:59.417539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:59.418037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:59.418720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:59.419683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:59.420215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:59.420848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:59.797521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:04:59.821584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21986da5-08a9-4671-91f8-cbe7adcbad06 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.422414 Oct 15 21:04:59.822662 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2382/3496] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:59 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:04:59.834721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:04:59.835447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:04:59.848158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:59.848745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:59.855898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:59.857628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:04:59.858168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:04:59.858850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:04:59.923627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:00.342424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:00.356577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb7d1076-6b44-4495-87de-294929462e07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1512 microversion: 2.1 time: 0.532343 Oct 15 21:05:00.366397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2383/3497] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:04:59 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1512 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:00.382138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f0d25ed-c38b-4f35-a3da-f8d49bfa3303 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '.version_select at 0x7fe74f23e550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:00.414569 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0f0d25ed-c38b-4f35-a3da-f8d49bfa3303 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Image not found. Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f0d25ed-c38b-4f35-a3da-f8d49bfa3303 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/images/ccabcec1-70bb-47fc-952f-ece500d99a4c HTTP/1.1 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: exception: Image not found. Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ccabcec1-70bb-47fc-952f-ece500d99a4c'} Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image ccabcec1-70bb-47fc-952f-ece500d99a4c to delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image ccabcec1-70bb-47fc-952f-ece500d99a4c could not be found. Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:00.416359 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f0d25ed-c38b-4f35-a3da-f8d49bfa3303 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Image not found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:00.422176 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f0d25ed-c38b-4f35-a3da-f8d49bfa3303 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/images/ccabcec1-70bb-47fc-952f-ece500d99a4c" status: 404 len: 62 microversion: 2.1 time: 0.051002 Oct 15 21:05:00.429610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2384/3498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:00 2022] DELETE /compute/v2.1/images/ccabcec1-70bb-47fc-952f-ece500d99a4c => generated 62 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:05:00.429610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65c56a28-0825-4fc4-a7dc-5cc137cb210a tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74f9d8550>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1882171766", "description": "tempest-description-187483762"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:00.829654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-65c56a28-0825-4fc4-a7dc-5cc137cb210a tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 0.405767 Oct 15 21:05:00.832048 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2385/3499] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:05:00 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:00.837600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db21979e-de8c-4918-9f24-afcaf5b72468 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db21979e-de8c-4918-9f24-afcaf5b72468 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 416 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['name']: Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.847967 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:05:00.852331 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:00.871628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db21979e-de8c-4918-9f24-afcaf5b72468 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:00.878736 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-db21979e-de8c-4918-9f24-afcaf5b72468 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 619 microversion: 2.1 time: 0.044824 Oct 15 21:05:00.883146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2386/3500] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:00 2022] POST /compute/v2.1/servers => generated 619 bytes in 50 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:05:00.892048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2450da22-6690-49f0-b901-e2dfc5324582 req-03aa920b-7533-47d2-8af6-47dd6a51009b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:00.927656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2450da22-6690-49f0-b901-e2dfc5324582 req-03aa920b-7533-47d2-8af6-47dd6a51009b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:00.928608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2450da22-6690-49f0-b901-e2dfc5324582 req-03aa920b-7533-47d2-8af6-47dd6a51009b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:00.929151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2450da22-6690-49f0-b901-e2dfc5324582 req-03aa920b-7533-47d2-8af6-47dd6a51009b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:00.953949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2450da22-6690-49f0-b901-e2dfc5324582 req-03aa920b-7533-47d2-8af6-47dd6a51009b service nova] Creating event network-changed:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:05:00.960082 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2450da22-6690-49f0-b901-e2dfc5324582 req-03aa920b-7533-47d2-8af6-47dd6a51009b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075583 Oct 15 21:05:00.961126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2387/3501] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:00.970108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:00.976124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:00.976582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.servers [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Removing options 'limit' from query {{(pid=138688) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Oct 15 21:05:00.977398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:00.991091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:00.991631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:00.992250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:00.995000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:01.000750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:01.001785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:01.004491 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79524bf6-3098-4cad-b724-6d1933363a3f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=0" status: 200 len: 15 microversion: 2.1 time: 0.042204 Oct 15 21:05:01.017657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2388/3502] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Oct 15 21:05:00 2022] GET /compute/v2.1/servers?limit=0 => generated 15 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:05:01.024110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:01.025054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:01.033018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:01.035664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:01.036322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:01.037510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:01.038149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:01.038784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:01.072212 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc HTTP/1.1 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc'} Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:01.077416 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d25f7671-de87-497f-8445-698ae443a8ce tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 404 len: 111 microversion: 2.1 time: 0.060205 Oct 15 21:05:01.083571 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2389/3503] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:05:01 2022] DELETE /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:01.088210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69b65863-cb94-42d7-bf4e-0a87313aa39f tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74efa75e0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1083479122", "description": "tempest-description-2028606073"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:01.928330 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69b65863-cb94-42d7-bf4e-0a87313aa39f tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.847745 Oct 15 21:05:01.930215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2390/3504] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:05:01 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:01.936554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:01.937845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:01.947596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:01.948230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:01.948530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:01.949628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:01.951018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:01.951395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:02.316186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:02.327522 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c50f1f2-fbd3-405e-b774-8c4e3be8fec4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.395956 Oct 15 21:05:02.328340 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2391/3505] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:01 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:02.335493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:02.336465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:02.352344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:02.353443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:02.354151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:02.355128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:02.355819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:02.356478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:02.638121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:02.646680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7afbb1d3-91ff-4d7b-9492-8a610a6f3e30 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1512 microversion: 2.1 time: 0.316941 Oct 15 21:05:02.647474 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2392/3506] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:02 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1512 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:02.652053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:02.652658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] [instance: 0b72dcc6-e2fb-40ad-860d-c65f66845edc] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:02.658449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:02.658971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:02.659986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:02.660683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:02.661210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:02.661614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:02.676142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] HTTP exception thrown: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Request method failure captured: Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc HTTP/1.1 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx_xvjkG-Zuzq0ZrcHbzMU8xfalDFjTpDffsHjAfjmbaylzfZsOm42h7uceTwEV7RJwtLhTvJEsAdlJQfHp1TFuQd07bxP340IGDX40rmcXZ5mxbKM8EnUnjdDSXd66U2tdaHe9aQ4_u0gecuBWiHV5Li5HvjTZtbzOIjM7U1Tg6uqc Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/994259bbbe124b4e9ff0542ca402b9f7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 994259bbbe124b4e9ff0542ca402b9f7 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-447170534 Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a8af9544fe68497c8557248575f184ed Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-447170534-project Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0b72dcc6-e2fb-40ad-860d-c65f66845edc'} Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:02.677772 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] Returning 404 to user: Instance 0b72dcc6-e2fb-40ad-860d-c65f66845edc could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f795311-a4b0-4aa7-8af9-20d70693e4c7 tempest-ImagesOneServerNegativeTestJSON-447170534 tempest-ImagesOneServerNegativeTestJSON-447170534-project] 10.222.0.16 "GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc" status: 404 len: 111 microversion: 2.1 time: 0.030928 Oct 15 21:05:02.683303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2393/3507] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:02 2022] GET /compute/v2.1/servers/0b72dcc6-e2fb-40ad-860d-c65f66845edc => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:02.685262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1583374300", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e8da6ca7-0893-4850-8148-25e3a241b376"}], "security_groups": [{"name": "tempest-securitygroup--899625577"}], "key_name": "tempest-keypair-808352224", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:02.850883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] validate_networks() for [('e8da6ca7-0893-4850-8148-25e3a241b376', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:05:03.436803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:05:03.437460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:05:03.437799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:05:03.438394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:05:03.438838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:05:03.448679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:03.449235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:03.449687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:03.450439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:03.450893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:03.451366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:03.468667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:03.474442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for user 2f4e4fc2f97f4c1a832c6348a97fd155 and project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:03.488924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:05:03.491608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:05:03.540688 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-baba0b4f-5089-40d6-b68f-8004e1893ab5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.859881 Oct 15 21:05:03.542059 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2394/3508] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:02 2022] POST /compute/v2.1/servers => generated 401 bytes in 862 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:05:03.552628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63d10a5b-8754-448d-b0e3-9dbea543692c tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74f1f8b80>, body: {"security_group_rule": {"parent_group_id": "9cb76274-4a6a-41c8-a3b9-33fd15672668", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "group_id": "aa505114-ef2c-4c5a-9ea2-a8a070f8c035"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:03.866303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63d10a5b-8754-448d-b0e3-9dbea543692c tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 331 microversion: 2.1 time: 0.322843 Oct 15 21:05:03.867006 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2395/3509] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:05:03 2022] POST /compute/v2.1/os-security-group-rules => generated 331 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:03.874492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:03.875121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:03.886330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:03.887212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:03.887827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:03.889450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:03.890429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:03.890626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.056498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:04.064936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b29d1f99-2660-4541-a280-49021952c4bb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.196561 Oct 15 21:05:04.065905 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2396/3510] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:03 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:04.095082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:04.095919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:04.104088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:04.104821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:04.105499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.106418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:04.107067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:04.107509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.286151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:04.294850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37f14948-874e-4fba-9d4e-ef5fe1f7f7e9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 200 len: 1511 microversion: 2.1 time: 0.204463 Oct 15 21:05:04.295442 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2397/3511] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:04 2022] GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 1511 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:04.302407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:04.303278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:04.311083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:04.311995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:04.312954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.314062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:04.314863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:04.315529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.411912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:04.423228 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e352593-9588-40f8-960e-cfcaf2bb22e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1241 microversion: 2.1 time: 0.126355 Oct 15 21:05:04.425102 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2398/3512] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:04 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1241 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:04.432348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87ebfdab-ebbb-4d6f-976e-99d9463ef6e7 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74f1f8e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:04.712305 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87ebfdab-ebbb-4d6f-976e-99d9463ef6e7 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/aa505114-ef2c-4c5a-9ea2-a8a070f8c035" status: 202 len: 0 microversion: 2.1 time: 0.285413 Oct 15 21:05:04.714031 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2399/3513] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:04 2022] DELETE /compute/v2.1/os-security-groups/aa505114-ef2c-4c5a-9ea2-a8a070f8c035 => generated 0 bytes in 288 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:04.721373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0abb935-0333-43fa-a887-ff6c50a8d4d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-2093663752", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": -1, "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:04.752371 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d0abb935-0333-43fa-a887-ff6c50a8d4d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Flavor -1 could not be found. Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0abb935-0333-43fa-a887-ff6c50a8d4d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 206 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-2093663752", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": -1, "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: exception: Flavor -1 could not be found. Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-2093663752', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': -1, 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 778, in create Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id( Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor -1 could not be found. Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:04.754981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor -1 could not be found. Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor -1 could not be found. Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0abb935-0333-43fa-a887-ff6c50a8d4d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Flavor -1 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0abb935-0333-43fa-a887-ff6c50a8d4d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 73 microversion: 2.1 time: 0.040674 Oct 15 21:05:04.765658 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2400/3514] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:04 2022] POST /compute/v2.1/servers => generated 73 bytes in 45 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:04.770267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:04.770267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:04.774567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:04.775069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:04.775516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.776233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:04.776699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:04.777170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:04.971033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:04.980552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c3e890e-b084-4b86-8921-12a781ab5faa tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1510 microversion: 2.1 time: 0.218681 Oct 15 21:05:04.981576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2401/3515] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:04 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1510 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:04.992228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-85ca13ea-81d9-4ea5-b6ce-78346cb7d83d tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe75845e160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:05.057737 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-85ca13ea-81d9-4ea5-b6ce-78346cb7d83d tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/9cb76274-4a6a-41c8-a3b9-33fd15672668" status: 200 len: 247 microversion: 2.1 time: 0.074922 Oct 15 21:05:05.058323 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2402/3516] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:05:04 2022] GET /compute/v2.1/os-security-groups/9cb76274-4a6a-41c8-a3b9-33fd15672668 => generated 247 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:05.064036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:05.064847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:05.088353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:05.090532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:05.091325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:05.092454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:05.093444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:05.095863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:05.499838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:05.516702 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-06905f2f-af47-4790-bfd7-8d5f91cea37b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.456937 Oct 15 21:05:05.525562 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2403/3517] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:05 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:05.545299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:05.551378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:05.552071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'ip': '172.20.0.13', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:05.705090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:05.706388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:05.706903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:05.707896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:05.708374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:05.708912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:05.829355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:05.906153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-603a3b08-3444-41b5-ae8f-6408bae76621 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?ip=172.20.0.13" status: 200 len: 345 microversion: 2.1 time: 0.379389 Oct 15 21:05:05.907026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2404/3518] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Oct 15 21:05:05 2022] GET /compute/v2.1/servers?ip=172.20.0.13 => generated 345 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:05.918990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73f3783f-97ab-466c-a4a1-d9f85c628b71 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74efa7160>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1983019766", "description": "tempest-description-2131384572"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:06.300515 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73f3783f-97ab-466c-a4a1-d9f85c628b71 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.390490 Oct 15 21:05:06.301467 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2405/3519] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:05:05 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:06.306451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:06.307118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:06.320537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:06.325670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:06.326343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:06.327439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:06.328551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:06.329001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:06.654804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:06.673437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10809470-8730-470b-8542-b8fbaea176ef tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1362 microversion: 2.1 time: 0.370501 Oct 15 21:05:06.674104 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2406/3520] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:06 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1362 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:06.680072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:06.680841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:06.689817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:06.691055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:06.691564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:06.692328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:06.692795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:06.693300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:06.973705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:06.983460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8046a28b-7c96-4bfb-a97a-6bd0e79e11f8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.307357 Oct 15 21:05:06.984256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2407/3521] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:06 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:06.990630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:06.991466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:06.998005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:06.998818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:06.999619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:07.000680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:07.001486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:07.002108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:07.270743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:07.279949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d235123e-6795-41d6-b0a5-24a112df31d9 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1512 microversion: 2.1 time: 0.294406 Oct 15 21:05:07.293380 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2408/3522] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:06 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1512 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:07.304299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ad50c1c-460d-4e60-b5be-4934a66f0036 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74ef9e8b0>, body: {"security_group_rule": {"parent_group_id": "8d02e970-c939-408d-b1a6-71344a3c3e68", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:07.693051 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0ad50c1c-460d-4e60-b5be-4934a66f0036 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.398156 Oct 15 21:05:07.693671 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2409/3523] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:05:07 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:07.698180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9989309d-7abe-4294-a777-0bcbeb3dad6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1510331614", "imageRef": -1, "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9989309d-7abe-4294-a777-0bcbeb3dad6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 173 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1510331614", "imageRef": -1, "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1510331614', 'imageRef': -1, 'flavorRef': '452', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: -1 is not valid under any of the given schemas Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'oneOf' in schema['properties']['server']['properties']['imageRef']: Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'oneOf': [{'format': 'uuid', 'type': 'string'}, Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'maxLength': 0, 'type': 'string'}]} Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi -1 Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:05:07.702848 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9989309d-7abe-4294-a777-0bcbeb3dad6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9989309d-7abe-4294-a777-0bcbeb3dad6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 145 microversion: 2.1 time: 0.009625 Oct 15 21:05:07.708158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2410/3524] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:07 2022] POST /compute/v2.1/servers => generated 145 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:05:07.709524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:07.710111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:07.714283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:07.718174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:07.718669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:07.719115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:07.719823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:07.720285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:07.720711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:08.236791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:08.246066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90cea212-25fe-4cad-ae30-296bc541cddd tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 200 len: 1511 microversion: 2.1 time: 0.540309 Oct 15 21:05:08.246864 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2411/3525] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:07 2022] GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 1511 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:08.251404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:08.252387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:08.259927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:08.260466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:08.260883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:08.261735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:08.262154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:08.262787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:08.481575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:08.490595 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55d495a1-dc93-4aab-bd97-7b77277c19b4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1362 microversion: 2.1 time: 0.242606 Oct 15 21:05:08.491175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2412/3526] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:08 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1362 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:08.504509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d6eb7cd-973f-4175-9571-73083caa0a7d tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Action: 'create', calling method: .version_select at 0x7fe74f8814c0>, body: {"security_group_rule": {"parent_group_id": "8d02e970-c939-408d-b1a6-71344a3c3e68", "ip_protocol": "icmp", "from_port": -1, "to_port": -1}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:08.865889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d6eb7cd-973f-4175-9571-73083caa0a7d tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 233 microversion: 2.1 time: 0.365619 Oct 15 21:05:08.866488 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2413/3527] 10.222.0.16 () {60 vars in 1195 bytes} [Sat Oct 15 21:05:08 2022] POST /compute/v2.1/os-security-group-rules => generated 233 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:08.871402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:08.872149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:08.879469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:08.879961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:08.880408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:08.881145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:08.881628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:08.882245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.036156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:09.044481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d59c0f3-b2c3-4882-9753-76854723a092 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.176962 Oct 15 21:05:09.045157 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2414/3528] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:08 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:09.172028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Calling method '.version_select at 0x7fe74efa7160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:09.185873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.186363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.186811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.187585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.188250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.188810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.200733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:09.217411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79089671-db4c-4cea-a239-07a10a314b86 tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] 10.222.0.16 "GET /compute/v2.1/limits" status: 200 len: 511 microversion: 2.1 time: 0.170904 Oct 15 21:05:09.218015 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2415/3529] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Oct 15 21:05:09 2022] GET /compute/v2.1/limits => generated 511 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:09.231701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-dd6e4403-1711-48f6-a065-04a6eeb5f01b req-8ed994cd-25d2-4c6a-b406-0701671f4d09 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d4ea7154-0623-44f0-8bad-076461f12dd5", "tag": "c039980d-116d-4619-8adf-362015a1de1c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:09.243220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd6e4403-1711-48f6-a065-04a6eeb5f01b req-8ed994cd-25d2-4c6a-b406-0701671f4d09 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.243755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd6e4403-1711-48f6-a065-04a6eeb5f01b req-8ed994cd-25d2-4c6a-b406-0701671f4d09 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.244170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd6e4403-1711-48f6-a065-04a6eeb5f01b req-8ed994cd-25d2-4c6a-b406-0701671f4d09 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.256107 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-dd6e4403-1711-48f6-a065-04a6eeb5f01b req-8ed994cd-25d2-4c6a-b406-0701671f4d09 service nova] Creating event network-changed:c039980d-116d-4619-8adf-362015a1de1c for instance d4ea7154-0623-44f0-8bad-076461f12dd5 on nch1-master-1 Oct 15 21:05:09.261644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-dd6e4403-1711-48f6-a065-04a6eeb5f01b req-8ed994cd-25d2-4c6a-b406-0701671f4d09 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042333 Oct 15 21:05:09.262196 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2416/3530] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:09.268018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:09.269272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:09.275991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.276701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.277420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.278181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.278684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.279157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.422688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:09.435224 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c74747eb-9a2c-49e3-a19c-12d470941c65 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1510 microversion: 2.1 time: 0.171718 Oct 15 21:05:09.435834 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2417/3531] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:09 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1510 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:09.442746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67a82fcf-8fd2-4966-b0bb-110beb717bbb tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74ef3d550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:09.499244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67a82fcf-8fd2-4966-b0bb-110beb717bbb tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/8d02e970-c939-408d-b1a6-71344a3c3e68" status: 200 len: 665 microversion: 2.1 time: 0.061804 Oct 15 21:05:09.499946 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2418/3532] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:05:09 2022] GET /compute/v2.1/os-security-groups/8d02e970-c939-408d-b1a6-71344a3c3e68 => generated 665 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:09.507985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:09.508732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:09.515400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.516083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.516664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.517574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:09.518253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:09.518723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:09.663499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:09.671642 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d40ac1f-f565-40c6-83e8-d7364838a483 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.167348 Oct 15 21:05:09.672223 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2419/3533] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:09 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:09.677958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AbsoluteLimitsNegativeTestJSON-server-27476609", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"0": "0", "1": "1", "2": "2", "3": "3", "4": "4", "5": "5", "6": "6", "7": "7", "8": "8", "9": "9", "10": "10", "11": "11", "12": "12", "13": "13", "14": "14", "15": "15", "16": "16", "17": "17", "18": "18", "19": "19", "20": "20", "21": "21", "22": "22", "23": "23", "24": "24", "25": "25", "26": "26", "27": "27", "28": "28", "29": "29", "30": "30", "31": "31", "32": "32", "33": "33", "34": "34", "35": "35", "36": "36", "37": "37", "38": "38", "39": "39", "40": "40", "41": "41", "42": "42", "43": "43", "44": "44", "45": "45", "46": "46", "47": "47", "48": "48", "49": "49", "50": "50", "51": "51", "52": "52", "53": "53", "54": "54", "55": "55", "56": "56", "57": "57", "58": "58", "59": "59", "60": "60", "61": "61", "62": "62", "63": "63", "64": "64", "65": "65", "66": "66", "67": "67", "68": "68", "69": "69", "70": "70", "71": "71", "72": "72", "73": "73", "74": "74", "75": "75", "76": "76", "77": "77", "78": "78", "79": "79", "80": "80", "81": "81", "82": "82", "83": "83", "84": "84", "85": "85", "86": "86", "87": "87", "88": "88", "89": "89", "90": "90", "91": "91", "92": "92", "93": "93", "94": "94", "95": "95", "96": "96", "97": "97", "98": "98", "99": "99", "100": "100", "101": "101", "102": "102", "103": "103", "104": "104", "105": "105", "106": "106", "107": "107", "108": "108", "109": "109", "110": "110", "111": "111", "112": "112", "113": "113", "114": "114", "115": "115", "116": "116", "117": "117", "118": "118", "119": "119", "120": "120", "121": "121", "122": "122", "123": "123", "124": "124", "125": "125", "126": "126", "127": "127", "128": "128"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:09.741980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:05:10.086532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Getting quotas for project 99b33358601542a1801dc998487a6867. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:10.093376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Getting quotas for user 9471c89b05814869897c5b0bbff4835f and project 99b33358601542a1801dc998487a6867. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:10.108380 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] HTTP exception thrown: Maximum number of metadata items exceeds 128 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Request method failure captured: Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 1750 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCDdVqmpoUg5VpHt-n0S_ak_yzvRwZ6Ew9ilGmpMQ-WpVMQ7y7LBF2AWnjFMOimZQFSJVpv8jx21hP5AlZXceN55AY7rFy0QiCAa4yS5owsHCMlXZQSU5WztDsT8hfjgzmP9pGYc2R8QQ8XBGdd062dJxHIc_Eqpzqh_GzkNerFquE Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 99b33358601542a1801dc998487a6867 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AbsoluteLimitsNegativeTestJSON-139345528 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/99b33358601542a1801dc998487a6867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_99b33358601542a1801dc998487a6867"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/99b33358601542a1801dc998487a6867"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/99b33358601542a1801dc998487a6867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AbsoluteLimitsNegativeTestJSON-139345528 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 99b33358601542a1801dc998487a6867 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AbsoluteLimitsNegativeTestJSON-139345528 Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AbsoluteLimitsNegativeTestJSON-139345528-project Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 9471c89b05814869897c5b0bbff4835f Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AbsoluteLimitsNegativeTestJSON-139345528-project Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-AbsoluteLimitsNegativeTestJSON-server-27476609", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "metadata": {"0": "0", "1": "1", "2": "2", "3": "3", "4": "4", "5": "5", "6": "6", "7": "7", "8": "8", "9": "9", "10": "10", "11": "11", "12": "12", "13": "13", "14": "14", "15": "15", "16": "16", "17": "17", "18": "18", "19": "19", "20": "20", "21": "21", "22": "22", "23": "23", "24": "24", "25": "25", "26": "26", "27": "27", "28": "28", "29": "29", "30": "30", "31": "31", "32": "32", "33": "33", "34": "34", "35": "35", "36": "36", "37": "37", "38": "38", "39": "39", "40": "40", "41": "41", "42": "42", "43": "43", "44": "44", "45": "45", "46": "46", "47": "47", "48": "48", "49": "49", "50": "50", "51": "51", "52": "52", "53": "53", "54": "54", "55": "55", "56": "56", "57": "57", "58": "58", "59": "59", "60": "60", "61": "61", "62": "62", "63": "63", "64": "64", "65": "65", "66": "66", "67": "67", "68": "68", "69": "69", "70": "70", "71": "71", "72": "72", "73": "73", "74": "74", "75": "75", "76": "76", "77": "77", "78": "78", "79": "79", "80": "80", "81": "81", "82": "82", "83": "83", "84": "84", "85": "85", "86": "86", "87": "87", "88": "88", "89": "89", "90": "90", "91": "91", "92": "92", "93": "93", "94": "94", "95": "95", "96": "96", "97": "97", "98": "98", "99": "99", "100": "100", "101": "101", "102": "102", "103": "103", "104": "104", "105": "105", "106": "106", "107": "107", "108": "108", "109": "109", "110": "110", "111": "111", "112": "112", "113": "113", "114": "114", "115": "115", "116": "116", "117": "117", "118": "118", "119": "119", "120": "120", "121": "121", "122": "122", "123": "123", "124": "124", "125": "125", "126": "126", "127": "127", "128": "128"}}} Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:10.110798 ncdv-master-1 devstack@n-api.service[138688]: exception: Maximum number of metadata items exceeds 128 Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-AbsoluteLimitsNegativeTestJSON-server-27476609', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'metadata': {'0': '0', '1': '1', '2': '2', '3': '3', '4': '4', '5': '5', '6': '6', '7': '7', '8': '8', '9': '9', '10': '10', '11': '11', '12': '12', '13': '13', '14': '14', '15': '15', '16': '16', '17': '17', '18': '18', '19': '19', '20': '20', '21': '21', '22': '22', '23': '23', '24': '24', '25': '25', '26': '26', '27': '27', '28': '28', '29': '29', '30': '30', '31': '31', '32': '32', '33': '33', '34': '34', '35': '35', '36': '36', '37': '37', '38': '38', '39': '39', '40': '40', '41': '41', '42': '42', '43': '43', '44': '44', '45': '45', '46': '46', '47': '47', '48': '48', '49': '49', '50': '50', '51': '51', '52': '52', '53': '53', '54': '54', '55': '55', '56': '56', '57': '57', '58': '58', '59': '59', '60': '60', '61': '61', '62': '62', '63': '63', '64': '64', '65': '65', '66': '66', '67': '67', '68': '68', '69': '69', '70': '70', '71': '71', '72': '72', '73': '73', '74': '74', '75': '75', '76': '76', '77': '77', '78': '78', '79': '79', '80': '80', '81': '81', '82': '82', '83': '83', '84': '84', '85': '85', '86': '86', '87': '87', '88': '88', '89': '89', '90': '90', '91': '91', '92': '92', '93': '93', '94': '94', '95': '95', '96': '96', '97': '97', '98': '98', '99': '99', '100': '100', '101': '101', '102': '102', '103': '103', '104': '104', '105': '105', '106': '106', '107': '107', '108': '108', '109': '109', '110': '110', '111': '111', '112': '112', '113': '113', '114': '114', '115': '115', '116': '116', '117': '117', '118': '118', '119': '119', '120': '120', '121': '121', '122': '122', '123': '123', '124': '124', '125': '125', '126': '126', '127': '127', '128': '128'}}}} Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 476, in _check_metadata_properties_quota Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi objects.Quotas.limit_check(context, metadata_items=num_metadata) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 290, in limit_check Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check( Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1118, in limit_check Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._driver.limit_check(context, self._resources, values, Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 468, in limit_check Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), quotas=quotas, Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['metadata_items'] Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1724, in _create_instance Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._checks_for_create_and_rebuild(context, image_id, boot_meta, Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1016, in _checks_for_create_and_rebuild Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._check_metadata_properties_quota(context, metadata) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 483, in _check_metadata_properties_quota Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.MetadataLimitExceeded(allowed=quota_metadata) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.MetadataLimitExceeded: Maximum number of metadata items exceeds 128 Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:10.114189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Maximum number of metadata items exceeds 128 Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Maximum number of metadata items exceeds 128 Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] Returning 403 to user: Maximum number of metadata items exceeds 128 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47fc668f-f902-4e7b-997f-f81589c09e6d tempest-AbsoluteLimitsNegativeTestJSON-139345528 tempest-AbsoluteLimitsNegativeTestJSON-139345528-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 87 microversion: 2.1 time: 0.439519 Oct 15 21:05:10.117618 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2420/3534] 10.222.0.16 () {60 vars in 1148 bytes} [Sat Oct 15 21:05:09 2022] POST /compute/v2.1/servers => generated 87 bytes in 441 msecs (HTTP/1.1 403) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:05:10.119083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:10.124928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:10.125906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:10.132068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:10.132600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:10.133183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:10.134271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:10.134783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:10.135326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:10.177950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:10.226912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:10.235451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:10.236115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:10.236621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:10.479611 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15041d3c-fcf8-4092-922a-0588f2a4258f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 4516 microversion: 2.1 time: 0.364625 Oct 15 21:05:10.480427 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2421/3535] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:05:10 2022] GET /compute/v2.1/servers/detail => generated 4516 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:10.484982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:10.485661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:10.494802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:10.495314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:10.495770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:10.496504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:10.496971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:10.497449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:10.709722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:10.720426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd7dfc3e-518a-401a-bbd4-5056d2b871e1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1362 microversion: 2.1 time: 0.239017 Oct 15 21:05:10.738397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2422/3536] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:10 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1362 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:10.746020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-553ec20d-057f-4090-aa16-0c149c718e9b tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74fa46280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:10.977744 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-553ec20d-057f-4090-aa16-0c149c718e9b tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-group-rules/0c75c061-0dbc-449b-8e4c-cf0d98927e80" status: 202 len: 0 microversion: 2.1 time: 0.237415 Oct 15 21:05:10.982112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2423/3537] 10.222.0.16 () {58 vars in 1287 bytes} [Sat Oct 15 21:05:10 2022] DELETE /compute/v2.1/os-security-group-rules/0c75c061-0dbc-449b-8e4c-cf0d98927e80 => generated 0 bytes in 240 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:10.986364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a391548-a36f-4953-b8d8-572188ce18fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-473936528", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"fixed_ip": "10.0.1.1", "uuid": "a-b-c-d-e-f-g-h-i-j"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:10.993442 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0a391548-a36f-4953-b8d8-572188ce18fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a391548-a36f-4953-b8d8-572188ce18fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 215 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-473936528", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"fixed_ip": "10.0.1.1", "uuid": "a-b-c-d-e-f-g-h-i-j"}]}} Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: exception: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-473936528', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'networks': [{'fixed_ip': '10.0.1.1', 'uuid': 'a-b-c-d-e-f-g-h-i-j'}]}}} Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 773, in create Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._process_networks_for_create( Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 547, in _process_networks_for_create Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi requested_networks = self._get_requested_networks( Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 447, in _get_requested_networks Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._validate_network_id( Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 396, in _validate_network_id Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:10.995577 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a391548-a36f-4953-b8d8-572188ce18fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a391548-a36f-4953-b8d8-572188ce18fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 123 microversion: 2.1 time: 0.017319 Oct 15 21:05:11.004524 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2424/3538] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:10 2022] POST /compute/v2.1/servers => generated 123 bytes in 23 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:05:11.009344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.015046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:11.015841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'ip': '172.20.0.', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:11.104111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.104733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.105309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.106095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.106785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.107358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.155073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:11.198715 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5471438-3241-41b8-a199-d8f84eef0d50 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?ip=172.20.0." status: 200 len: 1010 microversion: 2.1 time: 0.194695 Oct 15 21:05:11.200230 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2425/3539] 10.222.0.16 () {58 vars in 1150 bytes} [Sat Oct 15 21:05:11 2022] GET /compute/v2.1/servers?ip=172.20.0. => generated 1010 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:11.216254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb6f2099-f1d3-4b62-87f5-fad7008d2d84 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74f9a3430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.416921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb6f2099-f1d3-4b62-87f5-fad7008d2d84 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/8d02e970-c939-408d-b1a6-71344a3c3e68" status: 202 len: 0 microversion: 2.1 time: 0.214886 Oct 15 21:05:11.417619 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2426/3540] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:11 2022] DELETE /compute/v2.1/os-security-groups/8d02e970-c939-408d-b1a6-71344a3c3e68 => generated 0 bytes in 216 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:11.422515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.423167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:11.430804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.431593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.432206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.433135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.433834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.436024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.600103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:11.608120 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7ad46ba-5cb7-42e9-b574-eb9ea514a0e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.189530 Oct 15 21:05:11.608697 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2427/3541] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:11 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:11.613443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.618879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:11.619548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': '^.*\\-instance\\-[0-9]+$', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:11.624939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.625541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.626047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.626850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.627464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.628328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.720104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:11.781215 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54ed1ef6-55de-4e8e-9817-f393969e3c7b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-%5B0-9%5D%2B%24" status: 200 len: 1010 microversion: 2.1 time: 0.171326 Oct 15 21:05:11.781816 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2428/3542] 10.222.0.16 () {58 vars in 1212 bytes} [Sat Oct 15 21:05:11 2022] GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-%5B0-9%5D%2B%24 => generated 1010 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:11.786309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74f9a3280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.809138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG neutronclient.v2_0.client [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist", "detail": ""}} {{(pid=138688) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 21:05:11.809754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Neutron security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 not found {{(pid=138688) get /opt/stack/nova/nova/network/security_group_api.py:313}} Oct 15 21:05:11.810293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] HTTP exception thrown: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.810293 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Request method failure captured: Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-security-groups/aa505114-ef2c-4c5a-9ea2-a8a070f8c035 HTTP/1.1 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyBxJyzNugpjDcn5tJAvsy_VclIl6Ru8_KulhEUU9NAdG-tX-JcMolWCXeWZyPeXnPFamKLTOrHBuADEUukGSWnICY5QRreg9CrAQkIuxCLuti3pmRZXlsm7jjCYz-_aOtAcsKpgtgYbD6jw1pmdOot9RzYxNSBxY5KmXTx8Q1trN00 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b6a0bed8192d413da2cc4d413b5bc569 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-SecurityGroupRulesTestJSON-491918323 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6a0bed8192d413da2cc4d413b5bc569"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6a0bed8192d413da2cc4d413b5bc569"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6a0bed8192d413da2cc4d413b5bc569"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6a0bed8192d413da2cc4d413b5bc569"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-SecurityGroupRulesTestJSON-491918323 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b6a0bed8192d413da2cc4d413b5bc569 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-SecurityGroupRulesTestJSON-491918323 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-SecurityGroupRulesTestJSON-491918323-project Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c7deeb24e8834437a8410f8cc0c29f6d Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-SecurityGroupRulesTestJSON-491918323-project Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: exception: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'aa505114-ef2c-4c5a-9ea2-a8a070f8c035'} Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Oct 15 21:05:11.812131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Returning 404 to user: Security group aa505114-ef2c-4c5a-9ea2-a8a070f8c035 does not exist Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: Neutron server returns request_ids: ['req-d5846797-9cf7-4591-bddc-ce721f3a8f65'] {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca3994d7-483a-42f3-84b1-e16453b6aa97 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/aa505114-ef2c-4c5a-9ea2-a8a070f8c035" status: 404 len: 194 microversion: 2.1 time: 0.030905 Oct 15 21:05:11.817066 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2429/3543] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:11 2022] DELETE /compute/v2.1/os-security-groups/aa505114-ef2c-4c5a-9ea2-a8a070f8c035 => generated 194 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:11.820887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.820887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:11.831992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.832428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.832872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.833613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.834079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.834514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.969837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:11.979612 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3565a30-f349-4dc8-9da1-76b569b8576f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.164129 Oct 15 21:05:11.980719 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2430/3544] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:11 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:11.985846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:11.986689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:11.994003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.994730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.995318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:11.996113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:11.996643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:11.997272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:12.188936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:12.198209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b3365c3-2fae-4f1b-a96f-395e94d3f918 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1362 microversion: 2.1 time: 0.216133 Oct 15 21:05:12.199054 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2431/3545] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:11 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1362 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:12.219998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:12.226425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:12.227100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': '^.*\\-instance\\-.*$', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:12.232946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:12.233709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:12.234215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:12.234987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:12.235684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:12.236116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:12.333435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:12.396889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09e0d9ec-fd15-4285-be40-7e1e75aba772 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-.%2A%24" status: 200 len: 1010 microversion: 2.1 time: 0.182596 Oct 15 21:05:12.397442 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2432/3546] 10.222.0.16 () {58 vars in 1196 bytes} [Sat Oct 15 21:05:12 2022] GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-.%2A%24 => generated 1010 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:12.411549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f16de678-5674-4c0c-ad0a-9f0c19ba44a8 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74fa9c8b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:12.767868 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f16de678-5674-4c0c-ad0a-9f0c19ba44a8 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/9cb76274-4a6a-41c8-a3b9-33fd15672668" status: 202 len: 0 microversion: 2.1 time: 0.361214 Oct 15 21:05:12.768790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2433/3547] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:12 2022] DELETE /compute/v2.1/os-security-groups/9cb76274-4a6a-41c8-a3b9-33fd15672668 => generated 0 bytes in 371 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:12.773975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8022fb39-8a4d-4329-af84-bb363f5b484b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-545078809", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "key_name": "tempest-key-1615104170", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:12.836930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-8022fb39-8a4d-4329-af84-bb363f5b484b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] validate_networks() for [('e3c96722-97ae-4b06-ab5c-8a523bae70f3', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:05:13.079796 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8022fb39-8a4d-4329-af84-bb363f5b484b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Invalid key_name provided. Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8022fb39-8a4d-4329-af84-bb363f5b484b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 246 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-545078809", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "key_name": "tempest-key-1615104170", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid key_name provided. Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-545078809', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'key_name': 'tempest-key-1615104170', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1089, in _validate_and_build_base_options Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi key_pair = objects.KeyPair.get_by_name(context, Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-key-1615104170 not found for user 90e8754a100240a498e422de07c76bbf Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:13.083175 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 807, in create Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid key_name provided. Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid key_name provided. Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8022fb39-8a4d-4329-af84-bb363f5b484b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid key_name provided. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8022fb39-8a4d-4329-af84-bb363f5b484b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 70 microversion: 2.1 time: 0.315531 Oct 15 21:05:13.089264 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2434/3548] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:12 2022] POST /compute/v2.1/servers => generated 70 bytes in 318 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 21:05:13.093019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:13.099545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:13.100688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': '1942351304', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:13.110861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:13.110861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:13.111793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:13.112847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:13.113593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:13.114205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:13.220428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:13.291458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ff0edf9-f620-47e4-8126-73c9f4a12281 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?name=1942351304" status: 200 len: 346 microversion: 2.1 time: 0.203198 Oct 15 21:05:13.292340 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2435/3549] 10.222.0.16 () {58 vars in 1156 bytes} [Sat Oct 15 21:05:13 2022] GET /compute/v2.1/servers?name=1942351304 => generated 346 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:13.297196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-21e93e07-82a8-4f70-a8df-743924205efa req-b4494ff4-6802-4ee3-9f11-8eca18ef8d75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d4ea7154-0623-44f0-8bad-076461f12dd5", "name": "network-vif-plugged", "status": "completed", "tag": "c039980d-116d-4619-8adf-362015a1de1c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:13.308791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-21e93e07-82a8-4f70-a8df-743924205efa req-b4494ff4-6802-4ee3-9f11-8eca18ef8d75 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:13.309584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-21e93e07-82a8-4f70-a8df-743924205efa req-b4494ff4-6802-4ee3-9f11-8eca18ef8d75 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:13.310088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-21e93e07-82a8-4f70-a8df-743924205efa req-b4494ff4-6802-4ee3-9f11-8eca18ef8d75 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:13.326000 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-21e93e07-82a8-4f70-a8df-743924205efa req-b4494ff4-6802-4ee3-9f11-8eca18ef8d75 service nova] Creating event network-vif-plugged:c039980d-116d-4619-8adf-362015a1de1c for instance d4ea7154-0623-44f0-8bad-076461f12dd5 on nch1-master-1 Oct 15 21:05:13.334037 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-21e93e07-82a8-4f70-a8df-743924205efa req-b4494ff4-6802-4ee3-9f11-8eca18ef8d75 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039576 Oct 15 21:05:13.335091 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2436/3550] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:13.340952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bc6a071-715b-4911-9bab-4697d33b552f tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74fb07dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:13.753522 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bc6a071-715b-4911-9bab-4697d33b552f tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/b43f4877-9abf-4809-8cad-71fabfe79114" status: 202 len: 0 microversion: 2.1 time: 0.416678 Oct 15 21:05:13.754037 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2437/3551] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:13 2022] DELETE /compute/v2.1/os-security-groups/b43f4877-9abf-4809-8cad-71fabfe79114 => generated 0 bytes in 418 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:13.760082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:13.760824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:13.770871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:13.773376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:13.774021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:13.774938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:13.775588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:13.776201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:13.989675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:13.999357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc97ee8a-04cf-42df-bde9-8f01c84db479 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.243578 Oct 15 21:05:14.000537 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2438/3552] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:13 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:14.027906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:14.028785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:14.037312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.038122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.038676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.039703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.040265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.040810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.196398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:14.205840 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3df198c-6cec-4002-9af7-087f6f732d3d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1362 microversion: 2.1 time: 0.190021 Oct 15 21:05:14.206601 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2439/3553] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:14 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1362 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:14.220637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:14.226578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:14.227429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': '-instance', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:14.233495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.234078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.234612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.235419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.235957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.236466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.328226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:14.332867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:14.389948 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4fa41dd-2ad8-411d-82bf-477d94f1882d tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?name=-instance" status: 200 len: 1010 microversion: 2.1 time: 0.182319 Oct 15 21:05:14.390914 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2440/3554] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Oct 15 21:05:14 2022] GET /compute/v2.1/servers?name=-instance => generated 1010 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:14.395738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e819fb3d-35c4-4bea-9e6a-2654e52e35ef tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe7544ad790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:14.598050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e819fb3d-35c4-4bea-9e6a-2654e52e35ef tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/43cdcc72-02ab-497a-ad0f-54d292dac97e" status: 202 len: 0 microversion: 2.1 time: 0.205926 Oct 15 21:05:14.598935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2441/3555] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:14 2022] DELETE /compute/v2.1/os-security-groups/43cdcc72-02ab-497a-ad0f-54d292dac97e => generated 0 bytes in 207 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:14.604896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:14.606122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:14.614070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.616032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.616734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.617749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.618455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.619051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.787787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:14.796554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a36871dc-9525-4a29-aabc-76259dc3214d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.195887 Oct 15 21:05:14.797472 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2442/3556] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:14 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:14.803769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:14.808924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:14.809629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Searching by: {'name': 't-ListServerFiltersTestJSON-instance-194235130', 'deleted': False, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:05:14.815027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.816047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.816836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.817342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:14.817699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:14.820519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:14.941654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:05:15.022484 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efbe9192-8ddb-43ba-b2da-ab5797d79135 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers?name=t-ListServerFiltersTestJSON-instance-194235130" status: 200 len: 346 microversion: 2.1 time: 0.223642 Oct 15 21:05:15.023900 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2443/3557] 10.222.0.16 () {58 vars in 1228 bytes} [Sat Oct 15 21:05:14 2022] GET /compute/v2.1/servers?name=t-ListServerFiltersTestJSON-instance-194235130 => generated 346 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:15.038038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2153becf-1db8-4259-acd1-089546ef3561 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe74fa3c700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:15.474098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2153becf-1db8-4259-acd1-089546ef3561 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/923d82a5-4c1f-4a65-8a70-05de3d61a060" status: 202 len: 0 microversion: 2.1 time: 0.448573 Oct 15 21:05:15.475375 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2444/3558] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:15 2022] DELETE /compute/v2.1/os-security-groups/923d82a5-4c1f-4a65-8a70-05de3d61a060 => generated 0 bytes in 450 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:15.482080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46657140-9e05-4808-8cfd-3636f4572702 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-306198597", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "security_groups": [{"name": "does_not_exist"}], "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:15.654344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.security_group_api [None req-46657140-9e05-4808-8cfd-3636f4572702 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Neutron security group does_not_exist not found {{(pid=138688) validate_name /opt/stack/nova/nova/network/security_group_api.py:74}} Oct 15 21:05:15.655686 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-46657140-9e05-4808-8cfd-3636f4572702 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Unable to find security_group with name or id 'does_not_exist' Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46657140-9e05-4808-8cfd-3636f4572702 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 257 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-306198597", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "security_groups": [{"name": "does_not_exist"}], "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: exception: Unable to find security_group with name or id 'does_not_exist' Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-306198597', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'security_groups': [{'name': 'does_not_exist'}], 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 476, in find_resource Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.find_resource_by_id(resource, name_or_id, Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 440, in find_resource_by_id Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with id 'does_not_exist' Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 480, in find_resource Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._find_resource_by_name( Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 469, in _find_resource_by_name Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name 'does_not_exist' Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 68, in validate_name Oct 15 21:05:15.677143 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return neutronv20.find_resourceid_by_name_or_id( Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 56, in find_resourceid_by_name_or_id Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return find_resource_by_name_or_id(client, resource, name_or_id, Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 49, in find_resource_by_name_or_id Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return client.find_resource(resource, name_or_id, project_id, Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 488, in find_resource Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound( Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name or id 'does_not_exist' Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1074, in _validate_and_build_base_options Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi security_groups = self._check_requested_secgroups( Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 521, in _check_requested_secgroups Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi secgroup_uuid = security_group_api.validate_name(context, secgroup) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 75, in validate_name Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Unable to find security_group with name or id 'does_not_exist' Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Unable to find security_group with name or id 'does_not_exist' Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Unable to find security_group with name or id 'does_not_exist' Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:15.683315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46657140-9e05-4808-8cfd-3636f4572702 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Unable to find security_group with name or id 'does_not_exist' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:15.692272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46657140-9e05-4808-8cfd-3636f4572702 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 106 microversion: 2.1 time: 0.200272 Oct 15 21:05:15.692272 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2445/3559] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:15 2022] POST /compute/v2.1/servers => generated 106 bytes in 205 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:05:15.692272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:15.692272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:15.699851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:15.699851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:15.700248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:15.700957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:15.701718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:15.701849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:15.729209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:05:15.831881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0c73e62-962c-41e3-bf66-2a4be2713565 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "DELETE /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 204 len: 0 microversion: 2.1 time: 0.147251 Oct 15 21:05:15.833091 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2446/3560] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:05:15 2022] DELETE /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:05:15.839660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:15.841104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:15.848912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:15.849641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:15.850662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:15.851140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:15.851715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:15.852282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:16.135831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:16.145075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eedc7882-42f5-4c51-8f49-feab095b40cd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1548 microversion: 2.1 time: 0.310522 Oct 15 21:05:16.147034 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2447/3561] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:15 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1548 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:16.165124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15f80be7-c697-4328-80e5-b3d74c2178d5 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] Calling method '.version_select at 0x7fe754590550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:16.477628 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:05:16.478222 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:05:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:05:16.581987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15f80be7-c697-4328-80e5-b3d74c2178d5 tempest-SecurityGroupRulesTestJSON-491918323 tempest-SecurityGroupRulesTestJSON-491918323-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/4f923023-c258-449e-b560-40ca0d79954f" status: 202 len: 0 microversion: 2.1 time: 0.435193 Oct 15 21:05:16.582650 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2448/3562] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:05:16 2022] DELETE /compute/v2.1/os-security-groups/4f923023-c258-449e-b560-40ca0d79954f => generated 0 bytes in 436 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:16.604505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:16.605149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:16.628407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:16.629001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:16.630324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:16.631057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:16.631529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:16.632572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:16.925884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:16.936104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-15f8d72a-49f7-43f5-820f-d707306fc875 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.342152 Oct 15 21:05:16.937329 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2449/3563] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:16 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:16.943228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:16.944055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:16.969831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:16.970376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:16.970774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:16.971567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:16.985606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:16.987478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:17.227450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:17.254908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c9e990c-8567-4354-861c-43fedc8628ea tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1516 microversion: 2.1 time: 0.316093 Oct 15 21:05:17.258761 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2450/3564] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:16 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1516 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:17.264960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:17.267779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:17.275706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:17.276752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:17.277498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:17.278609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:17.279386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:17.280147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:17.453883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c624bfd5-e79b-446c-9180-61a76d4e2f7c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.193688 Oct 15 21:05:17.455046 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2451/3565] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:05:17 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-interface => generated 284 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:17.460259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:17.461775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:17.478007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:17.478519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:17.478961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:17.480355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:17.481169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:17.481840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:18.039419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:18.053435 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5efae17b-ba1a-4e54-8019-46fd9db5d5fb tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.596245 Oct 15 21:05:18.054736 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2452/3566] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:17 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:18.310867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-405de6aa-7fc2-447a-985e-53204d5523cd tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Action: 'create', calling method: .version_select at 0x7fe74fefcf70>, body: {"keypair": {"name": "tempest-keypair-2028445312", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMWRMCn96z4uZ6FKblnaFT60xMOZSckwcB/K3EQDtYj8MyBRmnUccKyUADCaXFpnz9x/nCTtz8Vu295pNrNixPxxA97zLsuGw/7uQFNZYwNyQJybq8Dh10EhFx8CdlSpoA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:18.347326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-405de6aa-7fc2-447a-985e-53204d5523cd tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Getting quotas for project 87739141ec764f6ca347224117dc45c2. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:18.367127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-405de6aa-7fc2-447a-985e-53204d5523cd tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Getting quotas for user 86129e282ee8469eba032a84e9a55096 and project 87739141ec764f6ca347224117dc45c2. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:18.405684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-405de6aa-7fc2-447a-985e-53204d5523cd tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.349110 Oct 15 21:05:18.407139 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2453/3567] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:05:18 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:18.413581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9967ec7-be48-4cc1-aafc-81924a2cf2e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:18.414564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f9967ec7-be48-4cc1-aafc-81924a2cf2e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 9ef5453a-0f59-447d-aef0-05a37853f8b9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:18.443108 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f9967ec7-be48-4cc1-aafc-81924a2cf2e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 9ef5453a-0f59-447d-aef0-05a37853f8b9 could not be found. Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9967ec7-be48-4cc1-aafc-81924a2cf2e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/9ef5453a-0f59-447d-aef0-05a37853f8b9 HTTP/1.1 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 9ef5453a-0f59-447d-aef0-05a37853f8b9 could not be found. Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '9ef5453a-0f59-447d-aef0-05a37853f8b9'} Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9ef5453a-0f59-447d-aef0-05a37853f8b9 could not be found. Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9ef5453a-0f59-447d-aef0-05a37853f8b9 could not be found. Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:18.447260 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9ef5453a-0f59-447d-aef0-05a37853f8b9 could not be found. Oct 15 21:05:18.452745 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:18.454492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9967ec7-be48-4cc1-aafc-81924a2cf2e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 9ef5453a-0f59-447d-aef0-05a37853f8b9 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:18.456645 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9967ec7-be48-4cc1-aafc-81924a2cf2e5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9ef5453a-0f59-447d-aef0-05a37853f8b9" status: 404 len: 111 microversion: 2.1 time: 0.047729 Oct 15 21:05:18.458013 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2454/3568] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:05:18 2022] DELETE /compute/v2.1/servers/9ef5453a-0f59-447d-aef0-05a37853f8b9 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:18.464386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:18.466295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:18.510357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:18.514561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:18.516033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:18.516974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:18.518883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:18.520645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:18.639041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:18.658038 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4be97fd9-79e5-4c5a-a9db-8f8aa4b44af4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 200 len: 1474 microversion: 2.1 time: 0.197950 Oct 15 21:05:18.673064 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2455/3569] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:18 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 1474 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:18.686622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:18.688529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:18.712754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:18.714713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:18.715165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:18.716066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:18.716353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:18.716776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:18.956619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:18.967301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4a712fc-3086-4051-8643-3f5b40fda2d7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.304650 Oct 15 21:05:18.969799 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2456/3570] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:18 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:18.976055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a8b51786-5a63-461f-91c5-d8c2eb7b45be req-ab1eb740-578a-4241-ac59-360e22d887a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35c7b923-29cf-4679-ae98-f8f8b14141aa", "name": "network-vif-deleted", "tag": "b54ef53b-2708-4479-8161-64ddb65c9afb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:18.988460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:18.994246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a8b51786-5a63-461f-91c5-d8c2eb7b45be req-ab1eb740-578a-4241-ac59-360e22d887a1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:18.994794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a8b51786-5a63-461f-91c5-d8c2eb7b45be req-ab1eb740-578a-4241-ac59-360e22d887a1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:18.995226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a8b51786-5a63-461f-91c5-d8c2eb7b45be req-ab1eb740-578a-4241-ac59-360e22d887a1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.011677 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a8b51786-5a63-461f-91c5-d8c2eb7b45be req-ab1eb740-578a-4241-ac59-360e22d887a1 service nova] Creating event network-vif-deleted:b54ef53b-2708-4479-8161-64ddb65c9afb for instance 35c7b923-29cf-4679-ae98-f8f8b14141aa on nch2-master-1 Oct 15 21:05:19.018717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a8b51786-5a63-461f-91c5-d8c2eb7b45be req-ab1eb740-578a-4241-ac59-360e22d887a1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046948 Oct 15 21:05:19.022866 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2457/3571] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:19.029154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.029857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:19.037599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.038096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.038539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.039319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.039787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.040344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.192940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:19.208374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2bf1704-c5cf-4d46-82ee-9557f4906349 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.183789 Oct 15 21:05:19.221731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2458/3572] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:19 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:19.233368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2db5e58e-4f5e-4f6e-95bd-00d3bbad2067 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.233871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2db5e58e-4f5e-4f6e-95bd-00d3bbad2067 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Failed to fetch instance by id 9223372036854775808 {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 21:05:19.234392 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-2db5e58e-4f5e-4f6e-95bd-00d3bbad2067 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 9223372036854775808 could not be found. Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2db5e58e-4f5e-4f6e-95bd-00d3bbad2067 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/9223372036854775808 HTTP/1.1 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 9223372036854775808 could not be found. Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '9223372036854775808'} Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9223372036854775808 could not be found. Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9223372036854775808 could not be found. Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.236560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9223372036854775808 could not be found. Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2db5e58e-4f5e-4f6e-95bd-00d3bbad2067 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 9223372036854775808 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2db5e58e-4f5e-4f6e-95bd-00d3bbad2067 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9223372036854775808" status: 404 len: 94 microversion: 2.1 time: 0.009975 Oct 15 21:05:19.246560 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2459/3573] 10.222.0.16 () {58 vars in 1188 bytes} [Sat Oct 15 21:05:19 2022] DELETE /compute/v2.1/servers/9223372036854775808 => generated 94 bytes in 11 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:05:19.248314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.248939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:19.258397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.258397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.258397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.258957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.259424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.260174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.450979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:19.461051 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2daa8c7-acc8-4d96-842d-8c705247f75c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.217239 Oct 15 21:05:19.474103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2460/3574] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:19 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:19.493573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.495335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:19.504410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.505310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.506128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.507061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.508447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.509753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.662526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:19.671169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-109f4888-cd6b-4a40-abf0-401603285904 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.185593 Oct 15 21:05:19.672404 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2461/3575] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:19 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:19.683754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.688588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 35c7b923-29cf-4679-ae98-f8f8b14141aa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:19.696463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.697293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.697606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.698220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.698701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.699163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.714375 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] HTTP exception thrown: Instance 35c7b923-29cf-4679-ae98-f8f8b14141aa could not be found. Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Request method failure captured: Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa HTTP/1.1 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyArzmlCEYIPl4mxNsPOXxe_PqEEzxxFXLVeMbQusSAwsVeVMXpvcatarH-XlZd_FAGoRdCsVCQKQ318ISD1ucjN7o_rNjFTT0eze8tjZz8LnsfpZMKRMM3hpdUGmXuctiDXxvaWFrPml9AqktXxMti2ILG2-crrbDdTM_LQ2woVKOA Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f20ae5c8217d412289d006ab9e01e7f4 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f20ae5c8217d412289d006ab9e01e7f4 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServerFiltersTestJSON-556029331-project Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 124511010d044f8fb07dc09e7d754303 Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServerFiltersTestJSON-556029331-project Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 35c7b923-29cf-4679-ae98-f8f8b14141aa could not be found. Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '35c7b923-29cf-4679-ae98-f8f8b14141aa'} Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35c7b923-29cf-4679-ae98-f8f8b14141aa could not be found. Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35c7b923-29cf-4679-ae98-f8f8b14141aa could not be found. Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:19.716424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35c7b923-29cf-4679-ae98-f8f8b14141aa could not be found. Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Returning 404 to user: Instance 35c7b923-29cf-4679-ae98-f8f8b14141aa could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:19.722719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-050bb328-2184-42d0-9d15-be152444ca54 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa" status: 404 len: 111 microversion: 2.1 time: 0.040732 Oct 15 21:05:19.725159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2462/3576] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:19 2022] GET /compute/v2.1/servers/35c7b923-29cf-4679-ae98-f8f8b14141aa => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:19.731496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4e5879d-87e1-4c29-96fe-7a1293a1538d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.731992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4e5879d-87e1-4c29-96fe-7a1293a1538d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Failed to fetch instance by id -1 {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 21:05:19.732505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f4e5879d-87e1-4c29-96fe-7a1293a1538d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance -1 could not be found. Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4e5879d-87e1-4c29-96fe-7a1293a1538d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/-1 HTTP/1.1 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance -1 could not be found. Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '-1'} Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance -1 could not be found. Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance -1 could not be found. Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:19.734029 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance -1 could not be found. Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4e5879d-87e1-4c29-96fe-7a1293a1538d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance -1 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4e5879d-87e1-4c29-96fe-7a1293a1538d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "DELETE /compute/v2.1/servers/-1" status: 404 len: 77 microversion: 2.1 time: 0.009604 Oct 15 21:05:19.741752 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2463/3577] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 21:05:19 2022] DELETE /compute/v2.1/servers/-1 => generated 77 bytes in 11 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:05:19.742786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:19.742786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:19.750153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.750663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.751111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.751823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:19.753431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:19.754316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:19.852877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:05:19.856676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:20.029790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c61b1fc3-8ae4-430b-8789-b7db3d022b07 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "DELETE /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 204 len: 0 microversion: 2.1 time: 0.291196 Oct 15 21:05:20.031846 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2464/3578] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:05:19 2022] DELETE /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:05:20.042806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:20.054368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:20.089010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:20.095445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:20.096724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:20.098314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:20.100254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:20.102707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:20.757416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:20.774133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed6e8963-47ba-4ab1-b6e6-e96060cbe4cc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.740161 Oct 15 21:05:20.775049 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2465/3579] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:20 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:20.944678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8a0e2835-b78e-46c0-89ce-abc3052455e7 req-7f576bef-6b2e-447c-bb41-6d7003c4a815 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d4ea7154-0623-44f0-8bad-076461f12dd5", "tag": "c039980d-116d-4619-8adf-362015a1de1c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:20.959974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8a0e2835-b78e-46c0-89ce-abc3052455e7 req-7f576bef-6b2e-447c-bb41-6d7003c4a815 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:20.960392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8a0e2835-b78e-46c0-89ce-abc3052455e7 req-7f576bef-6b2e-447c-bb41-6d7003c4a815 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:20.960815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8a0e2835-b78e-46c0-89ce-abc3052455e7 req-7f576bef-6b2e-447c-bb41-6d7003c4a815 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:21.003982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8a0e2835-b78e-46c0-89ce-abc3052455e7 req-7f576bef-6b2e-447c-bb41-6d7003c4a815 service nova] Creating event network-changed:c039980d-116d-4619-8adf-362015a1de1c for instance d4ea7154-0623-44f0-8bad-076461f12dd5 on nch1-master-1 Oct 15 21:05:21.024654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8a0e2835-b78e-46c0-89ce-abc3052455e7 req-7f576bef-6b2e-447c-bb41-6d7003c4a815 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.246939 Oct 15 21:05:21.025566 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2466/3580] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:21.030945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:21.040747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:21.052253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:21.062772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:21.063548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:21.064597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:21.065754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:21.066446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:21.707620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:21.729418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f0dcae84-6bdb-4009-ae72-52579bff3d3a tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 200 len: 1517 microversion: 2.1 time: 0.702344 Oct 15 21:05:21.730529 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2467/3581] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:21 2022] GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 1517 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:21.744615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:21.751733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:21.767650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:21.768917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:21.769177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:21.774190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:21.774677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:21.775119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:22.461275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:22.476112 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d8e44c5-82a3-446b-bfb8-e7b82b3ec8a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.743500 Oct 15 21:05:22.504471 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2468/3582] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:21 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:22.520839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aaee17f-6c85-4a95-a2b0-69259b9c68c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"forceDelete": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:22.522004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0aaee17f-6c85-4a95-a2b0-69259b9c68c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 50374607-1c72-4075-8459-face97c33ff5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:22.529885 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0aaee17f-6c85-4a95-a2b0-69259b9c68c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 50374607-1c72-4075-8459-face97c33ff5 could not be found. Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aaee17f-6c85-4a95-a2b0-69259b9c68c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/50374607-1c72-4075-8459-face97c33ff5/action HTTP/1.1 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 19 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: {"forceDelete": {}} Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 50374607-1c72-4075-8459-face97c33ff5 could not be found. Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '50374607-1c72-4075-8459-face97c33ff5', 'body': {'forceDelete': {}}} Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50374607-1c72-4075-8459-face97c33ff5 could not be found. Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 55, in _force_delete Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50374607-1c72-4075-8459-face97c33ff5 could not be found. Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:22.538013 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:22.544365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:22.544365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:22.544365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50374607-1c72-4075-8459-face97c33ff5 could not be found. Oct 15 21:05:22.544365 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:22.544365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aaee17f-6c85-4a95-a2b0-69259b9c68c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 50374607-1c72-4075-8459-face97c33ff5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:22.544365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0aaee17f-6c85-4a95-a2b0-69259b9c68c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/50374607-1c72-4075-8459-face97c33ff5/action" status: 404 len: 111 microversion: 2.1 time: 0.036986 Oct 15 21:05:22.555883 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2469/3583] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:22 2022] POST /compute/v2.1/servers/50374607-1c72-4075-8459-face97c33ff5/action => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:22.583141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:22.585912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:22.601098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:22.602802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:22.604841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:22.606721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:22.609922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:22.610500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:22.950847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:22.961464 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3277999f-2603-46e7-8d36-81e86bfdc99c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.393489 Oct 15 21:05:22.977125 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2470/3584] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:22 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:23.007740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:23.008573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:23.032187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.032724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.033270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:23.034581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.035071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.035564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:23.183637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:23.201213 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef3a417a-116f-40a8-88bb-0f32843d1b8b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 200 len: 1475 microversion: 2.1 time: 0.208154 Oct 15 21:05:23.222022 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2471/3585] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:22 2022] GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 1475 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:23.239149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:23.242101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:23.265748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.266368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.268422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:23.270542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.272784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.274029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:23.558040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:23.569487 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ace607a1-f989-43e1-b482-74bd1add8007 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.345339 Oct 15 21:05:23.571182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2472/3586] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:23 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:23.576725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-31beb032-8e3c-4707-91ec-66d2b045f4b3 req-4c234901-a798-4342-aaa4-3543a458f296 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820", "name": "network-vif-deleted", "tag": "8cf88a8d-33e1-47f1-bf30-24878a31214c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:23.605344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-31beb032-8e3c-4707-91ec-66d2b045f4b3 req-4c234901-a798-4342-aaa4-3543a458f296 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.605733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-31beb032-8e3c-4707-91ec-66d2b045f4b3 req-4c234901-a798-4342-aaa4-3543a458f296 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.606202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-31beb032-8e3c-4707-91ec-66d2b045f4b3 req-4c234901-a798-4342-aaa4-3543a458f296 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:23.661761 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-31beb032-8e3c-4707-91ec-66d2b045f4b3 req-4c234901-a798-4342-aaa4-3543a458f296 service nova] Creating event network-vif-deleted:8cf88a8d-33e1-47f1-bf30-24878a31214c for instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 on nch2-master-1 Oct 15 21:05:23.670844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-31beb032-8e3c-4707-91ec-66d2b045f4b3 req-4c234901-a798-4342-aaa4-3543a458f296 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098085 Oct 15 21:05:23.680400 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2473/3587] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:23.688407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f1bca95-93f7-4fcf-b2f0-2db7ecb140d3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 10}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:23.693745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f1bca95-93f7-4fcf-b2f0-2db7ecb140d3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: ba711570-045f-4eb7-97a2-40721ca32e33] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:23.701125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-2f1bca95-93f7-4fcf-b2f0-2db7ecb140d3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance ba711570-045f-4eb7-97a2-40721ca32e33 could not be found. Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f1bca95-93f7-4fcf-b2f0-2db7ecb140d3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ba711570-045f-4eb7-97a2-40721ca32e33/action HTTP/1.1 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 39 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: {"os-getConsoleOutput": {"length": 10}} Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ba711570-045f-4eb7-97a2-40721ca32e33 could not be found. Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ba711570-045f-4eb7-97a2-40721ca32e33', 'body': {'os-getConsoleOutput': {'length': 10}}} Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ba711570-045f-4eb7-97a2-40721ca32e33 could not be found. Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/console_output.py", line 41, in get_console_output Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ba711570-045f-4eb7-97a2-40721ca32e33 could not be found. Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:23.704226 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:23.710350 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:23.710350 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:23.710350 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:23.710350 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:23.710350 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ba711570-045f-4eb7-97a2-40721ca32e33 could not be found. Oct 15 21:05:23.710350 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:23.719694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f1bca95-93f7-4fcf-b2f0-2db7ecb140d3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance ba711570-045f-4eb7-97a2-40721ca32e33 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:23.721294 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f1bca95-93f7-4fcf-b2f0-2db7ecb140d3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/ba711570-045f-4eb7-97a2-40721ca32e33/action" status: 404 len: 111 microversion: 2.1 time: 0.038945 Oct 15 21:05:23.722479 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2474/3588] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:23 2022] POST /compute/v2.1/servers/ba711570-045f-4eb7-97a2-40721ca32e33/action => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:23.741620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:23.741620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:23.757686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.758428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.758847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:23.759673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:23.760075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:23.760529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.114805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:24.126003 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58ed68e2-ecf7-470c-a02c-64f83cb7491c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.398863 Oct 15 21:05:24.126889 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2475/3589] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:23 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:24.157995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:24.162336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:24.179665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.180167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.180805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.182310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.182985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.184015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.410501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:24.419919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60499e8a-7089-4d0e-b10e-9271b482fb6d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.269488 Oct 15 21:05:24.433651 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2476/3590] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:24 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:24.445687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:24.446805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:24.454308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.455537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.456320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.457224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.457863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.459197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.476304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] HTTP exception thrown: Instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 could not be found. Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Request method failure captured: Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 HTTP/1.1 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyArzmlCEYIPl4mxNsPOXxe_PqEEzxxFXLVeMbQusSAwsVeVMXpvcatarH-XlZd_FAGoRdCsVCQKQ318ISD1ucjN7o_rNjFTT0eze8tjZz8LnsfpZMKRMM3hpdUGmXuctiDXxvaWFrPml9AqktXxMti2ILG2-crrbDdTM_LQ2woVKOA Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f20ae5c8217d412289d006ab9e01e7f4 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f20ae5c8217d412289d006ab9e01e7f4 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServerFiltersTestJSON-556029331-project Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 124511010d044f8fb07dc09e7d754303 Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServerFiltersTestJSON-556029331-project Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 could not be found. Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820'} Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 could not be found. Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 could not be found. Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:24.477889 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 could not be found. Oct 15 21:05:24.483470 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.485099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Returning 404 to user: Instance 29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:24.486800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-281fcc54-2410-4654-84d9-bc50e31225d6 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820" status: 404 len: 111 microversion: 2.1 time: 0.051204 Oct 15 21:05:24.488211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2477/3591] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:24 2022] GET /compute/v2.1/servers/29f9bdbc-e2c6-4022-a1d1-2cfa71cb5820 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:24.493950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a77f76f7-67e8-49ee-9468-e9d45862a629 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:24.494747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a77f76f7-67e8-49ee-9468-e9d45862a629 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 8af7f097-ba99-4637-ad23-ab81689411f8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:24.501289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a77f76f7-67e8-49ee-9468-e9d45862a629 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 8af7f097-ba99-4637-ad23-ab81689411f8 could not be found. Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a77f76f7-67e8-49ee-9468-e9d45862a629 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8af7f097-ba99-4637-ad23-ab81689411f8 HTTP/1.1 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8af7f097-ba99-4637-ad23-ab81689411f8 could not be found. Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8af7f097-ba99-4637-ad23-ab81689411f8'} Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8af7f097-ba99-4637-ad23-ab81689411f8 could not be found. Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8af7f097-ba99-4637-ad23-ab81689411f8 could not be found. Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:24.503782 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:24.509021 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:24.509021 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:24.509021 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:24.509021 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8af7f097-ba99-4637-ad23-ab81689411f8 could not be found. Oct 15 21:05:24.509021 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:24.510793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a77f76f7-67e8-49ee-9468-e9d45862a629 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 8af7f097-ba99-4637-ad23-ab81689411f8 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:24.511837 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a77f76f7-67e8-49ee-9468-e9d45862a629 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/8af7f097-ba99-4637-ad23-ab81689411f8" status: 404 len: 111 microversion: 2.1 time: 0.022518 Oct 15 21:05:24.515913 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2478/3592] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:24 2022] GET /compute/v2.1/servers/8af7f097-ba99-4637-ad23-ab81689411f8 => generated 111 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:24.522505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:24.522717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:24.543154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.543628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.544078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.544860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.545619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.546271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.575806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:05:24.675030 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b424a36-1d37-49ec-a37b-783f854dee2f tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 204 len: 0 microversion: 2.1 time: 0.156609 Oct 15 21:05:24.676261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2479/3593] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:05:24 2022] DELETE /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:05:24.685148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:24.686495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:24.695761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.696344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.696875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.697805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:24.698753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:24.699196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:24.840498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:24.849234 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98976666-a6cf-4739-b7a9-7c4576ed3609 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.170793 Oct 15 21:05:24.849846 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2480/3594] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:24 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:24.855405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestBootFromVolume-server-445733848", "imageRef": "", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "9024abaa-657f-4308-9317-7b3544a3a128"}], "security_groups": [{"name": "tempest-securitygroup--29540497"}], "key_name": "tempest-keypair-2028445312", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "block_device_mapping_v2": [{"uuid": "ddb4ad67-f927-43a9-a415-d020e04d5095", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "OS-DCF:diskConfig": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:24.891194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=22 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:24.978886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] validate_networks() for [('9024abaa-657f-4308-9317-7b3544a3a128', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:05:25.391643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=23 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:25.394900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Getting quotas for project 87739141ec764f6ca347224117dc45c2. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:25.400304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Getting quotas for user 86129e282ee8469eba032a84e9a55096 and project 87739141ec764f6ca347224117dc45c2. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:25.413439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:05:25.414067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:05:25.414554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:05:25.414966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:05:25.415383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:05:25.423578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:25.424612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:25.425088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:25.425804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:25.426258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:25.426684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:25.443235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Getting quotas for project 87739141ec764f6ca347224117dc45c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:25.449776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Getting quotas for user 86129e282ee8469eba032a84e9a55096 and project 87739141ec764f6ca347224117dc45c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:25.511487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:05:25.513824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ddb4ad67-f927-43a9-a415-d020e04d5095',volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:05:25.668008 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f9e7573-d421-4fd0-8d94-ff41b1e57d8d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.816993 Oct 15 21:05:25.669097 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2481/3595] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:24 2022] POST /compute/v2.1/servers => generated 405 bytes in 818 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:05:25.674115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:25.674858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:25.681468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:25.681965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:25.682412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:25.683125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:25.683595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:25.684026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:25.904570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:25.913274 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f905ca37-7e92-4222-8a3b-e123e6cffe4b tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1518 microversion: 2.1 time: 0.242818 Oct 15 21:05:25.930921 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2482/3596] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:25 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1518 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:25.936547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:25.937463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:25.944495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:25.945102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:25.945597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:25.946313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:25.946771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:25.947487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:26.134550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:26.137933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:26.154574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72f13b92-a07f-413f-8fa3-60f7a2164bb1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.222136 Oct 15 21:05:26.165128 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2483/3597] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:25 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:26.174025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5975b7d0-8b0d-4ac5-ae44-456a95966ad9 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:26.181270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5975b7d0-8b0d-4ac5-ae44-456a95966ad9 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:26.260698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5975b7d0-8b0d-4ac5-ae44-456a95966ad9 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:26.273680 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5975b7d0-8b0d-4ac5-ae44-456a95966ad9 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1121 microversion: 2.1 time: 0.106279 Oct 15 21:05:26.275546 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2484/3598] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:26 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1121 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:26.338206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc003d86-2bd4-4ad8-8d99-079eed659fc4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1430506002", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "accessIPv4": "1.1.1.1.1.1", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc003d86-2bd4-4ad8-8d99-079eed659fc4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 238 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1430506002", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "accessIPv4": "1.1.1.1.1.1", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1430506002', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'accessIPv4': '1.1.1.1.1.1', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 263, in check Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = func(instance) Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/_format.py", line 253, in is_ipv4 Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return bool(ipaddress.IPv4Address(instance)) Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1257, in __init__ Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._ip = self._ip_int_from_string(addr_str) Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1144, in _ip_int_from_string Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise AddressValueError("Expected 4 octets in %r" % ip_str) Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ipaddress.AddressValueError: Expected 4 octets in '1.1.1.1.1.1' Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '1.1.1.1.1.1' is not a 'ipv4' Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['server']['properties']['accessIPv4']: Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'ipv4', 'type': 'string'} Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '1.1.1.1.1.1' Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.360005 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Oct 15 21:05:26.366198 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:26.371586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc003d86-2bd4-4ad8-8d99-079eed659fc4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:26.373900 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc003d86-2bd4-4ad8-8d99-079eed659fc4 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.038655 Oct 15 21:05:26.375199 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2485/3599] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:26 2022] POST /compute/v2.1/servers => generated 139 bytes in 42 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 15 21:05:26.389536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:26.390606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:26.398777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:26.399385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:26.399832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:26.400646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:26.401831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:26.403609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:27.226663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:27.236984 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a59c4da1-772b-499e-8f8f-0caf79e45f2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.854954 Oct 15 21:05:27.238273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2486/3600] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:26 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 858 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:27.243915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:27.244961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:27.253186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:27.254033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:27.254705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:27.255648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:27.256362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:27.257437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:27.738351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:27.747516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47242bd8-6f42-415f-8525-f4d5c18ed166 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1518 microversion: 2.1 time: 0.507718 Oct 15 21:05:27.748229 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2487/3601] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:27 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1518 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:27.753825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:27.754885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:27.764428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:27.765259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:27.765986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:27.766925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:27.768324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:27.769013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.202722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:28.222058 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43d20266-826e-47ab-841b-c45e46caa82a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.472479 Oct 15 21:05:28.222645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2488/3602] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:27 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:28.227222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:28.227927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:28.249466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.249952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.250821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.251177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.251661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.252189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.405364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:28.420764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3614f636-4e3b-4ddd-a325-547ab9eb2175 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1237 microversion: 2.1 time: 0.195206 Oct 15 21:05:28.421643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2489/3603] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:28 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1237 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:28.428348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86fc2948-b7c6-4299-9ebe-2a9066d731d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1729745403", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "accessIPv6": "notvalid", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86fc2948-b7c6-4299-9ebe-2a9066d731d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 235 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1729745403", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "accessIPv6": "notvalid", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1729745403', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'accessIPv6': 'notvalid', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 263, in check Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = func(instance) Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/_format.py", line 260, in is_ipv6 Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi address = ipaddress.IPv6Address(instance) Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1844, in __init__ Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._ip = self._ip_int_from_string(addr_str) Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1581, in _ip_int_from_string Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise AddressValueError(msg) Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi ipaddress.AddressValueError: At least 3 parts expected in 'notvalid' Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'notvalid' is not a 'ipv6' Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['server']['properties']['accessIPv6']: Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'ipv6', 'type': 'string'} Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'notvalid' Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.433750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Oct 15 21:05:28.439140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:28.442975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86fc2948-b7c6-4299-9ebe-2a9066d731d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:28.443850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86fc2948-b7c6-4299-9ebe-2a9066d731d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 133 microversion: 2.1 time: 0.020933 Oct 15 21:05:28.444602 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2490/3604] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:28 2022] POST /compute/v2.1/servers => generated 133 bytes in 22 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:05:28.449528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-76643df6-6a2f-44a4-afea-ba30a5e11857 req-ce61222b-8cb6-43f1-b280-d8a7da657ca2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ecdc3ab5-4d71-4604-9dba-bc9338bfafe2", "name": "network-vif-deleted", "tag": "3fb330a2-3034-4b0f-82fa-330b45824727"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:28.462867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-76643df6-6a2f-44a4-afea-ba30a5e11857 req-ce61222b-8cb6-43f1-b280-d8a7da657ca2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.463356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-76643df6-6a2f-44a4-afea-ba30a5e11857 req-ce61222b-8cb6-43f1-b280-d8a7da657ca2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.463801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-76643df6-6a2f-44a4-afea-ba30a5e11857 req-ce61222b-8cb6-43f1-b280-d8a7da657ca2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.476461 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-76643df6-6a2f-44a4-afea-ba30a5e11857 req-ce61222b-8cb6-43f1-b280-d8a7da657ca2 service nova] Creating event network-vif-deleted:3fb330a2-3034-4b0f-82fa-330b45824727 for instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 on nch1-master-1 Oct 15 21:05:28.481787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-76643df6-6a2f-44a4-afea-ba30a5e11857 req-ce61222b-8cb6-43f1-b280-d8a7da657ca2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035602 Oct 15 21:05:28.482439 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2491/3605] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:28.488160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:28.488943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:28.491156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:28.497964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.498449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.498895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.499826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.500110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.500545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.797091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:28.806470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6b044a6-22ea-4fa6-82a5-bb4cc319549c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.322435 Oct 15 21:05:28.813907 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2492/3606] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:28 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:28.821594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:28.822208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:28.830493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.831100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.831700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.832237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.832952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.833625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.872083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:28.872679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:28.873227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:28.873758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lazy-loading 'fault' on Instance uuid ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:05:28.973893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '124511010d044f8fb07dc09e7d754303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f20ae5c8217d412289d006ab9e01e7f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:28.983720 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef46d131-d5c3-4025-8f71-8e90a51eada4 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 200 len: 1311 microversion: 2.1 time: 0.168643 Oct 15 21:05:28.985451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2493/3607] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:28 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 1311 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:28.998434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:28.999350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:29.006481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:29.007207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:29.007656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:29.012873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:29.013389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:29.013985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:29.363130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:29.373924 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b0f70f3-c169-4baa-be5b-10538a8a8146 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.387599 Oct 15 21:05:29.374513 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2494/3608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:28 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:29.398995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a906ae24-32c0-4183-bc24-8b3ad9859c86 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:29.399649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a906ae24-32c0-4183-bc24-8b3ad9859c86 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5d4ba0b8-eac2-4416-9316-17a80cea661e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:29.408251 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a906ae24-32c0-4183-bc24-8b3ad9859c86 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 5d4ba0b8-eac2-4416-9316-17a80cea661e could not be found. Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a906ae24-32c0-4183-bc24-8b3ad9859c86 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5d4ba0b8-eac2-4416-9316-17a80cea661e/action HTTP/1.1 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 13 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: {"pause": {}} Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 5d4ba0b8-eac2-4416-9316-17a80cea661e could not be found. Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5d4ba0b8-eac2-4416-9316-17a80cea661e', 'body': {'pause': {}}} Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d4ba0b8-eac2-4416-9316-17a80cea661e could not be found. Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 36, in _pause Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, ctxt, id) Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d4ba0b8-eac2-4416-9316-17a80cea661e could not be found. Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:29.417492 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:29.422812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:29.422812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:29.422812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d4ba0b8-eac2-4416-9316-17a80cea661e could not be found. Oct 15 21:05:29.422812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:29.422812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a906ae24-32c0-4183-bc24-8b3ad9859c86 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 5d4ba0b8-eac2-4416-9316-17a80cea661e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:29.423816 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a906ae24-32c0-4183-bc24-8b3ad9859c86 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5d4ba0b8-eac2-4416-9316-17a80cea661e/action" status: 404 len: 111 microversion: 2.1 time: 0.027781 Oct 15 21:05:29.424846 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2495/3609] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:29 2022] POST /compute/v2.1/servers/5d4ba0b8-eac2-4416-9316-17a80cea661e/action => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:05:29.438182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:29.438954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:29.449470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:29.450066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:29.450585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:29.451551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:29.452121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:29.452631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.112474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:30.120647 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-168f36b8-27d3-4444-86ee-5198c860c1d6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.686502 Oct 15 21:05:30.121253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2496/3610] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:29 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:30.144429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:30.145173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:30.151571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.152069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.152513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.153255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.153723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.154161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.473293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:30.481139 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ae7e146-8bb0-4138-b075-861490667fec tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.343010 Oct 15 21:05:30.481719 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2497/3611] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:30 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:30.486657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:30.487487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] [instance: ecdc3ab5-4d71-4604-9dba-bc9338bfafe2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:30.493659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.494157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.494600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.495483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.496240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.496681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.512324 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] HTTP exception thrown: Instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 could not be found. Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Request method failure captured: Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 HTTP/1.1 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyArzmlCEYIPl4mxNsPOXxe_PqEEzxxFXLVeMbQusSAwsVeVMXpvcatarH-XlZd_FAGoRdCsVCQKQ318ISD1ucjN7o_rNjFTT0eze8tjZz8LnsfpZMKRMM3hpdUGmXuctiDXxvaWFrPml9AqktXxMti2ILG2-crrbDdTM_LQ2woVKOA Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f20ae5c8217d412289d006ab9e01e7f4 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f20ae5c8217d412289d006ab9e01e7f4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f20ae5c8217d412289d006ab9e01e7f4 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-556029331 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ListServerFiltersTestJSON-556029331-project Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 124511010d044f8fb07dc09e7d754303 Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ListServerFiltersTestJSON-556029331-project Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 could not be found. Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ecdc3ab5-4d71-4604-9dba-bc9338bfafe2'} Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 could not be found. Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 could not be found. Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:30.514317 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 could not be found. Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] Returning 404 to user: Instance ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e618b71b-3c5c-4816-a6c9-943c53f8c249 tempest-ListServerFiltersTestJSON-556029331 tempest-ListServerFiltersTestJSON-556029331-project] 10.222.0.16 "GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2" status: 404 len: 111 microversion: 2.1 time: 0.032670 Oct 15 21:05:30.520457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2498/3612] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:30 2022] GET /compute/v2.1/servers/ecdc3ab5-4d71-4604-9dba-bc9338bfafe2 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:30.521890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:30.521890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:30.528143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.528870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.529406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.530051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.530680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.531311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.875235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:30.886711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6cf3ac7-c031-4544-b924-537d9ed75127 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.369502 Oct 15 21:05:30.890212 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2499/3613] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:30 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:30.913908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:30.914801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:30.923084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.924974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.926636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:30.927696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:30.928344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:30.929271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:31.075544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6dcba57-b598-4860-9480-6546c4c593f2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 202 len: 0 microversion: 2.1 time: 0.166939 Oct 15 21:05:31.079083 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2500/3614] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:30 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:31.090078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:31.106278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:31.124963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:31.128232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:31.128820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:31.129630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:31.131663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:31.131663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:31.582516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:31.592784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3bb7af0-4f65-4c51-b490-ec56748b722e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1510 microversion: 2.1 time: 0.511557 Oct 15 21:05:31.593879 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2501/3615] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:31 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1510 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:31.600689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:31.601367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:31.608002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:31.608497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:31.609000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:31.609830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:31.610292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:31.610771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:32.009715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:32.025268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60e86e2c-659f-49bf-90ca-0f6215b39077 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.430002 Oct 15 21:05:32.027268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2502/3616] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:31 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:32.042853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-dd6807df-f6c7-4752-a7ac-7e0fc569a0f2 req-b65d2d12-69a5-41fb-ac5e-eb6f086caf3b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7e98faf5-e97a-430b-9d80-46bfdc6f2e61", "tag": "c5ab0d40-2dde-4427-834d-04796767878e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:32.058503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd6807df-f6c7-4752-a7ac-7e0fc569a0f2 req-b65d2d12-69a5-41fb-ac5e-eb6f086caf3b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:32.059260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd6807df-f6c7-4752-a7ac-7e0fc569a0f2 req-b65d2d12-69a5-41fb-ac5e-eb6f086caf3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:32.060288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dd6807df-f6c7-4752-a7ac-7e0fc569a0f2 req-b65d2d12-69a5-41fb-ac5e-eb6f086caf3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:32.078161 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-dd6807df-f6c7-4752-a7ac-7e0fc569a0f2 req-b65d2d12-69a5-41fb-ac5e-eb6f086caf3b service nova] Creating event network-changed:c5ab0d40-2dde-4427-834d-04796767878e for instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 on nch2-master-1 Oct 15 21:05:32.085393 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-dd6807df-f6c7-4752-a7ac-7e0fc569a0f2 req-b65d2d12-69a5-41fb-ac5e-eb6f086caf3b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055877 Oct 15 21:05:32.086534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2503/3617] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:32.622272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:32.623060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:32.629883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:32.630414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:32.630860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:32.632228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:32.632761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:32.633239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:32.927985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:32.937351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8ccf2538-8270-4ebb-aef0-d5becd4360c1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1490 microversion: 2.1 time: 0.319516 Oct 15 21:05:32.938060 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2504/3618] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:32 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1490 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:32.964513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:32.965615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:32.978537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:32.979040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:32.980041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:32.981337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:32.981796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:32.982226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:33.018575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:33.027585 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'pause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state paused Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 13 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: {"pause": {}} Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'pause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state paused Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'pause': {}}} Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 41, in _pause Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.pause(ctxt, server) Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state paused. Cannot pause while the instance is in this state. Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 45, in _pause Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'pause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state paused Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:33.029761 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'pause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state paused Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'pause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state paused {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9835a91e-04cb-4341-bdb9-b80254a3d004 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 143 microversion: 2.1 time: 0.071567 Oct 15 21:05:33.036131 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2505/3619] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:32 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 143 bytes in 77 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 15 21:05:33.041726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:33.042261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:33.049802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:33.050360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:33.050793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:33.051496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:33.051953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:33.052376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:33.159491 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a006313-ea6a-4301-b818-a9fe6eb6c58e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 202 len: 0 microversion: 2.1 time: 0.122030 Oct 15 21:05:33.160189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2506/3620] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:33 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:33.177820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:33.178610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:33.186078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:33.186822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:33.187473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:33.188265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:33.188731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:33.208075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:33.427541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:33.450508 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecb93d0c-b80e-4dde-aced-ab6db217c19e tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.288404 Oct 15 21:05:33.451113 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2507/3621] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:33 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:33.465424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:33.466531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:33.474264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:33.475033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:33.475635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:33.476575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:33.477224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:33.477805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.230060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:34.240320 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d42c0292-40e0-4eca-8961-15458883915e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.787187 Oct 15 21:05:34.240908 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2508/3622] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:33 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:34.271190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:34.272641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:34.283117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.283622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.284068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.284787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.285297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.285731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.586579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:34.598675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21abcd2c-e89f-4d08-b1ad-8a540cc01a57 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.331489 Oct 15 21:05:34.599826 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2509/3623] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:34 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:34.616000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:34.616623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:34.635120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.635855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.636564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.638047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.639319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.639925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.855074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:34.865911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57c3b0e0-3a97-473f-8d14-daf3d1f3c6f7 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.264038 Oct 15 21:05:34.866512 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2510/3624] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:34 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:34.872300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:34.875285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:34.882875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.883881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.884530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.885519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.886257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.886759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.902472 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525/action HTTP/1.1 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 28 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: {"reboot": {"type": "SOFT"}} Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '14e27a99-2936-474b-85fd-21419ba8f525', 'body': {'reboot': {'type': 'SOFT'}}} Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1031, in _action_reboot Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:34.904879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-89a480cb-94d4-487b-b302-d2feea672e0e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525/action" status: 404 len: 111 microversion: 2.1 time: 0.038348 Oct 15 21:05:34.909631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2511/3625] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:34 2022] POST /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525/action => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:34.917077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:34.917775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:34.925217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.925812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.926324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:34.927112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:34.927646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:34.928146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:35.117466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:35.127420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c887607-d6ef-4bb0-8dd3-12ee1df74711 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.214425 Oct 15 21:05:35.143097 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2512/3626] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:34 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:35.401993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a7efb6a-805a-422c-b59e-4a5aae97c875 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Action: 'create', calling method: .version_select at 0x7fe7546ae5e0>, body: {"keypair": {"name": "tempest-keypair-258895278", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFhmRjveK34TvSZH/A9LTuXnMNYYeFMN6YuunytDqICY/WZbWMO2fblcq5MkGCHhmiLyNzvC06IeULUaI2m6nt6AL2ppZ0thaHgokhEhcw7uUmd54KeM7+9EHRZIhx8Ayw=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:35.426121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1a7efb6a-805a-422c-b59e-4a5aae97c875 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Getting quotas for project b2c267ee2a9d4191a6e9de6e7bdee449. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:35.432846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1a7efb6a-805a-422c-b59e-4a5aae97c875 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Getting quotas for user be3724ed7cfa41cabc1c1a5e8fb1f218 and project b2c267ee2a9d4191a6e9de6e7bdee449. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:35.437280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:35.498910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a7efb6a-805a-422c-b59e-4a5aae97c875 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.354098 Oct 15 21:05:35.500700 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2513/3627] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:05:35 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:35.520608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:35.522026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:35.528652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:35.529437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:35.530293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:35.531083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:35.531692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:35.532304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:35.742633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:35.752854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb281b2a-ba2b-4f7b-8cfc-8ee38e55884f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.243002 Oct 15 21:05:35.753479 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2514/3628] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:35 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:35.777062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01457f66-c898-4aa0-8bb0-f56c939b8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:35.779794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01457f66-c898-4aa0-8bb0-f56c939b8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 61f318a1-3dcf-4494-b5c4-173ca763abc1] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:35.786555 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-01457f66-c898-4aa0-8bb0-f56c939b8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 61f318a1-3dcf-4494-b5c4-173ca763abc1 could not be found. Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01457f66-c898-4aa0-8bb0-f56c939b8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/61f318a1-3dcf-4494-b5c4-173ca763abc1/action HTTP/1.1 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 28 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: {"reboot": {"type": "SOFT"}} Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 61f318a1-3dcf-4494-b5c4-173ca763abc1 could not be found. Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '61f318a1-3dcf-4494-b5c4-173ca763abc1', 'body': {'reboot': {'type': 'SOFT'}}} Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 61f318a1-3dcf-4494-b5c4-173ca763abc1 could not be found. Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1031, in _action_reboot Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 61f318a1-3dcf-4494-b5c4-173ca763abc1 could not be found. Oct 15 21:05:35.788442 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 61f318a1-3dcf-4494-b5c4-173ca763abc1 could not be found. Oct 15 21:05:35.793856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:35.795502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01457f66-c898-4aa0-8bb0-f56c939b8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 61f318a1-3dcf-4494-b5c4-173ca763abc1 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:35.796569 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01457f66-c898-4aa0-8bb0-f56c939b8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/61f318a1-3dcf-4494-b5c4-173ca763abc1/action" status: 404 len: 111 microversion: 2.1 time: 0.023969 Oct 15 21:05:35.802855 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2515/3629] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:35 2022] POST /compute/v2.1/servers/61f318a1-3dcf-4494-b5c4-173ca763abc1/action => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:35.808271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:35.809196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:35.817302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:35.818152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:35.818775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:35.820252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:35.820906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:35.821706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.023885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:36.035209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-574f114a-0f4a-48f3-ada2-dfb1f069c6c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.230865 Oct 15 21:05:36.050500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2516/3630] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:35 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:36.064899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:36.065801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:36.073172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.073649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.074423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.075329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.079310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.080002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.320289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:36.346914 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a1b13a36-c9d6-487c-9b88-52d4b07d9529 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.294921 Oct 15 21:05:36.347544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2517/3631] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:36 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:36.359134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:36.360484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:36.377674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.379946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.380693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.382729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.383401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.386312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.662567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:36.673015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38c61527-c7bc-44c7-a462-9c7c618aa3ec tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.324326 Oct 15 21:05:36.674072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2518/3632] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:36 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:36.699372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:36.702999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:36.711289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.712184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.712806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.713702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.714350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.714941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.744717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525/action HTTP/1.1 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 65 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '14e27a99-2936-474b-85fd-21419ba8f525', 'body': {'rebuild': {'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9'}}} Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Oct 15 21:05:36.746765 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1160, in _action_rebuild Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:05:36.752487 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:36.755605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:36.755605 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4513cfe0-68ef-4f7a-a29e-6719882bd472 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525/action" status: 404 len: 111 microversion: 2.1 time: 0.060163 Oct 15 21:05:36.757987 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2519/3633] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:36 2022] POST /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525/action => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:05:36.769055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:36.770942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:36.778022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.785451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.786153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:36.787223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:36.791553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:36.793150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:37.326308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:37.354043 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0702b82-ac8e-4495-850f-c689a41e367e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.591602 Oct 15 21:05:37.354666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2520/3634] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:36 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:37.376942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:37.378195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:37.390385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:37.394578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:37.395197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:37.396036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:37.398804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:37.399364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:37.787071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:37.799741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7bbccf64-070d-4498-96af-e4cbd4bc7d14 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.427328 Oct 15 21:05:37.801335 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2521/3635] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:37 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:37.819549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:37.820481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:37.836294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:37.837086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:37.837734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:37.838662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:37.842470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:37.843182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:38.275792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:38.291662 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5bfc7324-bc34-4e3d-9d88-3f0d264a3829 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.486074 Oct 15 21:05:38.305101 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2522/3636] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:37 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:38.326522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86d23dfb-1cf1-424e-bed1-6c8f01330a01 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:38.329609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86d23dfb-1cf1-424e-bed1-6c8f01330a01 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: e8962e47-8257-4301-b372-463dbcb6c6d1] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:38.344545 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-86d23dfb-1cf1-424e-bed1-6c8f01330a01 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance e8962e47-8257-4301-b372-463dbcb6c6d1 could not be found. Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86d23dfb-1cf1-424e-bed1-6c8f01330a01 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/e8962e47-8257-4301-b372-463dbcb6c6d1/action HTTP/1.1 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 65 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e8962e47-8257-4301-b372-463dbcb6c6d1 could not be found. Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e8962e47-8257-4301-b372-463dbcb6c6d1', 'body': {'rebuild': {'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9'}}} Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e8962e47-8257-4301-b372-463dbcb6c6d1 could not be found. Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1160, in _action_rebuild Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Oct 15 21:05:38.347087 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e8962e47-8257-4301-b372-463dbcb6c6d1 could not be found. Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e8962e47-8257-4301-b372-463dbcb6c6d1 could not be found. Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86d23dfb-1cf1-424e-bed1-6c8f01330a01 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance e8962e47-8257-4301-b372-463dbcb6c6d1 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:38.352790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86d23dfb-1cf1-424e-bed1-6c8f01330a01 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/e8962e47-8257-4301-b372-463dbcb6c6d1/action" status: 404 len: 111 microversion: 2.1 time: 0.027350 Oct 15 21:05:38.355304 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2523/3637] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:38 2022] POST /compute/v2.1/servers/e8962e47-8257-4301-b372-463dbcb6c6d1/action => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:05:38.358745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:38.359435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:38.374652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:38.375419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:38.377741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:38.379513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:38.380267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:38.380969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:38.652380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:38.663307 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0ae3a4c-0d78-4af5-9f06-4d62855e2898 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.308234 Oct 15 21:05:38.664550 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2524/3638] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:38 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:38.682584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Action: 'create', calling method: >, body: {"server": {"name": "guest-instance-1.domain.com", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "accessIPv4": "2.2.2.2", "networks": [{"uuid": "93739fba-5ffb-4fa6-ab92-a4ce38458a0e"}], "security_groups": [{"name": "tempest-securitygroup--1997693440"}], "key_name": "tempest-keypair-258895278", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:38.910621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] validate_networks() for [('93739fba-5ffb-4fa6-ab92-a4ce38458a0e', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:05:39.600095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:05:39.600528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:05:39.601004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:05:39.601458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:05:39.601875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:05:39.613367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:39.614144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:39.614771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:39.615968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:39.616684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:39.617320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:39.638572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Getting quotas for project b2c267ee2a9d4191a6e9de6e7bdee449. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:39.644452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Getting quotas for user be3724ed7cfa41cabc1c1a5e8fb1f218 and project b2c267ee2a9d4191a6e9de6e7bdee449. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:39.660481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:05:39.663217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:05:39.717013 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdfcbaa8-ba11-4520-8e9d-31a8d716dc5f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.050974 Oct 15 21:05:39.717830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2525/3639] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:38 2022] POST /compute/v2.1/servers => generated 409 bytes in 1053 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:05:39.732981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:39.733665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:39.745977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:39.746674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:39.747292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:39.749208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:39.750641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:39.753797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:39.792812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:40.084936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:40.094153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b645d82f-c80f-46c5-81c1-cb69066f560a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.374982 Oct 15 21:05:40.095229 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2526/3640] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:39 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:40.100419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:40.103112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:40.111554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:40.112170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:40.112906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:40.114007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:40.114501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:40.114983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:40.310985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:40.320648 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b98eccf2-66f4-4d3e-9594-2482dfb8ea04 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.224103 Oct 15 21:05:40.333992 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2527/3641] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:40 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:40.345951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba92067a-02f3-4455-a622-5ac3dbf407e3 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:40.346974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba92067a-02f3-4455-a622-5ac3dbf407e3 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:40.349273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:40.416209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba92067a-02f3-4455-a622-5ac3dbf407e3 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:40.427469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba92067a-02f3-4455-a622-5ac3dbf407e3 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1225 microversion: 2.1 time: 0.090580 Oct 15 21:05:40.441074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2528/3642] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:40 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:40.441074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20fa841b-a3d7-4605-80e3-356eb5446a42 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"restore": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:40.441074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-20fa841b-a3d7-4605-80e3-356eb5446a42 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: fa188a73-e21c-4d2b-b027-559d657bcd8f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:40.441074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-20fa841b-a3d7-4605-80e3-356eb5446a42 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance fa188a73-e21c-4d2b-b027-559d657bcd8f could not be found. Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20fa841b-a3d7-4605-80e3-356eb5446a42 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/fa188a73-e21c-4d2b-b027-559d657bcd8f/action HTTP/1.1 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: {"restore": {}} Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance fa188a73-e21c-4d2b-b027-559d657bcd8f could not be found. Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'fa188a73-e21c-4d2b-b027-559d657bcd8f', 'body': {'restore': {}}} Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa188a73-e21c-4d2b-b027-559d657bcd8f could not be found. Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 38, in _restore Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa188a73-e21c-4d2b-b027-559d657bcd8f could not be found. Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:40.442767 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:40.448305 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:40.448305 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:40.448305 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa188a73-e21c-4d2b-b027-559d657bcd8f could not be found. Oct 15 21:05:40.448305 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:40.449668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20fa841b-a3d7-4605-80e3-356eb5446a42 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance fa188a73-e21c-4d2b-b027-559d657bcd8f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:40.450719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20fa841b-a3d7-4605-80e3-356eb5446a42 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/fa188a73-e21c-4d2b-b027-559d657bcd8f/action" status: 404 len: 111 microversion: 2.1 time: 0.021055 Oct 15 21:05:40.452025 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2529/3643] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:40 2022] POST /compute/v2.1/servers/fa188a73-e21c-4d2b-b027-559d657bcd8f/action => generated 111 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:40.522558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-730dd264-de7c-4a26-ab00-9d8be184f8a5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'create', calling method: .version_select at 0x7fe75845b670>, body: {"keypair": {"name": "tempest-keypair-2116871727", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCREDxhRZGYiaryeUcyf5hUG1v+p3Ihaa2FDsSiPOxwMzXe3SbEkwZEXTNfIKdebhWf0nnmPL543kZC1ypaKtKTdk5x7e8Do3tsB7fLWyd+0bZsd6BWFlOEI0xsaUB48Zg=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:40.542033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-730dd264-de7c-4a26-ab00-9d8be184f8a5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for project 38b9244bbab846b7baf42985cebc72c2. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:40.556939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-730dd264-de7c-4a26-ab00-9d8be184f8a5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for user 1aadd87b97c247cb95321664ba0c32a9 and project 38b9244bbab846b7baf42985cebc72c2. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:40.614289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-730dd264-de7c-4a26-ab00-9d8be184f8a5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.160931 Oct 15 21:05:40.617302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2530/3644] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:05:40 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:40.622068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:40.622757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:40.642809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:40.643724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:40.644359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:40.645344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:40.645979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:40.646599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:41.214621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:41.224416 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c725e1c5-2285-4010-8fff-87b9f6d63db6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.605818 Oct 15 21:05:41.241917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2531/3645] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:40 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:41.248593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-78c10f34-5759-45e3-88cb-7bf70b63b447 req-0f00d73e-60d1-42ff-8833-cf974e542923 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e98faf5-e97a-430b-9d80-46bfdc6f2e61", "name": "network-vif-plugged", "status": "completed", "tag": "c5ab0d40-2dde-4427-834d-04796767878e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:41.259986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-78c10f34-5759-45e3-88cb-7bf70b63b447 req-0f00d73e-60d1-42ff-8833-cf974e542923 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:41.260757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-78c10f34-5759-45e3-88cb-7bf70b63b447 req-0f00d73e-60d1-42ff-8833-cf974e542923 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:41.261214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-78c10f34-5759-45e3-88cb-7bf70b63b447 req-0f00d73e-60d1-42ff-8833-cf974e542923 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:41.274972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-78c10f34-5759-45e3-88cb-7bf70b63b447 req-0f00d73e-60d1-42ff-8833-cf974e542923 service nova] Creating event network-vif-plugged:c5ab0d40-2dde-4427-834d-04796767878e for instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 on nch2-master-1 Oct 15 21:05:41.283409 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-78c10f34-5759-45e3-88cb-7bf70b63b447 req-0f00d73e-60d1-42ff-8833-cf974e542923 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037939 Oct 15 21:05:41.284028 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2532/3646] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:41.289410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:41.290801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:41.299182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:41.299998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:41.300673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:41.301782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:41.302468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:41.303104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:41.690184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:41.699683 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e4b5424-eb44-4ec2-ace7-4bf164788c19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.414423 Oct 15 21:05:41.700271 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2533/3647] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:41 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:41.723085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:41.723876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:41.731342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:41.731828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:41.732268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:41.733136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:41.733620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:41.734254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:41.984188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:41.997792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6c9addc-3676-415d-8a6f-36538e7bd25b tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1291 microversion: 2.1 time: 0.284820 Oct 15 21:05:42.013260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2534/3648] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:41 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1291 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:42.019577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:42.020257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:42.027476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.028279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.029004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.030217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.030534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.031010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.147538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:42.160881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-130c0004-cb27-41a3-957d-430893e93d7e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1283 microversion: 2.1 time: 0.145958 Oct 15 21:05:42.161680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2535/3649] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:42 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1283 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:42.184021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"restore": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:42.187648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:42.218050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.218771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.221416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.223921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.225298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.226816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.261418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'restore' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: {"restore": {}} Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'restore' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'restore': {}}} Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 42, in _restore Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.restore(context, instance) Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state active. Cannot restore while the instance is in this state. Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 46, in _restore Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'restore' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:42.263674 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:42.271284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:42.271284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:42.271284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:42.271284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'restore' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:42.271284 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.273382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'restore' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:42.274411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57780893-0f8e-48b3-bdd8-6d4e9b6925dc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 145 microversion: 2.1 time: 0.111331 Oct 15 21:05:42.275174 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2536/3650] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:42 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 145 bytes in 113 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:05:42.289330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:42.290499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:42.299490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.300146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.300603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.301347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.301854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.302288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.528889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:42.537316 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02ccaf33-01a7-4af7-a2c2-8bac72f1d369 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.257432 Oct 15 21:05:42.537925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2537/3651] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:42 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:42.565824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:42.566642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:42.573625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.574394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.574896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.575674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:42.576199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:42.576712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:42.815279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:42.825017 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7d17230-33fc-48c9-8f21-ce991e76446e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.263244 Oct 15 21:05:42.846259 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2538/3652] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:42 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:42.853655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d1b16b-90fc-4db7-a258-66fbe17be9ad tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:42.854683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46d1b16b-90fc-4db7-a258-66fbe17be9ad tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: d6135965-e677-45ad-a423-f12604cd9aed] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:42.861483 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-46d1b16b-90fc-4db7-a258-66fbe17be9ad tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance d6135965-e677-45ad-a423-f12604cd9aed could not be found. Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d1b16b-90fc-4db7-a258-66fbe17be9ad tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/d6135965-e677-45ad-a423-f12604cd9aed/action HTTP/1.1 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 14 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: {"resume": {}} Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d6135965-e677-45ad-a423-f12604cd9aed could not be found. Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd6135965-e677-45ad-a423-f12604cd9aed', 'body': {'resume': {}}} Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6135965-e677-45ad-a423-f12604cd9aed could not be found. Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 55, in _resume Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, id) Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6135965-e677-45ad-a423-f12604cd9aed could not be found. Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:42.868554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:42.874166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:42.874166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:42.874166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6135965-e677-45ad-a423-f12604cd9aed could not be found. Oct 15 21:05:42.874166 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:42.874519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d1b16b-90fc-4db7-a258-66fbe17be9ad tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance d6135965-e677-45ad-a423-f12604cd9aed could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:42.876020 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46d1b16b-90fc-4db7-a258-66fbe17be9ad tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/d6135965-e677-45ad-a423-f12604cd9aed/action" status: 404 len: 111 microversion: 2.1 time: 0.026900 Oct 15 21:05:42.877512 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2539/3653] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:42 2022] POST /compute/v2.1/servers/d6135965-e677-45ad-a423-f12604cd9aed/action => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:05:42.885171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-2017111320", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd"}], "security_groups": [{"name": "tempest-securitygroup--1613255345"}], "key_name": "tempest-keypair-2116871727", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:43.089834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] validate_networks() for [('c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:05:43.800779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:05:43.802910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:05:43.803685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:05:43.804760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:05:43.805382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:05:43.823316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:43.825280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:43.825909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:43.827069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:43.828267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:43.828905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:43.851338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:05:43.857575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for user 1aadd87b97c247cb95321664ba0c32a9 and project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:05:43.873136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:05:43.876119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:05:43.974800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18afc0f9-bf37-4184-8b6a-df644db946cf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.095070 Oct 15 21:05:43.984935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2540/3654] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:42 2022] POST /compute/v2.1/servers => generated 409 bytes in 1107 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:05:43.993888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:43.994521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:44.006233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.007293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.007293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.007658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.008124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.008553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.279766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:44.290409 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d0f8471-feee-491b-9767-3702214f9598 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.304354 Oct 15 21:05:44.301640 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2541/3655] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:43 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:44.317713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:44.318589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:44.325393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.326039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.326559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.327583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.328285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.328951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.561567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:44.574042 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-11abf61b-a886-4d52-ac37-6a5d2219e2f6 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1481 microversion: 2.1 time: 0.270694 Oct 15 21:05:44.574643 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2542/3656] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:44 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1481 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:44.589351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:44.589351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:44.594832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.595456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.596259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.596964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.597536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.597967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.747802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:44.757654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-23408f11-b48c-49ef-93f5-e3914834f907 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1351 microversion: 2.1 time: 0.181713 Oct 15 21:05:44.758182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2543/3657] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:44 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1351 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:44.762737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1c102c28-2f30-4f94-a771-002f86e25d40 req-6f286ba8-3bfe-4368-ab41-945942a2363d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "290a5c59-7d47-4799-9890-1b887bebafeb", "tag": "d307cb43-3fcb-4d77-b2d3-4f42c7027f97"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:44.774099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1c102c28-2f30-4f94-a771-002f86e25d40 req-6f286ba8-3bfe-4368-ab41-945942a2363d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.774894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1c102c28-2f30-4f94-a771-002f86e25d40 req-6f286ba8-3bfe-4368-ab41-945942a2363d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.775346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1c102c28-2f30-4f94-a771-002f86e25d40 req-6f286ba8-3bfe-4368-ab41-945942a2363d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.788868 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1c102c28-2f30-4f94-a771-002f86e25d40 req-6f286ba8-3bfe-4368-ab41-945942a2363d service nova] Creating event network-changed:d307cb43-3fcb-4d77-b2d3-4f42c7027f97 for instance 290a5c59-7d47-4799-9890-1b887bebafeb on nch1-master-1 Oct 15 21:05:44.793951 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1c102c28-2f30-4f94-a771-002f86e25d40 req-6f286ba8-3bfe-4368-ab41-945942a2363d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034560 Oct 15 21:05:44.794571 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2544/3658] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:44.800599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:44.801400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:44.807946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.808443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.809110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.810068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.810558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.811033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.882871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:44.892411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38aff4fc-4c9d-4551-beef-8bcc45d6cd31 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1291 microversion: 2.1 time: 0.095752 Oct 15 21:05:44.893205 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2545/3659] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:44 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1291 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:44.898361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:44.899088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:44.912681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.913356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.913886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:44.914669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:44.915262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:44.915760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.061154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:45.069812 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63f926c4-3532-48f8-99fc-8fbf50b0b9d5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.175636 Oct 15 21:05:45.070396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2546/3660] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:44 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:45.074918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:45.075435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:45.082419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.082922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.083366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.084079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.084541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.084973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.170663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8fcadb2d-90f1-4e58-a357-9c31fabcdcd3 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099163 Oct 15 21:05:45.172091 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2547/3661] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:05:45 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:45.179335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:45.180285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:45.188431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.188965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.189430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.190129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.190591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.191015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.218508 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'resume' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 14 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: {"resume": {}} Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'resume' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'resume': {}}} Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 59, in _resume Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.resume(context, server) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state active. Cannot resume while the instance is in this state. Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 63, in _resume Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'resume' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:45.220579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'resume' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'resume' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b043395-5e28-40aa-aa9f-b6a277d9757a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 144 microversion: 2.1 time: 0.048793 Oct 15 21:05:45.225493 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2548/3662] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:45 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 144 bytes in 50 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:05:45.230500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:45.231103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:45.238190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.238685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.239124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.239871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.240580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.241004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.576388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:45.578918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:45.588901 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7af57f32-dcc7-4213-9f2e-7dfdaa69ed05 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.362060 Oct 15 21:05:45.589516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2549/3663] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:45 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:45.614325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:45.615260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:45.621394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.621884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.622323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:45.623026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:45.623483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:45.623913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:46.397438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:46.405722 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b49a62b2-0371-4c3d-9488-164e32c8b715 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.796205 Oct 15 21:05:46.406341 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2550/3664] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:45 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:46.411968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:46.412804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:46.418770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:46.419258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:46.419786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:46.420518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:46.421003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:46.421477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:46.773275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:46.782091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-daf469d3-b084-4850-a507-10f1ef206f65 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1351 microversion: 2.1 time: 0.374545 Oct 15 21:05:46.793553 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2551/3665] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:46 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1351 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:46.805268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:46.805918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:46.812880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:46.814527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:46.815162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:46.816085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:46.816720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:46.817357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:46.959032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:46.966936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2af4652b-c456-45c5-8249-e195200d772c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1297 microversion: 2.1 time: 0.172223 Oct 15 21:05:46.967516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2552/3666] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:46 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1297 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:46.972913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7170a20-48de-46a4-9a56-f0f6baf8ecb2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'create', calling method: >, body: {"server": {"name": "", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7170a20-48de-46a4-9a56-f0f6baf8ecb2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 160 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e3c96722-97ae-4b06-ab5c-8a523bae70f3"}]}} Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: . '' is too short Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': '', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'networks': [{'uuid': 'e3c96722-97ae-4b06-ab5c-8a523bae70f3'}]}}} Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['server']['properties']['name']: Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '' Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:05:46.977309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7170a20-48de-46a4-9a56-f0f6baf8ecb2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d7170a20-48de-46a4-9a56-f0f6baf8ecb2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 108 microversion: 2.1 time: 0.010459 Oct 15 21:05:46.987261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2553/3667] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:05:46 2022] POST /compute/v2.1/servers => generated 108 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:05:46.988766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:46.988880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:46.997308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:46.997807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:46.998254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:46.998970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:46.999442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:46.999875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:47.390481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:47.398899 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c76de4df-8357-4734-b4ba-707e20b8ce81 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.414661 Oct 15 21:05:47.399507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2554/3668] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:46 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:47.413206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:47.413206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:47.413206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:47.413206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:47.413206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:47.413739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:47.413940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:47.414373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:47.439907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3228ce4-ab73-4a9f-8fc6-7adffd1589dd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/ips" status: 200 len: 146 microversion: 2.1 time: 0.039371 Oct 15 21:05:47.441581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2555/3669] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:05:47 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/ips => generated 146 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:47.445857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:47.446467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:47.454678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:47.455172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:47.455613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:47.456325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:47.456786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:47.457245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:47.729200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:47.737984 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e3ba6b5-5826-44ec-9997-cef841270d20 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.295556 Oct 15 21:05:47.738768 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2556/3670] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:47 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:47.761172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1828d309-62f9-41f9-bf82-3d3103fcccdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:47.761736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1828d309-62f9-41f9-bf82-3d3103fcccdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 6050c7b3-9a8c-424c-be64-e6e7c549695b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:47.767985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-1828d309-62f9-41f9-bf82-3d3103fcccdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 6050c7b3-9a8c-424c-be64-e6e7c549695b could not be found. Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1828d309-62f9-41f9-bf82-3d3103fcccdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/6050c7b3-9a8c-424c-be64-e6e7c549695b/action HTTP/1.1 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 14 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: {"shelve": {}} Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6050c7b3-9a8c-424c-be64-e6e7c549695b could not be found. Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6050c7b3-9a8c-424c-be64-e6e7c549695b', 'body': {'shelve': {}}} Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6050c7b3-9a8c-424c-be64-e6e7c549695b could not be found. Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 44, in _shelve Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6050c7b3-9a8c-424c-be64-e6e7c549695b could not be found. Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:05:47.775870 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6050c7b3-9a8c-424c-be64-e6e7c549695b could not be found. Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1828d309-62f9-41f9-bf82-3d3103fcccdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 6050c7b3-9a8c-424c-be64-e6e7c549695b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1828d309-62f9-41f9-bf82-3d3103fcccdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/6050c7b3-9a8c-424c-be64-e6e7c549695b/action" status: 404 len: 111 microversion: 2.1 time: 0.020122 Oct 15 21:05:47.785679 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2557/3671] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:47 2022] POST /compute/v2.1/servers/6050c7b3-9a8c-424c-be64-e6e7c549695b/action => generated 111 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:05:47.790536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:47.791157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:47.800223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:47.800735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:47.801268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:47.801996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:47.802734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:47.803381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.081925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:48.090208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de92ee72-6eeb-42fc-89b5-390d10efde07 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.304218 Oct 15 21:05:48.091221 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2558/3672] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:47 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:48.097948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:48.098837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:48.114218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.114829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.115644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.116692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.117259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.117760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.383142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:48.393254 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7951cf1b-47ad-41a2-9187-8f0e38bc351e tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1351 microversion: 2.1 time: 0.300365 Oct 15 21:05:48.394093 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2559/3673] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:48 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1351 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:48.398805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:48.399424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:48.405603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.406424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.407049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.408529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.409206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.409813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.668568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:48.680586 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6f8483c4-2eef-4f32-9563-6c4b1cbbec8a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1481 microversion: 2.1 time: 0.285225 Oct 15 21:05:48.682090 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2560/3674] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:48 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1481 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:48.688037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d13eb2ed-e912-442b-aa50-8b8fdf5fffbe req-39b2b152-7c42-4b03-9dfc-146d044100f4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7e98faf5-e97a-430b-9d80-46bfdc6f2e61", "tag": "c5ab0d40-2dde-4427-834d-04796767878e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:48.699244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d13eb2ed-e912-442b-aa50-8b8fdf5fffbe req-39b2b152-7c42-4b03-9dfc-146d044100f4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.700231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d13eb2ed-e912-442b-aa50-8b8fdf5fffbe req-39b2b152-7c42-4b03-9dfc-146d044100f4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.701280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d13eb2ed-e912-442b-aa50-8b8fdf5fffbe req-39b2b152-7c42-4b03-9dfc-146d044100f4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.715994 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d13eb2ed-e912-442b-aa50-8b8fdf5fffbe req-39b2b152-7c42-4b03-9dfc-146d044100f4 service nova] Creating event network-changed:c5ab0d40-2dde-4427-834d-04796767878e for instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 on nch2-master-1 Oct 15 21:05:48.721495 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d13eb2ed-e912-442b-aa50-8b8fdf5fffbe req-39b2b152-7c42-4b03-9dfc-146d044100f4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037661 Oct 15 21:05:48.722415 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2561/3675] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:48.728580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:48.729362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:48.749518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.750025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.750473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:48.751704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:48.752185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:48.752627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:49.377578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:49.385636 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a468643f-70de-4a64-8cd5-039897e33c2a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1365 microversion: 2.1 time: 0.661612 Oct 15 21:05:49.397106 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2562/3676] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:48 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1365 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:49.397106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:49.397106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:49.404498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:49.405179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:49.405523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:49.406284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:49.406751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:49.407968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:49.790226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:49.801707 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-778eca0a-ee1d-45bf-bddc-de06659701c5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.414109 Oct 15 21:05:49.802865 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2563/3677] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:49 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:49.808361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4ba48bd2-e343-473e-a3ce-fe1ae1557086 req-76cf73eb-2964-469b-93f8-1c4e2d13f441 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "290a5c59-7d47-4799-9890-1b887bebafeb", "name": "network-vif-plugged", "status": "completed", "tag": "d307cb43-3fcb-4d77-b2d3-4f42c7027f97"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:49.828049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4ba48bd2-e343-473e-a3ce-fe1ae1557086 req-76cf73eb-2964-469b-93f8-1c4e2d13f441 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:49.828546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4ba48bd2-e343-473e-a3ce-fe1ae1557086 req-76cf73eb-2964-469b-93f8-1c4e2d13f441 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:49.829275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4ba48bd2-e343-473e-a3ce-fe1ae1557086 req-76cf73eb-2964-469b-93f8-1c4e2d13f441 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:49.842928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4ba48bd2-e343-473e-a3ce-fe1ae1557086 req-76cf73eb-2964-469b-93f8-1c4e2d13f441 service nova] Creating event network-vif-plugged:d307cb43-3fcb-4d77-b2d3-4f42c7027f97 for instance 290a5c59-7d47-4799-9890-1b887bebafeb on nch1-master-1 Oct 15 21:05:49.848209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4ba48bd2-e343-473e-a3ce-fe1ae1557086 req-76cf73eb-2964-469b-93f8-1c4e2d13f441 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043863 Oct 15 21:05:49.849387 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2564/3678] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:49.856261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:49.856876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:49.866450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:49.875049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:49.875551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:49.876048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:49.877177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:49.877513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:49.878116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.256845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:50.265257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aecf790b-d53b-4954-8dfd-6f88b9ec5b84 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1351 microversion: 2.1 time: 0.414508 Oct 15 21:05:50.265852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2565/3679] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:49 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1351 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:50.271170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:50.271718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:50.278018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.278551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.279195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.280080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.280721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.281407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.308884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lazy-loading 'flavor' on Instance uuid 5f318670-01e5-4565-8e32-ac7ef73ba33c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:05:50.505514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71983442-0f22-4f92-8cf2-c667e3455bf9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 202 len: 0 microversion: 2.1 time: 0.238213 Oct 15 21:05:50.506410 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2566/3680] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:50 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 0 bytes in 240 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:50.511815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:50.512527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:50.520201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.520699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.521166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.521896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.522345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.522809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.663047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:50.671357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-485e6583-2cc9-4a34-a03d-f429558dbedf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1365 microversion: 2.1 time: 0.163703 Oct 15 21:05:50.672074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2567/3681] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:50 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1365 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:50.677480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:50.678098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:50.684180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.684640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.685109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.685812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.686277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.686709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.690040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:50.824493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:50.832496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01ce17f2-351b-400f-b285-dcbd09341443 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.159418 Oct 15 21:05:50.833248 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2568/3682] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:50 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:50.837632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6f533a05-1587-430a-9b96-5ec159b6108e req-ab33932b-0336-420d-91c4-10ce1bb32505 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:50.846824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6f533a05-1587-430a-9b96-5ec159b6108e req-ab33932b-0336-420d-91c4-10ce1bb32505 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.847302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6f533a05-1587-430a-9b96-5ec159b6108e req-ab33932b-0336-420d-91c4-10ce1bb32505 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.847731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6f533a05-1587-430a-9b96-5ec159b6108e req-ab33932b-0336-420d-91c4-10ce1bb32505 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.862013 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6f533a05-1587-430a-9b96-5ec159b6108e req-ab33932b-0336-420d-91c4-10ce1bb32505 service nova] Creating event network-changed:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:05:50.867592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6f533a05-1587-430a-9b96-5ec159b6108e req-ab33932b-0336-420d-91c4-10ce1bb32505 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033231 Oct 15 21:05:50.868603 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2569/3683] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:50.893488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f2f40e6b-c045-4ef5-8b41-1c7ccc5cac81"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:50.895888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:50.902355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.902842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.903284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:50.903979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:50.904466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:50.904895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:51.165790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee8058e2-571d-4d9c-9e50-90d17f4b8266 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.277517 Oct 15 21:05:51.166479 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2570/3684] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:05:50 2022] POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-volume_attachments => generated 194 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:05:51.289392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:51.290139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:51.296319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:51.296802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:51.297409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:51.298157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:51.298622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:51.299051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:51.431548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:51.439473 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c97d487-1021-49b2-ac06-3ec0b9c18458 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1351 microversion: 2.1 time: 0.154867 Oct 15 21:05:51.440043 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2571/3685] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:51 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1351 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:51.695522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:51.696136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:51.702826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:51.703301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:51.703742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:51.705151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:51.705675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:51.706109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:51.869837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:51.879877 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53be209f-c5b7-4307-a639-a84115e920e9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1365 microversion: 2.1 time: 0.188268 Oct 15 21:05:51.880509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2572/3686] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:51 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1365 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:51.886505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:51.886668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:51.893910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:51.894476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:51.894804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:51.895894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:51.896615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:51.897152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:52.041183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:52.051165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f32de56f-02f4-4178-9d0f-a88e3137ccf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.168778 Oct 15 21:05:52.051960 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2573/3687] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:51 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:52.463719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:52.464532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:52.471645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:52.472203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:52.472733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:52.473637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:52.474164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:52.474797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:52.725123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:52.732938 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b155400-789d-447f-8ee7-92993c0a4813 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1540 microversion: 2.1 time: 0.273730 Oct 15 21:05:52.733545 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2574/3688] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:52 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1540 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:52.757915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:52.758876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:52.764408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:52.764897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:52.765368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:52.766074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:52.767022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:52.768010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:52.876554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4d590e3-65eb-4a2e-a0c3-1c32f0ca6264 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.121755 Oct 15 21:05:52.877708 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2575/3689] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:05:52 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb/os-interface => generated 284 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:05:52.905062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:52.905782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:52.912183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:52.912742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:52.913285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:52.914237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:52.914763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:52.915456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:53.294657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:53.304411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9a5b8ad-cdea-4d87-878a-b0f1c6b95411 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1365 microversion: 2.1 time: 0.403216 Oct 15 21:05:53.305216 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2576/3690] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:52 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1365 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:53.311435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:53.312041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:53.318334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:53.318829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:53.319272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:53.319985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:53.320448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:53.320883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:53.669645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:53.677783 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a17a1e90-0149-40cb-b017-17dd1f7a314d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.369924 Oct 15 21:05:53.678378 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2577/3691] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:53 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:54.328854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:54.329762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:54.334391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:54.337375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=33 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:54.349490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:54.352863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:54.354017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:54.355165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:54.358646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:54.361462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:54.698325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:54.709133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82ad9b8f-df33-4a1c-9104-bae89d32e2aa tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1365 microversion: 2.1 time: 0.384823 Oct 15 21:05:54.712631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2578/3692] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:54 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1365 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:54.732951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5619cb7-bd83-4f11-834a-00d2507dbc0b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:54.740446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5619cb7-bd83-4f11-834a-00d2507dbc0b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:54.741200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5619cb7-bd83-4f11-834a-00d2507dbc0b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:54.741825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5619cb7-bd83-4f11-834a-00d2507dbc0b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:54.768724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5619cb7-bd83-4f11-834a-00d2507dbc0b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:05:54.857229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5619cb7-bd83-4f11-834a-00d2507dbc0b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action" status: 202 len: 0 microversion: 2.1 time: 0.127814 Oct 15 21:05:54.861283 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2579/3693] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:05:54 2022] POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action => generated 0 bytes in 130 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:05:54.891455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:54.892521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:54.904710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:54.905342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:54.906209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:54.907348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:54.908140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:54.909532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.074355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:55.084225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1011dca0-4cc6-4837-92b8-a6e73bca0c3e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.224181 Oct 15 21:05:55.084949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2580/3694] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:54 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:55.091752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ff297fd9-4f30-40d9-846e-72ae02253456 req-a50f755c-1838-40fa-b971-20099fb78fc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-plugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:55.103876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ff297fd9-4f30-40d9-846e-72ae02253456 req-a50f755c-1838-40fa-b971-20099fb78fc5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:55.104574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ff297fd9-4f30-40d9-846e-72ae02253456 req-a50f755c-1838-40fa-b971-20099fb78fc5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:55.104995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ff297fd9-4f30-40d9-846e-72ae02253456 req-a50f755c-1838-40fa-b971-20099fb78fc5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.117508 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ff297fd9-4f30-40d9-846e-72ae02253456 req-a50f755c-1838-40fa-b971-20099fb78fc5 service nova] Creating event network-vif-plugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:05:55.122748 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ff297fd9-4f30-40d9-846e-72ae02253456 req-a50f755c-1838-40fa-b971-20099fb78fc5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035619 Oct 15 21:05:55.123712 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2581/3695] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:55.129131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:55.130242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:55.138638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:55.139420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:55.139984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.141516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:55.141516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:55.141932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.364649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:55.375091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e9de075-1f77-44bd-bf33-b6e96a766128 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.249942 Oct 15 21:05:55.389264 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2582/3696] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:55 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:55.389264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a5b57657-cb17-4664-8ecd-67269f814548 req-7fe6cb8f-c64b-4542-8711-611984b5d0e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "290a5c59-7d47-4799-9890-1b887bebafeb", "tag": "d307cb43-3fcb-4d77-b2d3-4f42c7027f97"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:55.392586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a5b57657-cb17-4664-8ecd-67269f814548 req-7fe6cb8f-c64b-4542-8711-611984b5d0e6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:55.392586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a5b57657-cb17-4664-8ecd-67269f814548 req-7fe6cb8f-c64b-4542-8711-611984b5d0e6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:55.392830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a5b57657-cb17-4664-8ecd-67269f814548 req-7fe6cb8f-c64b-4542-8711-611984b5d0e6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.404970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a5b57657-cb17-4664-8ecd-67269f814548 req-7fe6cb8f-c64b-4542-8711-611984b5d0e6 service nova] Creating event network-changed:d307cb43-3fcb-4d77-b2d3-4f42c7027f97 for instance 290a5c59-7d47-4799-9890-1b887bebafeb on nch1-master-1 Oct 15 21:05:55.411117 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a5b57657-cb17-4664-8ecd-67269f814548 req-7fe6cb8f-c64b-4542-8711-611984b5d0e6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034095 Oct 15 21:05:55.411751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2583/3697] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:05:55.737419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:55.738740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:55.750908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:55.751407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:55.751852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.753170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:55.754193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:55.754630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:55.987090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:55.989599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:05:55.998253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98dcfbff-e596-4943-85f9-507dac9430b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1365 microversion: 2.1 time: 0.266846 Oct 15 21:05:56.013062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2584/3698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:55 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1365 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:56.110357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:56.110887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:56.117924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:56.118409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:56.118847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:56.119531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:56.120033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:56.120460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:56.253392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:56.261202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c904df57-1840-4a43-93eb-4fae95e0d799 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.155133 Oct 15 21:05:56.261771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2585/3699] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:56 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:56.400120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:56.400767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:56.406746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:56.407228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:56.407665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:56.408351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:56.408837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:56.409281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:56.538507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:56.546758 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3677d9ca-78c5-4b3a-bb6d-c0732115fa96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.150522 Oct 15 21:05:56.547328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2586/3700] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:56 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:57.025415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:57.026054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:57.033337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.033827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.034333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.035067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.035529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.035953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.244602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:57.253031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee55263a-6b0a-4e0e-92fc-e94422575a3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1552 microversion: 2.1 time: 0.230949 Oct 15 21:05:57.253690 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2587/3701] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:57 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1552 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:57.275468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:57.275969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:57.282075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.282574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.283126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.283849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.284327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.284773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.403520 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6684b1d8-5653-4224-83c7-03269b9af307 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.131752 Oct 15 21:05:57.404298 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2588/3702] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:05:57 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/os-interface => generated 284 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:05:57.409676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:57.410449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:57.416678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.417227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.417676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.418385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.418848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.419278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.622492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:57.631717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a24257f1-6403-4bc9-b00b-dadec5061768 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.226123 Oct 15 21:05:57.632316 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2589/3703] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:57 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:57.637151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:57.638040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:57.644994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.645783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.646491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.647443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:57.648133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:57.648825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:57.863682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:57.873769 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aef9abcb-f26d-4cd9-b509-071b5a07f82e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.240088 Oct 15 21:05:57.885126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2590/3704] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:57 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:05:58.375898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6ebaa06f-cf44-48f0-9f1f-09016d5432c2 req-3b37b3dc-7356-4924-9561-432f19e293c7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:05:58.399792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ebaa06f-cf44-48f0-9f1f-09016d5432c2 req-3b37b3dc-7356-4924-9561-432f19e293c7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:58.400286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ebaa06f-cf44-48f0-9f1f-09016d5432c2 req-3b37b3dc-7356-4924-9561-432f19e293c7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:58.400786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ebaa06f-cf44-48f0-9f1f-09016d5432c2 req-3b37b3dc-7356-4924-9561-432f19e293c7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:58.414138 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6ebaa06f-cf44-48f0-9f1f-09016d5432c2 req-3b37b3dc-7356-4924-9561-432f19e293c7 service nova] Creating event network-changed:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:05:58.419844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6ebaa06f-cf44-48f0-9f1f-09016d5432c2 req-3b37b3dc-7356-4924-9561-432f19e293c7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047234 Oct 15 21:05:58.420645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2591/3705] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:05:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:05:58.658699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:58.659960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:58.666997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:58.667496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:58.668151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:58.668871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:58.669370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:58.669810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:58.976527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:58.985544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75ebcfcc-9fb7-4fb0-848a-8f6722635533 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.331235 Oct 15 21:05:58.987320 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2592/3706] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:58 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:05:58.992593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:05:58.993405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:05:59.000057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:59.000904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:59.001678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:59.003126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:05:59.003658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:05:59.004249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:05:59.318807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:05:59.333165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-23afce1b-7a04-42fe-95c7-e2d7752a4c72 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.344452 Oct 15 21:05:59.345657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2593/3707] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:05:58 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:00.011795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:00.012368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:00.019441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:00.019927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:00.020368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:00.021095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:00.021553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:00.021978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:00.624394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:00.632906 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37d3d407-5b4f-473a-bffb-48d15cc10440 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.625038 Oct 15 21:06:00.633652 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2594/3708] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:00 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:00.638760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:00.639485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:00.646431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:00.646986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:00.647538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:00.648368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:00.648925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:00.649498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:00.963453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:00.971954 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c1cbea5d-b99e-40bc-9a74-5b639208b85d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.337218 Oct 15 21:06:00.972541 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2595/3709] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:00 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:01.657747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:01.658973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:01.666679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:01.667234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:01.667730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:01.668442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:01.668946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:01.669615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:01.885377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:01.893944 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c1ea169-6c74-4313-a202-40ca5c4eec74 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.239385 Oct 15 21:06:01.894557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2596/3710] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:01 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:01.996591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:01.997162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:02.003632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:02.004138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:02.004575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:02.005372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:02.005850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:02.006276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:02.220817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:02.232248 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d6953e0-c86d-455b-ab48-1a1dbf503cc9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.239710 Oct 15 21:06:02.232827 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2597/3711] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:01 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:02.917855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:02.918502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:02.925546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:02.926035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:02.926472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:02.927162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:02.927623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:02.928048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:03.144640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:03.152646 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79a62352-5de9-49b6-ab27-5f201cae6142 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.238761 Oct 15 21:06:03.153257 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2598/3712] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:02 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:03.257149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:03.258187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:03.264973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:03.265505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:03.266018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:03.266772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:03.267197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:03.267708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:03.475546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:03.484571 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8bf9c8c9-67ae-4fde-9988-80be0b84d6fd tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.232386 Oct 15 21:06:03.485315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2599/3713] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:03 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:03.812772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:03.813312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:03.819584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:03.820070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:03.820512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:03.821239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:03.821700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:03.822130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:03.845717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2436ac85-ff00-4f89-8862-def2952d429c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/ips" status: 200 len: 151 microversion: 2.20 time: 0.135224 Oct 15 21:06:03.847022 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2600/3714] 10.222.0.16 () {60 vars in 1289 bytes} [Sat Oct 15 21:06:03 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/ips => generated 151 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 21:06:04.177498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:04.178398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:04.185545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:04.186102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:04.186641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:04.187424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:04.187938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:04.188478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:04.329218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:04.337715 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-567aaf66-84c5-4e6e-855e-1d1f783fbc40 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.164506 Oct 15 21:06:04.338398 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2601/3715] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:04 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:04.509117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:04.509773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:04.516250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:04.516748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:04.517292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:04.518518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:04.519013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:04.519449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:04.682631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:04.685173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:04.692231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb9786b1-bc07-43e5-a146-9a020d19f1c8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.187129 Oct 15 21:06:04.692804 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2602/3716] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:04 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:05.396782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:05.397424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:05.403273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.403760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.404196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.404892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.405385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.405821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.433078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:06:05.517723 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c3dc0fa-5fbb-4883-888f-d83661072c32 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "DELETE /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 204 len: 0 microversion: 2.42 time: 0.230955 Oct 15 21:06:05.518642 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2603/3717] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:06:05 2022] DELETE /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 15 21:06:05.523654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:05.524366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:05.530537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.531035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.531478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.532213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.532679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.533152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.673879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:05.682338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddcf8a38-7c57-40e8-a185-c396bc239f68 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.162975 Oct 15 21:06:05.682976 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2604/3718] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:05 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:05.687631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:05.688304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:05.695023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.695516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.695961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.696658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.697148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.697585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.894253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:05.905407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:05.906884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:05.908227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4aaa3eff-cdba-4a5e-8b48-57521d757a44 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2962 microversion: 2.42 time: 0.224310 Oct 15 21:06:05.908803 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2605/3719] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:05 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2962 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:05.914027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:05.914830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:05.922058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.922568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.923018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:05.923739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:05.924209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:05.924644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.342784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:06.352362 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd94a772-71a9-4efd-a9b5-97da79e24c97 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.442552 Oct 15 21:06:06.352935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2606/3720] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:05 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:06.357769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:06.358305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:06.364573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.365337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.365808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.366518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.366961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.367391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.390750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lazy-loading 'flavor' on Instance uuid 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:06:06.587381 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a273fb8-49b8-4c41-b467-69f1a3e32316 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/action" status: 202 len: 0 microversion: 2.20 time: 0.232876 Oct 15 21:06:06.588103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2607/3721] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 21:06:06 2022] POST /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/action => generated 0 bytes in 234 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:06:06.596416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:06.597278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:06.603663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.604150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.604592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.605324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.605787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.606251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.743084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:06.751271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:06.752711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:06.754547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d877b31e-c041-47b3-a1f5-2da00af1c2d6 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.162130 Oct 15 21:06:06.755237 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2608/3722] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:06 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:06.759927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:06.760526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:06.766308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.766830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.767313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.768028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.768483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.769551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.906499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:06.915403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d245549d-52a1-4a3a-8895-a1d95dfb0902 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1532 microversion: 2.1 time: 0.159132 Oct 15 21:06:06.916089 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2609/3723] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:06 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1532 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:06.939703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:06.941425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:06.948208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.948710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.949193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:06.950074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:06.950557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:06.950984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:07.150751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.162623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.164207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.165548 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0216d7c-ace0-476e-a8e1-58ce8c851464 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2962 microversion: 2.42 time: 0.230236 Oct 15 21:06:07.166121 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2610/3724] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:06 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2962 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:07.377019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:07.377908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:07.389800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:07.390297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:07.390742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:07.391455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:07.391944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:07.392376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:07.528785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.537374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-303a402f-5a3d-4bac-9905-b0499227618a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.164316 Oct 15 21:06:07.538030 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2611/3725] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:07 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:07.781554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:07.782207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:07.784068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:07.789958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:07.790440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:07.790875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:07.791562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:07.792015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:07.792440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:07.931250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.939618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.941716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:07.942888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-953f0860-3de1-4d4d-8247-36f9bfa65b12 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.165345 Oct 15 21:06:07.943454 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2612/3726] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:07 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:07.947834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:07.948442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:07.954246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:07.954739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:07.955183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:07.955882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:07.956346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:07.956951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:08.104539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:08.114374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c513fd0-63de-43a5-a8ce-7f876bff0b63 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1532 microversion: 2.1 time: 0.168914 Oct 15 21:06:08.114954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2613/3727] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:07 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1532 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:08.194450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:08.195066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:08.201470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:08.201965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:08.202408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:08.203225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:08.203566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:08.203992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:08.418682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:08.430941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:08.432419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:08.433902 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-574564a2-05c9-4aca-83e4-29cf76efd777 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2962 microversion: 2.42 time: 0.243935 Oct 15 21:06:08.434436 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2614/3728] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:08 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2962 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:08.568849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:08.570195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:08.577670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:08.578593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:08.579715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:08.580573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:08.581336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:08.582674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:08.756105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:08.758314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:08.766546 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8331a0df-23c4-4e40-a7f4-e579a6ce133c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.208272 Oct 15 21:06:08.767124 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2615/3729] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:08 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:08.969853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:08.970566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:08.978552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:08.979088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:08.979679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:08.980389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:08.981168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:08.982015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:09.306694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:09.319499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:09.320968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:09.322283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e8ac5cf-3cc3-42f6-a7dc-6f0c6710ed46 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.357816 Oct 15 21:06:09.323414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2616/3730] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:08 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:09.328312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:09.328926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:09.335625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:09.336124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:09.336563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:09.337321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:09.338010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:09.338445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:09.569750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:09.582300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03b20f9a-18fb-4023-b714-9c258e791205 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.257215 Oct 15 21:06:09.583474 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2617/3731] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:09 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:09.606032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c4a49eaa-10b6-46bf-9e77-4ba711afb04d req-18244ffd-812e-4b96-934a-7fc1e4e2c007 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f318670-01e5-4565-8e32-ac7ef73ba33c", "name": "network-vif-plugged", "status": "completed", "tag": "b779e022-9b5e-4fc4-8c92-ae2e834e6c81"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:09.617860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c4a49eaa-10b6-46bf-9e77-4ba711afb04d req-18244ffd-812e-4b96-934a-7fc1e4e2c007 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:09.618612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c4a49eaa-10b6-46bf-9e77-4ba711afb04d req-18244ffd-812e-4b96-934a-7fc1e4e2c007 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:09.619243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c4a49eaa-10b6-46bf-9e77-4ba711afb04d req-18244ffd-812e-4b96-934a-7fc1e4e2c007 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:09.632864 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c4a49eaa-10b6-46bf-9e77-4ba711afb04d req-18244ffd-812e-4b96-934a-7fc1e4e2c007 service nova] Creating event network-vif-plugged:b779e022-9b5e-4fc4-8c92-ae2e834e6c81 for instance 5f318670-01e5-4565-8e32-ac7ef73ba33c on nch1-master-1 Oct 15 21:06:09.638651 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c4a49eaa-10b6-46bf-9e77-4ba711afb04d req-18244ffd-812e-4b96-934a-7fc1e4e2c007 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039932 Oct 15 21:06:09.639880 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2618/3732] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:09.647514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:09.648504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:09.655620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:09.656379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:09.657007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:09.657925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:09.658546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:09.659151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:10.011433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.028795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.030380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.031715 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-184d209e-ded0-4b3f-ac0d-0b7a411fa49c tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2962 microversion: 2.42 time: 0.390440 Oct 15 21:06:10.032466 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2619/3733] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:09 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2962 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:10.038250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:10.039590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:10.069015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:10.069689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:10.070274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:10.080838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:10.081323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:10.082502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:10.423588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.433757 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-396d533b-b01e-4626-9e16-0a3795a40ad6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1723 microversion: 2.1 time: 0.399528 Oct 15 21:06:10.449510 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2620/3734] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:10 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1723 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:10.457686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:10.458621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:10.465390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:10.466198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:10.466900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:10.467904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:10.468377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:10.468989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:10.849851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.885600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.896101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:10.898674 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ceda7626-fc0d-44f1-b549-ac7549146d5b tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.447677 Oct 15 21:06:10.918684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2621/3735] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:10 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:10.925461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:10.926369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:10.934161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:10.934923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:10.935620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:10.936511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:10.937640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:10.938240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:11.290877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:11.305886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19982e5f-5604-4867-82f8-5138c4abcfb6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.385587 Oct 15 21:06:11.306412 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2622/3736] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:10 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:11.311183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:11.311777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:11.328214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:11.328703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:11.330320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:11.331051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:11.331759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:11.332183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:11.747837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:11.764002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:11.766608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:11.768092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a19821f-6412-4418-a484-90bcf19fd4f8 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2920 microversion: 2.42 time: 0.460749 Oct 15 21:06:11.771160 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2623/3737] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:11 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2920 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:11.776953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:11.777849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:11.787666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:11.788425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:11.789398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:11.790496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:11.792511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:11.793170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:12.049458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:12.064785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c02fb00d-56fc-43e2-8845-0433718253e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1700 microversion: 2.1 time: 0.292168 Oct 15 21:06:12.066690 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2624/3738] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:11 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1700 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:12.081843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:12.082458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:12.090638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:12.091192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:12.091568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:12.092391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:12.092847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:12.093791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:12.356855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:12.366880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:12.368535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:12.369791 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c3aa429-b15f-40d0-90c6-fc477b64198c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.302070 Oct 15 21:06:12.385924 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2625/3739] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:12 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:12.396253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-166eef5b-4618-446e-aca1-060fe34aab12 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:12.405132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166eef5b-4618-446e-aca1-060fe34aab12 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:12.405636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166eef5b-4618-446e-aca1-060fe34aab12 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:12.406284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-166eef5b-4618-446e-aca1-060fe34aab12 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:12.435124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-166eef5b-4618-446e-aca1-060fe34aab12 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Going to try to start instance {{(pid=138688) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 21:06:12.511859 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-166eef5b-4618-446e-aca1-060fe34aab12 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action" status: 202 len: 0 microversion: 2.1 time: 0.124351 Oct 15 21:06:12.514412 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2626/3740] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:06:12 2022] POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action => generated 0 bytes in 128 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:06:12.524796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:12.526420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:12.538486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:12.539283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:12.540165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:12.572837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:12.574026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:12.575658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:13.226611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:13.242784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2d502c4-4a1c-4059-99e1-e53f2afa9921 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.720615 Oct 15 21:06:13.243848 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2627/3741] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:12 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:13.248839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:13.249496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:13.266257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:13.266784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:13.267533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:13.268287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:13.268777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:13.269886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:13.712691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:13.729208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f32867eb-d62d-4341-8729-15ba2fcc3d52 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1709 microversion: 2.1 time: 0.484164 Oct 15 21:06:13.730548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2628/3742] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:13 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1709 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:13.735599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8396b53f-601a-4c23-b5a7-b3fd489dc8fc req-7dc68e54-2a37-4392-9b0a-14997e280493 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-deleted", "tag": "142ac24e-4393-42c7-a3ab-d8d47ad61698"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:13.761858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8396b53f-601a-4c23-b5a7-b3fd489dc8fc req-7dc68e54-2a37-4392-9b0a-14997e280493 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:13.764013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8396b53f-601a-4c23-b5a7-b3fd489dc8fc req-7dc68e54-2a37-4392-9b0a-14997e280493 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:13.765268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8396b53f-601a-4c23-b5a7-b3fd489dc8fc req-7dc68e54-2a37-4392-9b0a-14997e280493 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:13.789819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8396b53f-601a-4c23-b5a7-b3fd489dc8fc req-7dc68e54-2a37-4392-9b0a-14997e280493 service nova] Creating event network-vif-deleted:142ac24e-4393-42c7-a3ab-d8d47ad61698 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:13.806975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8396b53f-601a-4c23-b5a7-b3fd489dc8fc req-7dc68e54-2a37-4392-9b0a-14997e280493 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074851 Oct 15 21:06:13.808689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2629/3743] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:13.814509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:13.815534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:13.848123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:13.849701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:13.850499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:13.851464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:13.852155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:13.852794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:14.292255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:14.302506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:14.312294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:14.313644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccf74143-1524-47d2-854f-9a7be5923aa6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2873 microversion: 2.42 time: 0.503496 Oct 15 21:06:14.324283 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2630/3744] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:13 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2873 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:14.324283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cfda669f-9b42-4eff-abb2-c2db8b743e9a req-6e6bffba-700d-4585-bf1a-726cc534ed7d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-unplugged", "status": "completed", "tag": "a3ed0c87-6f02-402a-a80b-a011756b969c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:14.345706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cfda669f-9b42-4eff-abb2-c2db8b743e9a req-6e6bffba-700d-4585-bf1a-726cc534ed7d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:14.346198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cfda669f-9b42-4eff-abb2-c2db8b743e9a req-6e6bffba-700d-4585-bf1a-726cc534ed7d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:14.346681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cfda669f-9b42-4eff-abb2-c2db8b743e9a req-6e6bffba-700d-4585-bf1a-726cc534ed7d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:14.373058 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cfda669f-9b42-4eff-abb2-c2db8b743e9a req-6e6bffba-700d-4585-bf1a-726cc534ed7d service nova] Creating event network-vif-unplugged:a3ed0c87-6f02-402a-a80b-a011756b969c for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:14.380049 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cfda669f-9b42-4eff-abb2-c2db8b743e9a req-6e6bffba-700d-4585-bf1a-726cc534ed7d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.064251 Oct 15 21:06:14.381257 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2631/3745] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:14.392090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:14.393204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:14.403816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:14.404305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:14.404747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:14.405988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:14.407173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:14.407767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:14.699335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:14.711585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:14.713671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:14.716321 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-978673bb-7d7a-46b0-8556-1f52f258d980 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.333405 Oct 15 21:06:14.718345 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2632/3746] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:14 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:14.725068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:14.726184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:14.736706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:14.738917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:14.739838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:14.742443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:14.743445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:14.744223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:14.792503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:15.298607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:15.315209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc6f1472-3cf5-4d91-b23e-06006e647cac tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.595153 Oct 15 21:06:15.333121 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2633/3747] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:14 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:15.333121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:15.333121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:15.350208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:15.351377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:15.351377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:15.352016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:15.352527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:15.357231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:15.611267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:15.621976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a003d413-cc3b-4e95-832b-21047ab7496c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1713 microversion: 2.1 time: 0.296171 Oct 15 21:06:15.622415 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2634/3748] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:15 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1713 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:15.792610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:15.793386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:15.801877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:15.802689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:15.803407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:15.804185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:15.804743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:15.805276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:15.842711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:06:15.959566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-625bddf3-96cd-479b-8dd4-526322170087 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 204 len: 0 microversion: 2.32 time: 0.335098 Oct 15 21:06:15.969659 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2635/3749] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:06:15 2022] DELETE /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:06:15.980614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:15.981960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:15.992061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:15.992868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:15.998040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:15.999093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:15.999901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:16.000518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:16.441853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:16.453150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:16.454772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:16.457202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0608529d-552a-46b4-92e7-a8a2bf7ba030 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 2488 microversion: 2.42 time: 0.483124 Oct 15 21:06:16.458765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2636/3750] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:15 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 2488 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:16.463625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:16.464251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:16.471234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:16.471733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:16.472294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:16.473118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:16.473518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:16.474038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:16.753728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:16.762323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:16.764978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:16.767082 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f75df73-c8cc-4135-abda-d02bd3932530 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.307682 Oct 15 21:06:16.767848 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2637/3751] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:16 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:16.773027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-deleted", "tag": "77b4ac77-9b8d-4c4f-a944-9e3a506c9e67"}, {"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-deleted", "tag": "fd2bd5a8-8477-474b-8090-dc55b657fa33"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:16.785670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:16.786367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:16.786995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:16.803513 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] Creating event network-vif-deleted:77b4ac77-9b8d-4c4f-a944-9e3a506c9e67 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:16.809881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] Creating event network-vif-deleted:fd2bd5a8-8477-474b-8090-dc55b657fa33 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:16.861227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0f7ab7e7-839f-46ef-87ae-d79a7d78c966 req-8392f8d9-0c72-41a0-836e-5f0d8ee3c0b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.089805 Oct 15 21:06:16.862548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2638/3752] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:16 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:16.874729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:16.875783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:16.895547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:16.899746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:16.900652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:16.901658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:16.902650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:16.903369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.225226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.250598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.252422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.254467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9803b9d3-5b29-4548-8ad2-c63a0ff89276 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2951 microversion: 2.32 time: 0.390569 Oct 15 21:06:17.255314 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2639/3753] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:16 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2951 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:17.290514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:17.290731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:17.298411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.298913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.299356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.300108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.300635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.301172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.515403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.523982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4907c1b4-dc85-415c-84bd-c3854927bb6a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.243550 Oct 15 21:06:17.524578 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2640/3754] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:17 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:17.537514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-unplugged", "status": "completed", "tag": "fd2bd5a8-8477-474b-8090-dc55b657fa33"}, {"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-unplugged", "status": "completed", "tag": "79d61df5-38a3-4192-a9e3-ed2c945abba0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:17.544320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.544320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.544320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.557024 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] Creating event network-vif-unplugged:fd2bd5a8-8477-474b-8090-dc55b657fa33 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:17.557578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] Creating event network-vif-unplugged:79d61df5-38a3-4192-a9e3-ed2c945abba0 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:17.563705 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a4a4d7e9-3200-4c0a-b819-829cd905eb4f req-a31ca6b8-0791-4d72-994a-78aca6cdf037 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 358 microversion: 2.1 time: 0.037910 Oct 15 21:06:17.569685 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2641/3755] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:17 2022] POST /compute/v2.1/os-server-external-events => generated 358 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:17.575394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:17.576311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:17.584053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.584958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.585705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.586870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.587597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.588260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.677071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.693568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.696378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '84c19569dc6d49f4a29c814ef921e144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12a874b752564923ad6a4747c7a44d61', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:17.698475 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed27aa85-7349-441b-acd1-4ba8c26f63ac tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 200 len: 1604 microversion: 2.42 time: 0.127242 Oct 15 21:06:17.717383 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2642/3756] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:17 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 1604 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:17.803079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:17.803699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:17.811616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.812299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.813351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:17.814878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:17.817758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:17.818394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.000168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.012534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.016791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.018134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-891cc193-f0c5-41f0-8502-9272c61ffb48 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.219808 Oct 15 21:06:18.038840 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2643/3757] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:17 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:18.288919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:18.289583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:18.296552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.297079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.299087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.299087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.299087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.300405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.541843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.558867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.563561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.564852 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5c882a0-8605-4853-89f9-f5002b89de62 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2951 microversion: 2.32 time: 0.282279 Oct 15 21:06:18.566298 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2644/3758] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:18 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2951 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:18.571901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:18.572527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:18.592847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.593693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.594141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.594862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.595329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.595766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.765074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:18.777015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c4c4b8ee-b11a-4cf5-b99a-55530a434dc9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.209584 Oct 15 21:06:18.777629 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2645/3759] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:18 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:18.782154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:18.782820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:18.790139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.790139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.790421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.791251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.792327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.793676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.811064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] HTTP exception thrown: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Request method failure captured: Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d HTTP/1.1 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9SkQQvKOuqMaTeDdMrnbt7OXDPHpYvBZWG0gsZEx5thwjlolxXz7e0erabBcXaVCCKMIo9nKsC5nHNGJ1rUp5K4eeYo8JA7m2ZQfUB9XiWKZMaSbgm1Vs5BvJb0yFbmKDxxiXoB00TUnWD05vgIXTwT4zENCNp_UPpeIzTYTZDhUA Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.42 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 12a874b752564923ad6a4747c7a44d61 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_12a874b752564923ad6a4747c7a44d61"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 12a874b752564923ad6a4747c7a44d61 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TaggedBootDevicesTest_v242-1418230037-project Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 84c19569dc6d49f4a29c814ef921e144 Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1418230037-project Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'baec4512-cb15-4da3-b512-cd28dc89cf5d'} Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:18.813297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Returning 404 to user: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30232a27-1fc5-4d30-8e89-015db0eed6c6 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 404 len: 111 microversion: 2.42 time: 0.037296 Oct 15 21:06:18.820548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2646/3760] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:18 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:06:18.825689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55f7c62e-419e-4018-b471-4b63bbc96606 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '.version_select at 0x7fe74ed6d3a0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:18.847144 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55f7c62e-419e-4018-b471-4b63bbc96606 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-487010634" status: 204 len: 0 microversion: 2.42 time: 0.025042 Oct 15 21:06:18.848036 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2647/3761] 10.222.0.16 () {60 vars in 1259 bytes} [Sat Oct 15 21:06:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-487010634 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:06:18.882570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-deleted", "tag": "a3ed0c87-6f02-402a-a80b-a011756b969c"}, {"server_uuid": "baec4512-cb15-4da3-b512-cd28dc89cf5d", "name": "network-vif-deleted", "tag": "79d61df5-38a3-4192-a9e3-ed2c945abba0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:18.894984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:18.895802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:18.896468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:18.909950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] Creating event network-vif-deleted:a3ed0c87-6f02-402a-a80b-a011756b969c for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:18.910796 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] Creating event network-vif-deleted:79d61df5-38a3-4192-a9e3-ed2c945abba0 for instance baec4512-cb15-4da3-b512-cd28dc89cf5d on nch2-master-1 Oct 15 21:06:18.918739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-daca0705-c30e-404f-b5fb-5c47a0471e6a req-3a21e9b8-5b18-40dd-a288-2990b53ac706 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.041330 Oct 15 21:06:18.920830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2648/3762] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:18 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:06:19.050212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:19.051318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:19.072182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:19.072663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:19.073157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:19.073875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:19.087537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:19.088319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:19.297595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:19.318890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:19.320579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:19.322409 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5eb3bda-52be-4f4f-9097-6aecc43c0908 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.278390 Oct 15 21:06:19.341649 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2649/3763] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:19 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:19.598910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:19.600295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:19.614768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:19.620525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:19.621092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:19.621846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:19.622329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:19.622968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:19.922417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:19.932258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:19.933769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:19.938436 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b92af0a2-1faf-4c32-9278-c7006cda75bd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2951 microversion: 2.32 time: 0.347718 Oct 15 21:06:19.944664 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2650/3764] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:19 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2951 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:19.951316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:19.952150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:19.959333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:19.960069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:19.960707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:19.961632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:19.962255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:19.962855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:20.242743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:20.256371 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-677e9ff3-9f4a-4a2a-9678-888dbc7acb0c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.308691 Oct 15 21:06:20.256979 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2651/3765] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:19 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:20.359127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:20.372424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:20.399800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:20.409224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:20.410045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:20.411041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:20.411683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:20.412375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:20.421916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:20.793616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:20.838708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:20.840572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:20.842057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b30d1c6b-ce15-47df-b1e1-87d4fae6c59d tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.492882 Oct 15 21:06:20.857884 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2652/3766] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:20 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:20.984348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:20.985164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:21.009376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:21.009906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:21.010345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:21.011075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:21.011554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:21.011971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:21.779191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:21.796061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:21.801315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:21.802779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb2e26f3-f598-4ce2-89f5-2372c6ad6238 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2930 microversion: 2.32 time: 0.824020 Oct 15 21:06:21.806036 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2653/3767] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:20 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2930 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:21.819886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5baa59e2-92a5-4c6b-a8c2-b7071bce6fe9 req-c49029d6-43f3-4d85-aeed-4fdd64f79ab6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-unplugged", "status": "completed", "tag": "675ba1c4-0b55-4936-9005-97673bfc6b22"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:21.859537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5baa59e2-92a5-4c6b-a8c2-b7071bce6fe9 req-c49029d6-43f3-4d85-aeed-4fdd64f79ab6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:21.860929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5baa59e2-92a5-4c6b-a8c2-b7071bce6fe9 req-c49029d6-43f3-4d85-aeed-4fdd64f79ab6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:21.863723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5baa59e2-92a5-4c6b-a8c2-b7071bce6fe9 req-c49029d6-43f3-4d85-aeed-4fdd64f79ab6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:21.901753 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5baa59e2-92a5-4c6b-a8c2-b7071bce6fe9 req-c49029d6-43f3-4d85-aeed-4fdd64f79ab6 service nova] Creating event network-vif-unplugged:675ba1c4-0b55-4936-9005-97673bfc6b22 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:21.920827 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5baa59e2-92a5-4c6b-a8c2-b7071bce6fe9 req-c49029d6-43f3-4d85-aeed-4fdd64f79ab6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.112558 Oct 15 21:06:21.923322 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2654/3768] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:21 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:21.946754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:21.949796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:21.962362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:21.968776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:21.969734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:21.971121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:21.971835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:21.972616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:22.432763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:22.448496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e77dcffb-b544-4237-afaa-88c08204f9a7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1527 microversion: 2.1 time: 0.523806 Oct 15 21:06:22.452669 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2655/3769] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:21 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1527 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:22.465920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:22.467367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:22.489281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:22.490419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:22.493998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:22.495630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:22.498944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:22.499681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:22.897572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:22.918781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:22.920752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:22.922449 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b69b57c-4e45-4719-af59-dfa925f1c3a4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.467790 Oct 15 21:06:22.938517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2656/3770] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:22 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:22.954758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:22.955393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:22.963323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:22.963980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:22.964742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:22.965864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:22.966500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:22.967103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:23.278770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:23.305750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:23.310031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:23.315113 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e3054dea-5845-41df-89d5-0da8d99375d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2909 microversion: 2.32 time: 0.372415 Oct 15 21:06:23.317936 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2657/3771] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:22 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2909 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:23.510188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:23.510188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:23.510188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:23.522737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:23.523527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:23.524358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:23.528992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:23.530155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:23.531249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.273551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:24.285165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f70be02e-e8d1-4c74-a487-5750dbf27f9a tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1509 microversion: 2.1 time: 0.808387 Oct 15 21:06:24.286103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2658/3772] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:23 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1509 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:24.292713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:24.293675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:24.301354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.302109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.302802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.303863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.304450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.304985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.694032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:24.704735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:24.706717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:24.708229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27bf8598-57c2-4cf9-8c76-d6da9c5aa26f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1743 microversion: 2.20 time: 0.421028 Oct 15 21:06:24.709222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2659/3773] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:24 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1743 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:24.714505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-unplugged", "status": "completed", "tag": "f425e84b-b112-4afe-9505-a001d8f8ce16"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-unplugged", "status": "completed", "tag": "5621f2f9-6408-416e-9e9b-6029170044f4"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-unplugged", "status": "completed", "tag": "2a2ab61a-72e6-4577-a7e1-d1b4987a2389"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-unplugged", "status": "completed", "tag": "c874ec72-d847-4302-a82c-1a15a1e24a5f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:24.728981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.729694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.730368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.747963 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Creating event network-vif-unplugged:f425e84b-b112-4afe-9505-a001d8f8ce16 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:24.748735 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Creating event network-vif-unplugged:5621f2f9-6408-416e-9e9b-6029170044f4 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:24.749486 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Creating event network-vif-unplugged:2a2ab61a-72e6-4577-a7e1-d1b4987a2389 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:24.750427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] Creating event network-vif-unplugged:c874ec72-d847-4302-a82c-1a15a1e24a5f for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:24.757907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-36778f3a-04b4-4f02-9d60-797739633128 req-6f4a615a-2df3-4502-bc6e-b808865c5a26 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 704 microversion: 2.1 time: 0.047039 Oct 15 21:06:24.759221 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2660/3774] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:24 2022] POST /compute/v2.1/os-server-external-events => generated 704 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:24.765392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:24.766793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:24.773793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.775096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.775821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.776852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.777594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.778399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.799567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] HTTP exception thrown: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Request method failure captured: Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d HTTP/1.1 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9SkQQvKOuqMaTeDdMrnbt7OXDPHpYvBZWG0gsZEx5thwjlolxXz7e0erabBcXaVCCKMIo9nKsC5nHNGJ1rUp5K4eeYo8JA7m2ZQfUB9XiWKZMaSbgm1Vs5BvJb0yFbmKDxxiXoB00TUnWD05vgIXTwT4zENCNp_UPpeIzTYTZDhUA Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.42 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 12a874b752564923ad6a4747c7a44d61 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_12a874b752564923ad6a4747c7a44d61"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 12a874b752564923ad6a4747c7a44d61 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TaggedBootDevicesTest_v242-1418230037-project Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 84c19569dc6d49f4a29c814ef921e144 Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1418230037-project Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'baec4512-cb15-4da3-b512-cd28dc89cf5d'} Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:24.802564 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:24.806169 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:24.807659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Returning 404 to user: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:24.808743 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39bd2710-b952-4e71-8e2d-7646e4f53999 tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "DELETE /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 404 len: 111 microversion: 2.42 time: 0.048268 Oct 15 21:06:24.810048 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2661/3775] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:06:24 2022] DELETE /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:06:24.816121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b7a74000-b347-4705-b6ae-5d871b4c6784 req-9f14c6b7-f964-48bc-99a1-3d9a9768ed12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-deleted", "tag": "675ba1c4-0b55-4936-9005-97673bfc6b22"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:24.833091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b7a74000-b347-4705-b6ae-5d871b4c6784 req-9f14c6b7-f964-48bc-99a1-3d9a9768ed12 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.833813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b7a74000-b347-4705-b6ae-5d871b4c6784 req-9f14c6b7-f964-48bc-99a1-3d9a9768ed12 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.834420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b7a74000-b347-4705-b6ae-5d871b4c6784 req-9f14c6b7-f964-48bc-99a1-3d9a9768ed12 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.848795 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b7a74000-b347-4705-b6ae-5d871b4c6784 req-9f14c6b7-f964-48bc-99a1-3d9a9768ed12 service nova] Creating event network-vif-deleted:675ba1c4-0b55-4936-9005-97673bfc6b22 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:24.872121 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b7a74000-b347-4705-b6ae-5d871b4c6784 req-9f14c6b7-f964-48bc-99a1-3d9a9768ed12 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056238 Oct 15 21:06:24.872815 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2662/3776] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:24.884644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:24.886366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:24.896684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.897614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.898442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:24.899642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:24.900206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:24.900903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.214158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.228159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.229808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.231322 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a232fee-65c0-4709-bf40-2223075a66d5 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2579 microversion: 2.32 time: 0.353445 Oct 15 21:06:25.231966 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2663/3777] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:24 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2579 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:25.250127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:25.250869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] [instance: baec4512-cb15-4da3-b512-cd28dc89cf5d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:25.259411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:25.260190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:25.260882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.261864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:25.262526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:25.263183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.283227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] HTTP exception thrown: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Request method failure captured: Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d HTTP/1.1 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9SkQQvKOuqMaTeDdMrnbt7OXDPHpYvBZWG0gsZEx5thwjlolxXz7e0erabBcXaVCCKMIo9nKsC5nHNGJ1rUp5K4eeYo8JA7m2ZQfUB9XiWKZMaSbgm1Vs5BvJb0yFbmKDxxiXoB00TUnWD05vgIXTwT4zENCNp_UPpeIzTYTZDhUA Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.42 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 12a874b752564923ad6a4747c7a44d61 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_12a874b752564923ad6a4747c7a44d61"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/12a874b752564923ad6a4747c7a44d61"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 12a874b752564923ad6a4747c7a44d61 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1418230037 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TaggedBootDevicesTest_v242-1418230037-project Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 84c19569dc6d49f4a29c814ef921e144 Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1418230037-project Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'baec4512-cb15-4da3-b512-cd28dc89cf5d'} Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:25.286401 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. Oct 15 21:06:25.289471 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:25.290812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] Returning 404 to user: Instance baec4512-cb15-4da3-b512-cd28dc89cf5d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:25.291894 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7163e88b-aa5c-425a-8077-088d6a0f60ad tempest-TaggedBootDevicesTest_v242-1418230037 tempest-TaggedBootDevicesTest_v242-1418230037-project] 10.222.0.16 "GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d" status: 404 len: 111 microversion: 2.42 time: 0.049942 Oct 15 21:06:25.295895 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2664/3778] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:25 2022] GET /compute/v2.1/servers/baec4512-cb15-4da3-b512-cd28dc89cf5d => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:06:25.311509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:25.312180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:25.320483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:25.321017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:25.321663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.322415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:25.323744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:25.323744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.527834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.540745 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8dc38a54-c3a5-4c2e-9809-9b7e1b2cff84 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1509 microversion: 2.1 time: 0.234144 Oct 15 21:06:25.542056 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2665/3779] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:25 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1509 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:25.743483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:25.744431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:25.753713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:25.754666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:25.755372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.756394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:25.757501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:25.758329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:25.943737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.957703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.960975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:25.962486 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a46bc5a-de24-4d67-875c-b3644bda2b66 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1743 microversion: 2.20 time: 0.225937 Oct 15 21:06:25.973223 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2666/3780] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:25 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1743 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:26.265522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:26.266305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:26.274228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:26.274792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:26.275301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:26.276092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:26.276630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:26.277129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:26.446473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:26.456934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:26.459093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:26.460631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bbe3fbd0-1a85-4772-82c0-fc15b52a66db tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2533 microversion: 2.32 time: 0.199605 Oct 15 21:06:26.461365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2667/3781] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:26 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2533 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:26.568591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:26.570687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:26.588745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:26.590088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:26.590938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:26.592732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:26.593863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:26.594742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:26.778951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:26.788258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5992d2d7-777c-4db1-b193-decc56293b28 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1509 microversion: 2.1 time: 0.226508 Oct 15 21:06:26.788859 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2668/3782] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:26 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1509 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:26.793433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-088544af-849f-466c-b1d6-ff7a5a126352 req-d340bf1a-21d9-421f-8170-8c3ac19888c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "name": "network-vif-plugged", "status": "completed", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:26.803789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-088544af-849f-466c-b1d6-ff7a5a126352 req-d340bf1a-21d9-421f-8170-8c3ac19888c9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:26.804212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-088544af-849f-466c-b1d6-ff7a5a126352 req-d340bf1a-21d9-421f-8170-8c3ac19888c9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:26.804627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-088544af-849f-466c-b1d6-ff7a5a126352 req-d340bf1a-21d9-421f-8170-8c3ac19888c9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:26.816469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-088544af-849f-466c-b1d6-ff7a5a126352 req-d340bf1a-21d9-421f-8170-8c3ac19888c9 service nova] Creating event network-vif-plugged:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:06:26.821477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-088544af-849f-466c-b1d6-ff7a5a126352 req-d340bf1a-21d9-421f-8170-8c3ac19888c9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031378 Oct 15 21:06:26.822069 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2669/3783] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:26.940858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0f0f1495-e688-4e16-b47d-c54a32aeab82 req-ea42d7cc-87ec-4a24-9cb6-4aaac44cc833 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-deleted", "tag": "c874ec72-d847-4302-a82c-1a15a1e24a5f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:26.954520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f0f1495-e688-4e16-b47d-c54a32aeab82 req-ea42d7cc-87ec-4a24-9cb6-4aaac44cc833 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:26.955649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f0f1495-e688-4e16-b47d-c54a32aeab82 req-ea42d7cc-87ec-4a24-9cb6-4aaac44cc833 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:26.957063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f0f1495-e688-4e16-b47d-c54a32aeab82 req-ea42d7cc-87ec-4a24-9cb6-4aaac44cc833 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:26.974701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0f0f1495-e688-4e16-b47d-c54a32aeab82 req-ea42d7cc-87ec-4a24-9cb6-4aaac44cc833 service nova] Creating event network-vif-deleted:c874ec72-d847-4302-a82c-1a15a1e24a5f for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:26.982041 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0f0f1495-e688-4e16-b47d-c54a32aeab82 req-ea42d7cc-87ec-4a24-9cb6-4aaac44cc833 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047863 Oct 15 21:06:26.983307 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2670/3784] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:27.004301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:27.006059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:27.017528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:27.018088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:27.021375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:27.022322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:27.022815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:27.023324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:27.332635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:27.341585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:27.343168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:27.344433 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04b963d3-6892-4e43-abb4-91ff3e3e018f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1743 microversion: 2.20 time: 0.347415 Oct 15 21:06:27.345214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2671/3785] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:26 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1743 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:27.494256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:27.494917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:27.501357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:27.502044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:27.502488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:27.504075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:27.504722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:27.505228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:27.693791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:27.707375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:27.709583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:27.711180 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5cdcfec-c859-474b-a5b0-3384faa14893 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2373 microversion: 2.32 time: 0.221530 Oct 15 21:06:27.711758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2672/3786] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:27 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2373 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:27.811712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:27.812304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:27.821008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:27.821875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:27.822481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:27.823361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:27.824270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:27.825718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:27.845309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:28.092254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:28.105366 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b5827ac-bf77-468f-8745-ba74ea68f98c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1512 microversion: 2.1 time: 0.297533 Oct 15 21:06:28.123327 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2673/3787] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:27 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1512 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:28.140069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:28.140749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:28.150404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.151127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.151764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.152857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.153621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.154231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.422570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:28.430835 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-17beab7a-e267-45a8-a768-a7d7d000173e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1452 microversion: 2.1 time: 0.294646 Oct 15 21:06:28.431740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2674/3788] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:28 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1452 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:28.436277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:28.436888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:28.443399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.443924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.444370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.445446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.446497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.446497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.723825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:28.735406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:28.736894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:28.738467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd782f2e-7d4b-49c0-81d1-8402d54cffc0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1743 microversion: 2.20 time: 0.305655 Oct 15 21:06:28.739311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2675/3789] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:28 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1743 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:28.745952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:28.746490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:28.762706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.763351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.764109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.765021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.765761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.766366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.800839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lazy-loading 'flavor' on Instance uuid 5f318670-01e5-4565-8e32-ac7ef73ba33c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:06:28.816659 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'shelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state shelved_offloaded Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 14 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: {"shelve": {}} Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'shelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state shelved_offloaded Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'shelve': {}}} Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 49, in _shelve Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.shelve(context, instance) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state shelved_offloaded. Cannot shelve while the instance is in this state. Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 56, in _shelve Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:06:28.818297 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'shelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state shelved_offloaded Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'shelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state shelved_offloaded Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'shelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state shelved_offloaded {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4dc90ee-1ce5-45f6-bf37-242427019b45 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 155 microversion: 2.1 time: 0.079861 Oct 15 21:06:28.822036 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2676/3790] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:06:28 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 155 bytes in 81 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:06:28.834556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:28.835432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:28.843905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.843905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.843905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:28.844557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:28.845020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:28.845564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.153057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:29.167796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:29.169543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:29.170824 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1195e6d6-1c11-4e4a-8052-02457b409752 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 2327 microversion: 2.32 time: 0.345965 Oct 15 21:06:29.177465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2677/3791] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:28 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 2327 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:29.177465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:29.178173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:29.185686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.186436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.187099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.188378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.189529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.190917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.232924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=138688) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 21:06:29.330720 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-395088b4-5558-4271-aa52-6c0f64da09da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 202 len: 0 microversion: 2.1 time: 0.157978 Oct 15 21:06:29.331814 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2678/3792] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:06:29 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:06:29.340875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5a503bd4-4e4c-434e-a3cb-b8db2dd09424 req-2d0f4051-0e01-4810-b18c-0998d0227916 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f318670-01e5-4565-8e32-ac7ef73ba33c", "name": "network-vif-unplugged", "status": "completed", "tag": "b779e022-9b5e-4fc4-8c92-ae2e834e6c81"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:29.352639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5a503bd4-4e4c-434e-a3cb-b8db2dd09424 req-2d0f4051-0e01-4810-b18c-0998d0227916 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.353261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5a503bd4-4e4c-434e-a3cb-b8db2dd09424 req-2d0f4051-0e01-4810-b18c-0998d0227916 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.353690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5a503bd4-4e4c-434e-a3cb-b8db2dd09424 req-2d0f4051-0e01-4810-b18c-0998d0227916 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.370702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_external_events [req-5a503bd4-4e4c-434e-a3cb-b8db2dd09424 req-2d0f4051-0e01-4810-b18c-0998d0227916 service nova] Unable to find a host for instance 5f318670-01e5-4565-8e32-ac7ef73ba33c. Dropping event network-vif-unplugged {{(pid=138688) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 15 21:06:29.373580 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5a503bd4-4e4c-434e-a3cb-b8db2dd09424 req-2d0f4051-0e01-4810-b18c-0998d0227916 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.035880 Oct 15 21:06:29.381233 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2679/3793] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:29 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 49 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 15 21:06:29.388878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-deleted", "tag": "f425e84b-b112-4afe-9505-a001d8f8ce16"}, {"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-deleted", "tag": "2a2ab61a-72e6-4577-a7e1-d1b4987a2389"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:29.402200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.402766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.403209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.415972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] Creating event network-vif-deleted:f425e84b-b112-4afe-9505-a001d8f8ce16 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:29.416420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] Creating event network-vif-deleted:2a2ab61a-72e6-4577-a7e1-d1b4987a2389 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:29.421934 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0a46011d-4021-4e89-8311-ea4d00af8006 req-85975707-bf71-42d4-bba2-e1908e2afd78 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.038198 Oct 15 21:06:29.422567 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2680/3794] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:29 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:29.429539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:29.430680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:29.437678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.438283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.438727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.439502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.439965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.440476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.653516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:29.665842 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6fd25b10-31bf-4395-8de5-0e4f27d582c8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1460 microversion: 2.1 time: 0.241805 Oct 15 21:06:29.667253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2681/3795] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:29 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1460 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:29.766685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:29.767391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:29.776228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.776712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.777207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:29.778721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:29.779255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:29.779679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.046935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:30.062115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:30.063663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:30.064950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8239b31-a23d-4804-b321-ac77dfc84a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.302325 Oct 15 21:06:30.069355 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2682/3796] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:29 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:30.207265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:30.207876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:30.217863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.220124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.220564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.222072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.222685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.223390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.387876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.389535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.390281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.391071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lazy-loading 'fault' on Instance uuid c17ea85b-930c-4f69-bbea-ad8b6dad052d {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:06:30.469186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:30.490885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:30.497531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f00da962dc434ae18a1ab29228846688', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90ca0b4db2b64900aaa1a4e05dec8a6f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:30.503400 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14844285-7df6-47b3-925e-ae452d804d7c tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 200 len: 1371 microversion: 2.32 time: 0.304065 Oct 15 21:06:30.506495 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2683/3797] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:30 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 1371 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:30.530180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:30.531331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:30.581427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.582001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.582692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.584326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.585345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.586649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.659737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:06:30.870774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4685a82d-bfa5-46a9-b3db-3639f6569987 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "DELETE /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 204 len: 0 microversion: 2.1 time: 0.347661 Oct 15 21:06:30.872865 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2684/3798] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:06:30 2022] DELETE /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 0 bytes in 351 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 21:06:30.888791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:30.902013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:30.916071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.916873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.917387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:30.923352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:30.923798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:30.924314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.162846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:31.173729 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1996bad-07fa-44fc-ab76-ee9b377185a9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.298972 Oct 15 21:06:31.176242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2685/3799] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:30 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:31.181875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:31.182678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:31.190667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.191730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.192314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.193281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.194028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.194618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.388279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:31.400708 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07a75204-69ad-4e04-aab5-fd17a399c5ea tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1664 microversion: 2.1 time: 0.221693 Oct 15 21:06:31.413079 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2686/3800] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:31 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1664 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:31.413079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:31.413079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:31.420368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.420896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.422389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.425986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.426916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.427582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.638499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:31.649296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:31.651837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:31.655460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee780155-a90a-442c-892a-b5fd6ee8e33c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.251241 Oct 15 21:06:31.656396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2687/3801] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:31 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:31.687141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ee6b5539-3372-4332-981b-50226737b940 req-84993e2f-0870-417e-96da-cc8baeb25b17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c17ea85b-930c-4f69-bbea-ad8b6dad052d", "name": "network-vif-deleted", "tag": "5621f2f9-6408-416e-9e9b-6029170044f4"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:31.715015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee6b5539-3372-4332-981b-50226737b940 req-84993e2f-0870-417e-96da-cc8baeb25b17 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.716430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee6b5539-3372-4332-981b-50226737b940 req-84993e2f-0870-417e-96da-cc8baeb25b17 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.717075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee6b5539-3372-4332-981b-50226737b940 req-84993e2f-0870-417e-96da-cc8baeb25b17 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.740370 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ee6b5539-3372-4332-981b-50226737b940 req-84993e2f-0870-417e-96da-cc8baeb25b17 service nova] Creating event network-vif-deleted:5621f2f9-6408-416e-9e9b-6029170044f4 for instance c17ea85b-930c-4f69-bbea-ad8b6dad052d on nch1-master-1 Oct 15 21:06:31.750385 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ee6b5539-3372-4332-981b-50226737b940 req-84993e2f-0870-417e-96da-cc8baeb25b17 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067105 Oct 15 21:06:31.751159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2688/3802] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:31.757299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:31.758202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:31.767348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.768919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.770371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.771228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:31.771739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:31.772230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:31.779051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:31.795052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] HTTP exception thrown: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Request method failure captured: Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d HTTP/1.1 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-G-TBWUG34joACYA2SU7wY1BfEd6SltpFqjxlW-AK0WU_UDLcnOFPbI77K43CINnKCrKwVa_cm7WdTd3pvebsKWwF_IpNzStt4vKJxLXIF8xBRQbDMIxbP7idnFwZWXzJKo-zsyAyRrlsNGLOfUzN4R0krUIVjpGZqxd_3CE0Y8lw Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.32 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 90ca0b4db2b64900aaa1a4e05dec8a6f Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 90ca0b4db2b64900aaa1a4e05dec8a6f Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TaggedBootDevicesTest-268164479-project Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f00da962dc434ae18a1ab29228846688 Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TaggedBootDevicesTest-268164479-project Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c17ea85b-930c-4f69-bbea-ad8b6dad052d'} Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:31.796664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Returning 404 to user: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81f1b926-4557-4366-8add-e57bdfbd6d28 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 404 len: 111 microversion: 2.32 time: 0.045861 Oct 15 21:06:31.800830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2689/3803] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:31 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:06:31.808628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c4e6534-fb5c-4446-8a23-98d0895db3fd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '.version_select at 0x7fe74e8251f0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:31.839258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c4e6534-fb5c-4446-8a23-98d0895db3fd tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1763522385" status: 204 len: 0 microversion: 2.32 time: 0.035554 Oct 15 21:06:31.843648 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2690/3804] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Oct 15 21:06:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1763522385 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:06:32.202105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:32.203756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:32.240628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:32.241611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:32.244705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:32.245340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:32.245943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:32.248052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:33.045400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:33.068157 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a52a83de-eb7b-4b30-90aa-6e76b00e8af2 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.871060 Oct 15 21:06:33.069971 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2691/3805] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:32 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 874 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:33.075939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:33.077148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:33.110716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:33.111813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:33.113844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:33.116072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:33.119725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:33.120361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:34.322459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:34.338056 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25f1d0a9-c3e1-4862-989a-9ba5dc0c4313 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1664 microversion: 2.1 time: 1.266750 Oct 15 21:06:34.338981 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2692/3806] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:33 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1664 bytes in 1268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:34.353969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:34.354821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:34.366456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:34.367665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:34.369947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:34.370868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:34.373928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:34.374533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:35.009274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:35.034280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:35.050784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:35.060031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-680022a6-d464-4829-a287-9eed4fc25390 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.717059 Oct 15 21:06:35.062030 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2693/3807] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:34 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:35.074235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:35.075732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:35.094266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:35.094777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:35.098314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:35.100874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:35.102198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:35.109406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:35.591436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:35.607186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9cd34878-767d-4654-bf06-2658fd7a51c9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.542984 Oct 15 21:06:35.608874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2694/3808] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:35 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:35.622026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:35.622418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:35.663130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:35.664967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:35.667657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:35.669089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:35.673017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:35.673264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:36.443823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:36.467481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c0112500-4322-4c08-9a03-952664254be2 tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1596 microversion: 2.1 time: 0.848266 Oct 15 21:06:36.473835 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2695/3809] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:35 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1596 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:36.499623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:36.503854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:36.527807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:36.529719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:36.531948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:36.534618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:36.535178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:36.535634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:36.802372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:36.814225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:36.816316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:36.817997 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2474c2b2-ae88-4c18-8fa2-4c65e204a1b2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.338408 Oct 15 21:06:36.819014 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2696/3810] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:36 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:36.823861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:36.824632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:36.832060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:36.833400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:36.834007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:36.834835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:36.835404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:36.835959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:37.041615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:37.050614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af3bcd3f-b3d0-4a0f-ad2c-8cdbe73f43f1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.230380 Oct 15 21:06:37.077155 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2697/3811] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:36 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:37.160239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e23c1041-4abf-4dee-91eb-7a88460d84b0 req-a8afa27b-0963-4ab3-ae48-32d9c3ea93f2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "290a5c59-7d47-4799-9890-1b887bebafeb", "name": "network-vif-deleted", "tag": "d307cb43-3fcb-4d77-b2d3-4f42c7027f97"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:37.178336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e23c1041-4abf-4dee-91eb-7a88460d84b0 req-a8afa27b-0963-4ab3-ae48-32d9c3ea93f2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:37.179443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e23c1041-4abf-4dee-91eb-7a88460d84b0 req-a8afa27b-0963-4ab3-ae48-32d9c3ea93f2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:37.180101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e23c1041-4abf-4dee-91eb-7a88460d84b0 req-a8afa27b-0963-4ab3-ae48-32d9c3ea93f2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:37.226292 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e23c1041-4abf-4dee-91eb-7a88460d84b0 req-a8afa27b-0963-4ab3-ae48-32d9c3ea93f2 service nova] Creating event network-vif-deleted:d307cb43-3fcb-4d77-b2d3-4f42c7027f97 for instance 290a5c59-7d47-4799-9890-1b887bebafeb on nch1-master-1 Oct 15 21:06:37.237540 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e23c1041-4abf-4dee-91eb-7a88460d84b0 req-a8afa27b-0963-4ab3-ae48-32d9c3ea93f2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082603 Oct 15 21:06:37.255473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2698/3812] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:37.502100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:37.503023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:37.510948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:37.518476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:37.519053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:37.520560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:37.524020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:37.524571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:37.601380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:37.609867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:37.615548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:37.616127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:37.616702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lazy-loading 'fault' on Instance uuid 290a5c59-7d47-4799-9890-1b887bebafeb {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:06:37.719957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be3724ed7cfa41cabc1c1a5e8fb1f218', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b2c267ee2a9d4191a6e9de6e7bdee449', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:37.740898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c657a4ce-719d-42bd-82a7-599aa2fd1b9b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 200 len: 1315 microversion: 2.1 time: 0.241774 Oct 15 21:06:37.743081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2699/3813] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:37 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 1315 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:37.848426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:37.849142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:37.866239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:37.870231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:37.870695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:37.876083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:37.878302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:37.880652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:38.295030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:38.307719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:38.321129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:38.325237 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a894e56-234a-4f5f-b02d-c82f2feb3428 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.482169 Oct 15 21:06:38.335623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2700/3814] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:37 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:38.341121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:38.341920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:38.360496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:38.363934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:38.364709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:38.366026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:38.371733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:38.372417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:38.867503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:38.879059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10dd7f8d-8e56-47ec-b42a-00dc8435ce19 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.541998 Oct 15 21:06:38.884799 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2701/3815] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:38 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:38.890791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:38.891729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] [instance: 290a5c59-7d47-4799-9890-1b887bebafeb] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:38.899630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:38.902823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:38.903508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:38.904684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:38.905427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:38.906153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:38.942749 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] HTTP exception thrown: Instance 290a5c59-7d47-4799-9890-1b887bebafeb could not be found. Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Request method failure captured: Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb HTTP/1.1 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCelGBxPhb2qvuWpBb_mp6yMj6bUa7QVe4nyw7aokJZhAZdbKOR6iwE7eiG5V0hwtUe2icAuACLTy3Fduzej8TADUjAHarGNWMWKlkg7ESjXHaGVScMcn8WyAkb8Gt_gSgsYDN06oNQWW1iLSCLhURvJN_p8el9MyCSzjCBnEwEh6Q Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b2c267ee2a9d4191a6e9de6e7bdee449 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestFqdnHostnames-135298658 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b2c267ee2a9d4191a6e9de6e7bdee449"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b2c267ee2a9d4191a6e9de6e7bdee449"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b2c267ee2a9d4191a6e9de6e7bdee449"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b2c267ee2a9d4191a6e9de6e7bdee449"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestFqdnHostnames-135298658 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b2c267ee2a9d4191a6e9de6e7bdee449 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestFqdnHostnames-135298658 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestFqdnHostnames-135298658-project Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: be3724ed7cfa41cabc1c1a5e8fb1f218 Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestFqdnHostnames-135298658-project Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 290a5c59-7d47-4799-9890-1b887bebafeb could not be found. Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '290a5c59-7d47-4799-9890-1b887bebafeb'} Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 290a5c59-7d47-4799-9890-1b887bebafeb could not be found. Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 290a5c59-7d47-4799-9890-1b887bebafeb could not be found. Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:38.945127 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 290a5c59-7d47-4799-9890-1b887bebafeb could not be found. Oct 15 21:06:38.949651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:38.951020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Returning 404 to user: Instance 290a5c59-7d47-4799-9890-1b887bebafeb could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:38.952093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-016ed027-b5fb-4e80-9959-645c29f8680f tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb" status: 404 len: 111 microversion: 2.1 time: 0.065972 Oct 15 21:06:38.953178 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2702/3816] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:38 2022] GET /compute/v2.1/servers/290a5c59-7d47-4799-9890-1b887bebafeb => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:06:38.962922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1326d8b2-b51c-488c-8ec0-52096d13e91b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] Calling method '.version_select at 0x7fe74e37d280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:38.990271 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1326d8b2-b51c-488c-8ec0-52096d13e91b tempest-ServersTestFqdnHostnames-135298658 tempest-ServersTestFqdnHostnames-135298658-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-258895278" status: 202 len: 0 microversion: 2.1 time: 0.031224 Oct 15 21:06:38.991165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2703/3817] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 21:06:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-258895278 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:06:39.353199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:39.354775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:39.368342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:39.368844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:39.369413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:39.370289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:39.370753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:39.371225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:40.324509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:40.357233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:40.359406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:40.367401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46ad0f86-5fd7-4106-a7f1-0574422037d0 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 1.018730 Oct 15 21:06:40.373400 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2704/3818] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:39 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 1025 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:40.395941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:40.397790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:40.427586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:40.433494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:40.434013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:40.434849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:40.435413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:40.438063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:41.299255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:41.322924 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e266fdc3-f7a7-46fd-ad6c-d71cfe0e8788 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.945305 Oct 15 21:06:41.332360 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2705/3819] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:40 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 958 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:41.400253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:41.401414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:41.416221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:41.416748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:41.417240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:41.417986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:41.431716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:41.432375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:41.983719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:42.004675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:42.010665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:42.014441 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dcf53c8c-871f-4467-93a7-72cd8fc11c0a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.618782 Oct 15 21:06:42.016805 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2706/3820] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:41 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:06:42.356715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:42.359450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:42.391062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:42.398070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:42.398888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:42.405131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:42.409963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:42.411101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:43.090054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:43.123386 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-59221362-4ef6-4f5b-82f8-8b1222ca63a0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.774608 Oct 15 21:06:43.126415 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2707/3821] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:42 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:06:43.154125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:43.157957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:43.179141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:43.180245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:43.180850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:43.216376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:43.217766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:43.218675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:44.923864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:44.941392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:44.943272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:44.944679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9ba2d45-2c61-4b58-b16b-de5dddb277b7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 1.813562 Oct 15 21:06:44.946584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2708/3822] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:43 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 1819 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:44.962590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:44.969944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:44.971624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Searching by: {'deleted': False, 'project_id': '87739141ec764f6ca347224117dc45c2'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:06:44.986104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:44.986657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:44.987752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:44.989303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:44.989862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:44.990453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:45.040820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:45.045482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:06:45.142921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efff3ecf-618b-4e90-9676-995bb201f1ae tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.183950 Oct 15 21:06:45.149771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2709/3823] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:06:44 2022] GET /compute/v2.1/servers => generated 343 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:45.165884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:45.168175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:45.187661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:45.197084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:45.197874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:45.198902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:45.199606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:45.200265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:46.145791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:46.177096 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad7e6586-e7b6-4ea9-9cfe-18f942de8d2b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 1.026007 Oct 15 21:06:46.188676 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2710/3824] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:45 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 1038 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:46.204031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:46.225388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:46.226240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Searching by: {'deleted': False, 'project_id': '87739141ec764f6ca347224117dc45c2'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:06:46.250071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:46.253013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:46.253509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:46.258832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:46.260912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:46.263495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:46.410794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:06:46.513193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:46.531722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:46.535498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:46.537775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:46.780361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6773b8e8-580e-4cc9-a1d5-f4631820ad0c tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 1603 microversion: 2.1 time: 0.587130 Oct 15 21:06:46.780968 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2711/3825] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:06:46 2022] GET /compute/v2.1/servers/detail => generated 1603 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:46.786646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:46.787455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:46.802526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:46.805700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:46.806427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:46.807392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:46.811715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:46.813985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:47.271162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:47.299388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:47.301422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:47.302779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37e2c3bf-8b68-4970-a7b4-bdb6b64eb1d2 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.520438 Oct 15 21:06:47.303473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2712/3826] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:46 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:47.320163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:47.337484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:47.356422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:47.357073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:47.370119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:47.378593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:47.379097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:47.379552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:47.441791 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] HTTP exception thrown: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Request method failure captured: Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d HTTP/1.1 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-G-TBWUG34joACYA2SU7wY1BfEd6SltpFqjxlW-AK0WU_UDLcnOFPbI77K43CINnKCrKwVa_cm7WdTd3pvebsKWwF_IpNzStt4vKJxLXIF8xBRQbDMIxbP7idnFwZWXzJKo-zsyAyRrlsNGLOfUzN4R0krUIVjpGZqxd_3CE0Y8lw Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.32 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 90ca0b4db2b64900aaa1a4e05dec8a6f Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 90ca0b4db2b64900aaa1a4e05dec8a6f Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TaggedBootDevicesTest-268164479-project Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f00da962dc434ae18a1ab29228846688 Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TaggedBootDevicesTest-268164479-project Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c17ea85b-930c-4f69-bbea-ad8b6dad052d'} Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:47.443480 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Returning 404 to user: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:47.449785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f86b1ba0-92a7-425a-a075-d1acac5206bc tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 404 len: 111 microversion: 2.32 time: 0.129519 Oct 15 21:06:47.451889 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2713/3827] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:06:47 2022] DELETE /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 111 bytes in 147 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:06:47.484188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c6e790d-eef4-425a-965d-eb8aa7f49b95 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:47.509410 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c6e790d-eef4-425a-965d-eb8aa7f49b95 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.048926 Oct 15 21:06:47.519782 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2714/3828] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 21:06:47 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:47.524887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:47.527245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:47.549015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:47.551781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:47.552481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:47.554434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:47.556807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:47.560299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:48.238501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:48.255991 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c3ba382-3f70-485a-b162-e4009f7fee23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.734761 Oct 15 21:06:48.269663 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2715/3829] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:47 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 749 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:48.291983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:48.292668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] [instance: c17ea85b-930c-4f69-bbea-ad8b6dad052d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:48.300187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:48.310724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:48.311244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:48.312365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:48.312881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:48.313978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:48.333550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] HTTP exception thrown: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Request method failure captured: Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d HTTP/1.1 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx-G-TBWUG34joACYA2SU7wY1BfEd6SltpFqjxlW-AK0WU_UDLcnOFPbI77K43CINnKCrKwVa_cm7WdTd3pvebsKWwF_IpNzStt4vKJxLXIF8xBRQbDMIxbP7idnFwZWXzJKo-zsyAyRrlsNGLOfUzN4R0krUIVjpGZqxd_3CE0Y8lw Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.32 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 90ca0b4db2b64900aaa1a4e05dec8a6f Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/90ca0b4db2b64900aaa1a4e05dec8a6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 90ca0b4db2b64900aaa1a4e05dec8a6f Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TaggedBootDevicesTest-268164479 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TaggedBootDevicesTest-268164479-project Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: f00da962dc434ae18a1ab29228846688 Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TaggedBootDevicesTest-268164479-project Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c17ea85b-930c-4f69-bbea-ad8b6dad052d'} Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:06:48.335942 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. Oct 15 21:06:48.340063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:06:48.341738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] Returning 404 to user: Instance c17ea85b-930c-4f69-bbea-ad8b6dad052d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:06:48.343200 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8dea7fa-376e-44f4-8e04-50eebf343897 tempest-TaggedBootDevicesTest-268164479 tempest-TaggedBootDevicesTest-268164479-project] 10.222.0.16 "GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d" status: 404 len: 111 microversion: 2.32 time: 0.071937 Oct 15 21:06:48.344490 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2716/3830] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:48 2022] GET /compute/v2.1/servers/c17ea85b-930c-4f69-bbea-ad8b6dad052d => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:06:48.371317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:48.381450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:48.398716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:48.402530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:48.407863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:48.409317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:48.410497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:48.417841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:48.524905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:06:48.810963 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4a8bc7e-8a3b-4252-af13-1921e8ec66d1 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 204 len: 0 microversion: 2.1 time: 0.462603 Oct 15 21:06:48.811566 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2717/3831] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:06:48 2022] DELETE /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:06:48.833152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:48.833969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:48.852214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:48.852790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:48.853272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:48.853994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:48.869544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:48.870148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:49.613623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:49.633119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:49.635017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:49.646979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfa5a70d-5621-46a1-b4cf-d174e0803a06 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 0.829747 Oct 15 21:06:49.647604 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2718/3832] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:48 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:49.660527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:49.678253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:49.706456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:49.713991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:49.716970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:49.726439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:49.726958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:49.734681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:50.419751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:50.441672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d76574bd-3b4d-46e4-a53b-a89605d1e2fc tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1606 microversion: 2.1 time: 0.791789 Oct 15 21:06:50.442280 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2719/3833] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:49 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1606 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:50.476134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:50.476134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:50.495218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:50.500462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:50.501093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:50.502727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:50.506747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:50.507549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:50.735337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lazy-loading 'services' on Instance uuid d4ea7154-0623-44f0-8bad-076461f12dd5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:06:50.935692 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a695377-bef3-4961-acbb-f12c61ae0ed5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-volume_attachments/f2f40e6b-c045-4ef5-8b41-1c7ccc5cac81" status: 202 len: 0 microversion: 2.1 time: 0.489837 Oct 15 21:06:50.942136 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2720/3834] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:06:50 2022] DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-volume_attachments/f2f40e6b-c045-4ef5-8b41-1c7ccc5cac81 => generated 0 bytes in 499 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:06:50.988293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:50.991167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:51.010248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:51.011759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:51.014249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:51.017320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:51.019376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:51.020661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:51.689090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:51.705078 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5cc3bd0d-e260-4e55-8f65-65484d65292b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.739881 Oct 15 21:06:51.718418 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2721/3835] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:50 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 775 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:51.725761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:51.726776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:51.771214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:51.774151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:51.776416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:51.780796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:51.781657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:51.782318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:52.758818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:52.768572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:52.776930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:52.779228 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0aa8c5f-857e-4a20-a44f-09df8a9e034a tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1738 microversion: 2.20 time: 1.058704 Oct 15 21:06:52.780411 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2722/3836] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:51 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1738 bytes in 1061 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:52.789615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:52.793959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:52.811817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:52.813169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:52.814187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:52.816948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:52.818180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:52.818876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:54.050263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:54.067895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-290051dd-b47b-420b-8554-9ec98284defb tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1606 microversion: 2.1 time: 1.281846 Oct 15 21:06:54.068763 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2723/3837] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:52 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1606 bytes in 1288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:54.073780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01bef95e-9670-40b5-b699-4ce65cd53e18 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:54.097415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01bef95e-9670-40b5-b699-4ce65cd53e18 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:54.097829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01bef95e-9670-40b5-b699-4ce65cd53e18 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:54.098273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01bef95e-9670-40b5-b699-4ce65cd53e18 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:54.146344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01bef95e-9670-40b5-b699-4ce65cd53e18 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:06:54.179639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:54.291084 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01bef95e-9670-40b5-b699-4ce65cd53e18 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action" status: 202 len: 0 microversion: 2.1 time: 0.218300 Oct 15 21:06:54.299110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2724/3838] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:06:54 2022] POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action => generated 0 bytes in 228 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:06:54.330580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:54.333964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:54.358530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:54.360384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:54.362156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:54.364523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:54.366145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:54.367106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:54.894278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:54.903985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b59f0beb-b5b9-4205-9bce-5c76e4d0f19d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.601555 Oct 15 21:06:54.908767 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2725/3839] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:54 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:54.919675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5634c356-8c20-44e8-9d95-ab7b6e1e2f0a req-287bc5dd-dba8-43aa-8d87-4988147532ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "name": "network-vif-unplugged", "status": "completed", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:54.932836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5634c356-8c20-44e8-9d95-ab7b6e1e2f0a req-287bc5dd-dba8-43aa-8d87-4988147532ca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:54.943466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5634c356-8c20-44e8-9d95-ab7b6e1e2f0a req-287bc5dd-dba8-43aa-8d87-4988147532ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:54.945194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5634c356-8c20-44e8-9d95-ab7b6e1e2f0a req-287bc5dd-dba8-43aa-8d87-4988147532ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:54.984142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5634c356-8c20-44e8-9d95-ab7b6e1e2f0a req-287bc5dd-dba8-43aa-8d87-4988147532ca service nova] Creating event network-vif-unplugged:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:06:55.003761 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5634c356-8c20-44e8-9d95-ab7b6e1e2f0a req-287bc5dd-dba8-43aa-8d87-4988147532ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.090222 Oct 15 21:06:55.005950 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2726/3840] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:55.023382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:55.024973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:55.047829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.048359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.052970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.053947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.054480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.054961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.424749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:55.443049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:55.451058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:55.458230 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6be09cb3-ff72-44c0-af61-591dfe0f1ac3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1720 microversion: 2.20 time: 0.450933 Oct 15 21:06:55.460771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2727/3841] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:55 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1720 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:55.485573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:55.486828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:55.499276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.500409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.500985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.502643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.503297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.503881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.726159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:55.739028 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c5a135ec-d610-4805-98e2-d3e40b7812d8 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1677 microversion: 2.1 time: 0.258068 Oct 15 21:06:55.740556 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2728/3842] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:55 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1677 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:55.747557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:55.747557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:55.755868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.756484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.757262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.758182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.759057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.759761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.853073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:55.866442 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-773ea915-6af4-4661-8b33-dc8e73c5983a tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1540 microversion: 2.1 time: 0.124211 Oct 15 21:06:55.867257 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2729/3843] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:55 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1540 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:55.872738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e1d0917a-92e5-45ee-b627-76d41043678a req-4119cbbf-299d-4cd0-82a2-3d40e26aeab6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e98faf5-e97a-430b-9d80-46bfdc6f2e61", "name": "network-vif-deleted", "tag": "c5ab0d40-2dde-4427-834d-04796767878e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:55.904430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e1d0917a-92e5-45ee-b627-76d41043678a req-4119cbbf-299d-4cd0-82a2-3d40e26aeab6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.904920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e1d0917a-92e5-45ee-b627-76d41043678a req-4119cbbf-299d-4cd0-82a2-3d40e26aeab6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.905482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e1d0917a-92e5-45ee-b627-76d41043678a req-4119cbbf-299d-4cd0-82a2-3d40e26aeab6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.926417 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e1d0917a-92e5-45ee-b627-76d41043678a req-4119cbbf-299d-4cd0-82a2-3d40e26aeab6 service nova] Creating event network-vif-deleted:c5ab0d40-2dde-4427-834d-04796767878e for instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 on nch2-master-1 Oct 15 21:06:55.932203 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e1d0917a-92e5-45ee-b627-76d41043678a req-4119cbbf-299d-4cd0-82a2-3d40e26aeab6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063421 Oct 15 21:06:55.933004 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2730/3844] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:55.940697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:55.941515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:55.949907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.950425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.950872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:55.951605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:55.952064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:55.952498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:56.224641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:56.257592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2da6e178-5eeb-444e-82af-4263fa8a4277 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.322933 Oct 15 21:06:56.259585 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2731/3845] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:55 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:56.491751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:56.493585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:56.505276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:56.512748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:56.514158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:56.515471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:56.519834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:56.521888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:56.522595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:56.526706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:06:57.060751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:57.081560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:57.083461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:57.084731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3ed2a21-0afc-4f7e-ba88-727e7af3bde3 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1723 microversion: 2.20 time: 0.598960 Oct 15 21:06:57.088249 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2732/3846] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:06:56 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1723 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:06:57.130243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:57.134501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:57.150827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.152458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.153249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.154792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.157124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.158772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.489488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:57.508358 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a39b1a8-ff35-405f-8490-dd86cb62ccf1 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1677 microversion: 2.1 time: 0.388665 Oct 15 21:06:57.509852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2733/3847] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:57 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1677 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:57.533721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:57.535678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:57.548237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.556640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.557507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.558504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.559191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.560226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.656127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.656649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.657179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.664082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lazy-loading 'fault' on Instance uuid 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:06:57.796461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86129e282ee8469eba032a84e9a55096', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87739141ec764f6ca347224117dc45c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:57.811264 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b9b4f1b-032f-4469-b2e0-e91198d62213 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 200 len: 1257 microversion: 2.1 time: 0.288367 Oct 15 21:06:57.812560 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2734/3848] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:57 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 1257 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:57.822520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0bcaa324-862f-4469-a280-f3033c5c90cb req-a7c2c24a-a622-4594-b93d-ea65766341c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e98faf5-e97a-430b-9d80-46bfdc6f2e61", "name": "network-vif-unplugged", "status": "completed", "tag": "c5ab0d40-2dde-4427-834d-04796767878e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:57.868158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0bcaa324-862f-4469-a280-f3033c5c90cb req-a7c2c24a-a622-4594-b93d-ea65766341c2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.868749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0bcaa324-862f-4469-a280-f3033c5c90cb req-a7c2c24a-a622-4594-b93d-ea65766341c2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.869239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0bcaa324-862f-4469-a280-f3033c5c90cb req-a7c2c24a-a622-4594-b93d-ea65766341c2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.899662 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0bcaa324-862f-4469-a280-f3033c5c90cb req-a7c2c24a-a622-4594-b93d-ea65766341c2 service nova] Creating event network-vif-unplugged:c5ab0d40-2dde-4427-834d-04796767878e for instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 on nch2-master-1 Oct 15 21:06:57.910195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0bcaa324-862f-4469-a280-f3033c5c90cb req-a7c2c24a-a622-4594-b93d-ea65766341c2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.095054 Oct 15 21:06:57.911473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2735/3849] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:06:57 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:06:57.917877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "692c92a4-9e08-4b5b-84af-eee3e771229b"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:06:57.921169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:57.928592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.929269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.929812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:57.930762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:57.933315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:57.933974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:58.514050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c5fcdad-54d9-4a89-b010-fcac3aac4a1f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.601247 Oct 15 21:06:58.524890 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2736/3850] 10.222.0.16 () {62 vars in 1364 bytes} [Sat Oct 15 21:06:57 2022] POST /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-volume_attachments => generated 188 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:06:58.538062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:58.540089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:58.561922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:58.566901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:58.568973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:58.571148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:58.575606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:58.575606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:58.974356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:58.996922 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3bdbd252-8d3f-4afa-9d79-2756d4d6f0d9 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.465739 Oct 15 21:06:59.000364 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2737/3851] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:58 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:59.011612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:59.012270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:59.031622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:59.032200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:59.032714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:59.035604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:59.036751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:59.037237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:59.481022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:06:59.495058 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ff6daed-2ca4-4e37-a6a1-029a0fce8e34 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1677 microversion: 2.1 time: 0.490525 Oct 15 21:06:59.496049 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2738/3852] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:06:58 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1677 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:06:59.513243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:06:59.514828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:06:59.534457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:59.534905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:59.535517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:59.536415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:06:59.537018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:06:59.537609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:06:59.719796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=5,nova_cell1:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:00.061347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36cae914-0203-4223-8ec3-51015f3eba50 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-volume_attachments/692c92a4-9e08-4b5b-84af-eee3e771229b" status: 202 len: 0 microversion: 2.20 time: 0.559796 Oct 15 21:07:00.063279 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2739/3853] 10.222.0.16 () {60 vars in 1457 bytes} [Sat Oct 15 21:06:59 2022] DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-volume_attachments/692c92a4-9e08-4b5b-84af-eee3e771229b => generated 0 bytes in 563 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:00.082358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:00.083085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] [instance: 7e98faf5-e97a-430b-9d80-46bfdc6f2e61] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:00.120477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:00.121267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:00.121993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:00.125831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:00.126319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:00.134305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:00.183697 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] HTTP exception thrown: Instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 could not be found. Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Request method failure captured: Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 HTTP/1.1 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCNV37070TEULZah3YEQqnKjTAav11UdqiXO-o6dPCzwtuc4cv_5Nzd1FdBEZFumlrRTsH4dQdBJp5HrrBkRLjEPBzLUEq2O6pSPLiM_WZDr7SmHP9CqhL3a5CUR7m7Us16ArlCN-693eN0LXrKZaR7ETl8NxsuynRFHi9K685LyGY Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87739141ec764f6ca347224117dc45c2 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersTestBootFromVolume-1963301208 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87739141ec764f6ca347224117dc45c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87739141ec764f6ca347224117dc45c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87739141ec764f6ca347224117dc45c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87739141ec764f6ca347224117dc45c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersTestBootFromVolume-1963301208 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87739141ec764f6ca347224117dc45c2 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersTestBootFromVolume-1963301208 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersTestBootFromVolume-1963301208-project Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 86129e282ee8469eba032a84e9a55096 Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersTestBootFromVolume-1963301208-project Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 could not be found. Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '7e98faf5-e97a-430b-9d80-46bfdc6f2e61'} Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 could not be found. Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 could not be found. Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:00.185662 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 could not be found. Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Returning 404 to user: Instance 7e98faf5-e97a-430b-9d80-46bfdc6f2e61 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:00.190998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75d4994d-61f7-4460-837e-c372aac16d73 tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61" status: 404 len: 111 microversion: 2.1 time: 0.122746 Oct 15 21:07:00.210459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2740/3854] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:00 2022] GET /compute/v2.1/servers/7e98faf5-e97a-430b-9d80-46bfdc6f2e61 => generated 111 bytes in 146 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:00.230606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:00.231252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:00.248665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:00.249190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:00.249628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:00.260920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:00.261466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:00.261917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:00.889132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:00.909830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da67855f-4186-4ea8-bc99-822842062460 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.697588 Oct 15 21:07:00.914122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2741/3855] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:00 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:00.920057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:00.931999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:00.950562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:00.951058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:00.951500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:00.952220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:00.952985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:00.953651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:01.006063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:01.009005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=138688) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 21:07:01.167989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7521bc86-d028-45a9-81f4-5a5619e10e3f tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "POST /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/action" status: 202 len: 0 microversion: 2.20 time: 0.250364 Oct 15 21:07:01.177841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2742/3856] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 21:07:00 2022] POST /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/action => generated 0 bytes in 263 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:01.207154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99414154-5a42-4c44-b3fe-d56ab388912d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] Calling method '.version_select at 0x7fe74e324a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:01.259779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99414154-5a42-4c44-b3fe-d56ab388912d tempest-ServersTestBootFromVolume-1963301208 tempest-ServersTestBootFromVolume-1963301208-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2028445312" status: 202 len: 0 microversion: 2.1 time: 0.074249 Oct 15 21:07:01.260812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2743/3857] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:07:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2028445312 => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:07:01.285518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:01.290370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:01.348458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:01.349357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:01.350046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:01.350777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:01.351343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:01.351882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:02.703310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:02.713740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d446f73-9d94-46ea-bf72-f22be96723e4 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1677 microversion: 2.1 time: 1.451627 Oct 15 21:07:02.732365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2744/3858] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:01 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1677 bytes in 1471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:02.738170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:02.739043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:02.749635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:02.750720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:02.751773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:02.752574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:02.754036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:02.754892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:03.419878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:03.431637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:03.434372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:03.436428 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d7c7cf9a-d22d-4165-a202-220f96e114f8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1671 microversion: 2.20 time: 0.702806 Oct 15 21:07:03.437948 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2745/3859] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:02 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1671 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:03.453171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-49de0d75-8615-4898-b308-4d3ac26fd038 req-944a5e91-a1d5-448e-bf3b-011bf04beccb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f318670-01e5-4565-8e32-ac7ef73ba33c", "name": "network-vif-plugged", "status": "completed", "tag": "b779e022-9b5e-4fc4-8c92-ae2e834e6c81"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:03.472133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-49de0d75-8615-4898-b308-4d3ac26fd038 req-944a5e91-a1d5-448e-bf3b-011bf04beccb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:03.472979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-49de0d75-8615-4898-b308-4d3ac26fd038 req-944a5e91-a1d5-448e-bf3b-011bf04beccb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:03.473490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-49de0d75-8615-4898-b308-4d3ac26fd038 req-944a5e91-a1d5-448e-bf3b-011bf04beccb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:03.485790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-49de0d75-8615-4898-b308-4d3ac26fd038 req-944a5e91-a1d5-448e-bf3b-011bf04beccb service nova] Creating event network-vif-plugged:b779e022-9b5e-4fc4-8c92-ae2e834e6c81 for instance 5f318670-01e5-4565-8e32-ac7ef73ba33c on nch1-master-1 Oct 15 21:07:03.491376 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-49de0d75-8615-4898-b308-4d3ac26fd038 req-944a5e91-a1d5-448e-bf3b-011bf04beccb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052009 Oct 15 21:07:03.491993 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2746/3860] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:03.498283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:03.499010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:03.514827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:03.515318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:03.515938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:03.516840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:03.518076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:03.518511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:04.087484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:04.096569 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e906a5a7-fa52-46c0-b543-f5ec0bfcaa0d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.603188 Oct 15 21:07:04.097499 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2747/3861] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:03 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:04.102236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:04.103052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:04.109272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:04.109953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:04.111103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:04.111979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:04.112600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:04.113235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:04.943095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:04.956624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-414b905c-4cf7-446d-b786-9e2a7159597e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1677 microversion: 2.1 time: 0.857518 Oct 15 21:07:04.957388 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2748/3862] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:04 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1677 bytes in 859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:04.966417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:04.967849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:04.979541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:04.980259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:04.980859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:04.981756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:04.982368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:04.982952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:05.486179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:05.500508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:05.502755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:05.504349 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c685c25a-854d-4ebe-9f7f-0d99ace3d627 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.545452 Oct 15 21:07:05.522667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2749/3863] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:04 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:05.531750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:05.534072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:05.540664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:05.541498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:05.543143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:05.544490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:05.545475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:05.546090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:05.965637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:05.979669 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea343444-0016-4817-bea7-a1241ef5eeaf tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1518 microversion: 2.1 time: 0.455713 Oct 15 21:07:05.980616 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2750/3864] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:05 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1518 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:06.011586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:06.012665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:06.021451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:06.022292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:06.023068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:06.023978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:06.024625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:06.026315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:06.540966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:06.557341 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cad5b6a4-9bb8-4123-952d-4586daeac65e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1654 microversion: 2.1 time: 0.552341 Oct 15 21:07:06.563011 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2751/3865] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:05 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1654 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:06.716697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-785417611", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7cae7909-1e8f-4b21-a25f-4a74daa4a2c1"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:06.855511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] validate_networks() for [('7cae7909-1e8f-4b21-a25f-4a74daa4a2c1', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:07:07.950818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:07.951371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:07.952024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:07.952589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:07.953013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:07.971493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:07.989770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:07.990697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:07.993499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:07.994160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:07.997345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:08.068935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Getting quotas for project 96091c3dac03410dbb4f59faa1482781. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:08.104922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Getting quotas for user dd615ca74887465b8e21b7f4011fb53f and project 96091c3dac03410dbb4f59faa1482781. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:08.162885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:07:08.167746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:07:08.292107 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-76dc034a-475f-43c5-9f9e-5d16b6b9470c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.727232 Oct 15 21:07:08.311514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2752/3866] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 1748 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:07:08.326399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:08.329826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:08.351193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:08.354651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:08.355944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:08.358571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:08.361740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:08.363643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:08.902184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:08.917725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:08.927025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:08.929057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5863f11a-a3be-4ce6-a4a2-0981d0226656 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.616195 Oct 15 21:07:08.931218 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2753/3867] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:08 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:08.941149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d99fb616-0aa9-4db6-aecf-8c0fcb075507 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:08.962479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d99fb616-0aa9-4db6-aecf-8c0fcb075507 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:08.963022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d99fb616-0aa9-4db6-aecf-8c0fcb075507 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:08.963399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d99fb616-0aa9-4db6-aecf-8c0fcb075507 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:09.093493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d99fb616-0aa9-4db6-aecf-8c0fcb075507 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Going to try to start instance {{(pid=138688) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 21:07:09.260356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=4,nova_cell1:UPDATE=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:09.286972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d99fb616-0aa9-4db6-aecf-8c0fcb075507 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action" status: 202 len: 0 microversion: 2.1 time: 0.348337 Oct 15 21:07:09.290514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2754/3868] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:08 2022] POST /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/action => generated 0 bytes in 359 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:07:09.309412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:09.316981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:09.362546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:09.364061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:09.365697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:09.370108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:09.376202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:09.380995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:12.068526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:12.090379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e89e8070-5d5c-40da-bdf7-669e26fb71ff tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 2.786718 Oct 15 21:07:12.091626 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2755/3869] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:09 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 2800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:12.099722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:12.100375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:12.116352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:12.123234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:12.123870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:12.124765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:12.125398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:12.126042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:12.681231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:12.691639 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2eda053c-4462-4456-8154-9e79f096634a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1275 microversion: 2.1 time: 0.598644 Oct 15 21:07:12.701141 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2756/3870] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:12 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1275 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:12.711901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:12.712523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:12.744668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:12.745198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:12.745670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:12.746349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:12.755958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:12.756655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:13.845302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:13.857214 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f93daed0-22fa-4a15-ac4a-0929f7dd4850 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1663 microversion: 2.1 time: 1.150594 Oct 15 21:07:13.861645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2757/3871] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:12 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1663 bytes in 1158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:13.886310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:13.888257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:13.920371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:13.925761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:13.926539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:13.927454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:13.928070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:13.928665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:15.048472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:15.059130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:15.060647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:15.061872 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b830cdef-9524-4857-af22-1cfdb1d70560 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 1.199079 Oct 15 21:07:15.078236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2758/3872] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:13 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 1216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:15.105336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:15.106500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:15.121929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:15.122449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:15.123107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:15.123851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:15.128117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:15.128657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:16.853443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:16.890547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de965067-a53c-4b96-aca3-d003cbd1e72d tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 1.810977 Oct 15 21:07:16.891704 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2759/3873] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:15 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 1813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:17.092069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSON-server-862765880", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:17.205077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] validate_networks() for [('114bda92-662d-4788-82b0-9f1053954aa2', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:07:18.034195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:18.149191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:18.796228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:18.796988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:18.798893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:18.801680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:18.802191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:18.819542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:18.820045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:18.820489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:18.821343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:18.821884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:18.822428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:18.869336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Getting quotas for project 5572b130042c482384a1a06e6a91c784. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:18.877732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Getting quotas for user 461a5c1e334a47eb9d70908227cca0b2 and project 5572b130042c482384a1a06e6a91c784. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:18.905713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:07:18.911127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:07:18.992894 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c83b27f0-3315-46ee-ba4d-ac51fdee160d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.099921 Oct 15 21:07:18.996276 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2760/3874] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:16 2022] POST /compute/v2.1/servers => generated 383 bytes in 2104 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:07:19.007418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:19.008466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:19.022146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:19.026812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:19.027468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:19.031245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:19.032122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:19.032865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:19.645547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:19.659407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d783d412-fce9-4b9d-9ca8-a34ff57339f2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.661461 Oct 15 21:07:19.690807 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2761/3875] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:18 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:19.704897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:19.706117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:19.719927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:19.720718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:19.721583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:19.722639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:19.723248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:19.724397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.053335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.072929 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c189eeb0-fec3-429a-9b2b-9e9f30907fc8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1317 microversion: 2.1 time: 0.380562 Oct 15 21:07:20.076203 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2762/3876] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:19 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1317 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:20.084348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:20.085311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:20.091947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.092707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.093394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.094429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.095865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.096390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.341701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.353272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e9f9ec2-8345-4ca4-b121-879ec2b53cca tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1667 microversion: 2.1 time: 0.273748 Oct 15 21:07:20.355159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2763/3877] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:20 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1667 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:20.361086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:20.362245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:20.374484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.375414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.376679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.378040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.378928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.380269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.617864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.629806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.631689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.632882 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6395332-1ff2-4a1f-9005-c498c49d2005 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.276577 Oct 15 21:07:20.634283 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2764/3878] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:20 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:20.639110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:20.639728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:20.646735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.648515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.649136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.650008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.651084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.651729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.829685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.840288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51f53838-af5c-4179-bc54-897f9ed73608 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.204942 Oct 15 21:07:20.840888 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2765/3879] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:20 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:20.854037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:20.855040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:20.862398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.862992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.863321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.864232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.865118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.866060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.939119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:20.954424 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6cf22974-7e53-4a95-8632-caf011ef4fb8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1217 microversion: 2.1 time: 0.111776 Oct 15 21:07:20.955085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2766/3880] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:20 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1217 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:20.963770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:20.964496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:20.971477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.972031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.972411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:20.973271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:20.973868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:20.974417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.135551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:21.143996 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8cf25c65-8ae2-4532-af90-0ae71d1f1c53 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.187948 Oct 15 21:07:21.144832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2767/3881] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:20 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:21.153077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1612a529-129e-459b-b080-10ed3184ac3e req-51dff4e6-2aae-4650-b7b3-1cdedda604fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "18be360f-e339-42ec-89ff-72acf428932a", "tag": "83f1bf1e-ab89-427f-af3d-df7b7e6537b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:21.170662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1612a529-129e-459b-b080-10ed3184ac3e req-51dff4e6-2aae-4650-b7b3-1cdedda604fd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.171142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1612a529-129e-459b-b080-10ed3184ac3e req-51dff4e6-2aae-4650-b7b3-1cdedda604fd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.171576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1612a529-129e-459b-b080-10ed3184ac3e req-51dff4e6-2aae-4650-b7b3-1cdedda604fd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.186605 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1612a529-129e-459b-b080-10ed3184ac3e req-51dff4e6-2aae-4650-b7b3-1cdedda604fd service nova] Creating event network-changed:83f1bf1e-ab89-427f-af3d-df7b7e6537b8 for instance 18be360f-e339-42ec-89ff-72acf428932a on nch1-master-1 Oct 15 21:07:21.191806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1612a529-129e-459b-b080-10ed3184ac3e req-51dff4e6-2aae-4650-b7b3-1cdedda604fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045669 Oct 15 21:07:21.192411 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2768/3882] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:21.197791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa1dcf7-652d-45db-b3ec-e7d091f38a47 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:21.204888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0aa1dcf7-652d-45db-b3ec-e7d091f38a47 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc has no mapping to a cell. Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa1dcf7-652d-45db-b3ec-e7d091f38a47 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc/action HTTP/1.1 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: {"os-stop": {}} Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc has no mapping to a cell. Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc', 'body': {'os-stop': {}}} Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1411, in _get_instance Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi mapping = objects.InstanceMapping.get_by_instance_uuid( Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_mapping.py", line 112, in get_by_instance_uuid Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_instance_uuid_from_db(context, instance_uuid) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_mapping.py", line 106, in _get_by_instance_uuid_from_db Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceMappingNotFound(uuid=instance_uuid) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceMappingNotFound: Instance f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc has no mapping to a cell. Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1444, in _stop_server Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, id) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1418, in _get_instance Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:21.217312 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc has no mapping to a cell. Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc has no mapping to a cell. Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0aa1dcf7-652d-45db-b3ec-e7d091f38a47 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc has no mapping to a cell. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0aa1dcf7-652d-45db-b3ec-e7d091f38a47 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc/action" status: 404 len: 117 microversion: 2.1 time: 0.025398 Oct 15 21:07:21.222877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2769/3883] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:21 2022] POST /compute/v2.1/servers/f3d77f91-b9aa-4f9d-ad0c-55d23ce3e7cc/action => generated 117 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:21.224742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:21.225482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:21.233342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.233842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.234481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.235906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.236535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.236975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.397090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:21.406511 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7fe5c345-d7f0-4af6-8160-abf70bfc1931 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1317 microversion: 2.1 time: 0.185216 Oct 15 21:07:21.408457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2770/3884] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:21 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1317 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:21.421648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:21.422382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:21.428979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.429688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.430138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.431128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.431720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.432168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.691994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:21.699888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3557aceb-8958-445b-9a57-6beeeb94b02f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.290103 Oct 15 21:07:21.700473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2771/3885] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:21 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:21.877274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-463fe981-ed32-464b-ab80-2711f9875f4a tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-925350928", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 3, "max_count": 2}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:21.883418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-463fe981-ed32-464b-ab80-2711f9875f4a tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] HTTP exception thrown: min_count must be <= max_count Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-463fe981-ed32-464b-ab80-2711f9875f4a tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Request method failure captured: Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 183 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJ0Ey1zwtIFRCzHLOJZfu3qZOQpLy-6bWkh59xU59ZZ8C8NqEdAp3VuA80hLmFCcCOnYkntS5vJQ9HqKTRdVNb8EuII_af2p90LwstS0DaWqsk3c-RYF6N4i9rn908gqIT_bsYiwVCLJIe1d6Tkqhri5zSP3WzjC7V4ZKOG8lYN2U Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5d4c5f85b7e4a09b56a423e42274a79 Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-925350928", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 3, "max_count": 2}} Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: exception: min_count must be <= max_count Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-925350928', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'min_count': 3, 'max_count': 2}}} Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 726, in create Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: min_count must be <= max_count Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: min_count must be <= max_count Oct 15 21:07:21.885430 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:21.889844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-463fe981-ed32-464b-ab80-2711f9875f4a tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Returning 400 to user: min_count must be <= max_count {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:21.889844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-463fe981-ed32-464b-ab80-2711f9875f4a tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 74 microversion: 2.1 time: 0.185553 Oct 15 21:07:21.914487 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2772/3886] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:21 2022] POST /compute/v2.1/servers => generated 74 bytes in 212 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Oct 15 21:07:21.919330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:21.919946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:21.928216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.929686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.930412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:21.931169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:21.932020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:21.932626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.138851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:22.148293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d86b682f-4a58-4df0-a2c0-9fb8e830084e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.232681 Oct 15 21:07:22.161136 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2773/3887] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:21 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:22.206312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV247Test-server-2125481783", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:22.261373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:22.261751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:22.262233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:22.262646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:22.263061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:22.272613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.273133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.273581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.274325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.274787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.275226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.292670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Getting quotas for project 8c7c33c41d0a4f96bd720b6235d0acad. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:22.294969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:22.299319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Getting quotas for user 508ea6505b99421d8d65a42f1b6b3876 and project 8c7c33c41d0a4f96bd720b6235d0acad. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:22.314325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:07:22.316632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: 1d88df21-cddf-4cef-a308-055d6727397d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:07:22.361663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0539997-fccc-4469-97c8-1f6f2135b66f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.211294 Oct 15 21:07:22.362514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2774/3888] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:07:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 212 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:07:22.367111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:22.367735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:22.374595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.375081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.376061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.376766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.377278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.377712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.399172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:22.538326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:22.546907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:22.549021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:22.550202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8ba5b6a2-5245-4492-b3a7-6da8e3c46bf9 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.186956 Oct 15 21:07:22.550762 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2775/3889] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:22 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:22.555417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-1-242837089"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:22.557976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:22.565090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.565585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.566031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.566734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.567196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.567624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.779308 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4adb9678-cb95-4172-833e-cc1488b44fc0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.227322 Oct 15 21:07:22.783991 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2776/3890] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:22 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 229 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Oct 15 21:07:22.786062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5dbbad2-c140-44ed-9103-2e40a0657e26 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-657193198", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "max_count": 0}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5dbbad2-c140-44ed-9103-2e40a0657e26 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Request method failure captured: Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 167 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJ0Ey1zwtIFRCzHLOJZfu3qZOQpLy-6bWkh59xU59ZZ8C8NqEdAp3VuA80hLmFCcCOnYkntS5vJQ9HqKTRdVNb8EuII_af2p90LwstS0DaWqsk3c-RYF6N4i9rn908gqIT_bsYiwVCLJIe1d6Tkqhri5zSP3WzjC7V4ZKOG8lYN2U Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5d4c5f85b7e4a09b56a423e42274a79 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-657193198", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "max_count": 0}} Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-657193198', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'max_count': 0}}} Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['server']['properties']['max_count']: Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minimum': 1, Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 0 Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:07:22.790101 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5dbbad2-c140-44ed-9103-2e40a0657e26 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Returning 400 to user: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d5dbbad2-c140-44ed-9103-2e40a0657e26 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 130 microversion: 2.1 time: 0.010298 Oct 15 21:07:22.793997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2777/3891] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:22 2022] POST /compute/v2.1/servers => generated 130 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:07:22.798652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:22.799648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:22.811736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.812239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.812678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.814206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.814671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.815104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.880664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:22.888616 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9e217a2-2a43-49f3-bc69-2004ff6c32b0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1273 microversion: 2.1 time: 0.095548 Oct 15 21:07:22.889209 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2778/3892] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:22 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1273 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:22.894693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:22.895550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:22.901843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.902343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.902785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:22.903487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:22.903951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:22.904385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.042572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:23.051113 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8c03db7a-4bd2-4d4c-b0b6-868ff59539f5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.160827 Oct 15 21:07:23.051912 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2779/3893] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:22 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:23.057303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:23.057979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: 1d88df21-cddf-4cef-a308-055d6727397d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:23.065558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.066394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.066987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.067870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.068337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.068769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.138030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:23.148721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:23.151034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:23.152358 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a6fcafb-a4b3-49ba-b2e3-558dcd90b0f4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d" status: 200 len: 1330 microversion: 2.47 time: 0.099167 Oct 15 21:07:23.153027 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2780/3894] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:23 2022] GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d => generated 1330 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:23.159276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:23.160040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:23.166259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.166942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.167767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.168690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.169323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.169921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.322752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:23.330753 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d958c80-ff1d-4a98-83fe-34acae2a39c8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1317 microversion: 2.1 time: 0.176644 Oct 15 21:07:23.331330 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2781/3895] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:23 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1317 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:23.335961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd68232c-ec20-41d1-8146-b807f18703cc tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-588628958", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "max_count": 2.5}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd68232c-ec20-41d1-8146-b807f18703cc tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Request method failure captured: Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 169 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJ0Ey1zwtIFRCzHLOJZfu3qZOQpLy-6bWkh59xU59ZZ8C8NqEdAp3VuA80hLmFCcCOnYkntS5vJQ9HqKTRdVNb8EuII_af2p90LwstS0DaWqsk3c-RYF6N4i9rn908gqIT_bsYiwVCLJIe1d6Tkqhri5zSP3WzjC7V4ZKOG8lYN2U Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5d4c5f85b7e4a09b56a423e42274a79 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-588628958", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "max_count": 2.5}} Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-588628958', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'max_count': 2.5}}} Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 2.5 is not of type 'integer', 'string' Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['max_count']: Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minimum': 1, Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 2.5 Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:07:23.340043 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd68232c-ec20-41d1-8146-b807f18703cc tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Returning 400 to user: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:23.343503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd68232c-ec20-41d1-8146-b807f18703cc tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.009186 Oct 15 21:07:23.349290 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2782/3896] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:23 2022] POST /compute/v2.1/servers => generated 139 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:07:23.354310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05055ad7-9a0a-4a65-8afc-21f89215fdf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:23.354763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-05055ad7-9a0a-4a65-8afc-21f89215fdf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: ef793b11-b893-401e-a870-c605e545fc36] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:23.361743 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-05055ad7-9a0a-4a65-8afc-21f89215fdf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance ef793b11-b893-401e-a870-c605e545fc36 could not be found. Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05055ad7-9a0a-4a65-8afc-21f89215fdf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/ef793b11-b893-401e-a870-c605e545fc36/action HTTP/1.1 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: {"suspend": {}} Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ef793b11-b893-401e-a870-c605e545fc36 could not be found. Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ef793b11-b893-401e-a870-c605e545fc36', 'body': {'suspend': {}}} Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ef793b11-b893-401e-a870-c605e545fc36 could not be found. Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 35, in _suspend Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, id) Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ef793b11-b893-401e-a870-c605e545fc36 could not be found. Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:23.363788 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ef793b11-b893-401e-a870-c605e545fc36 could not be found. Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05055ad7-9a0a-4a65-8afc-21f89215fdf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance ef793b11-b893-401e-a870-c605e545fc36 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-05055ad7-9a0a-4a65-8afc-21f89215fdf8 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/ef793b11-b893-401e-a870-c605e545fc36/action" status: 404 len: 111 microversion: 2.1 time: 0.016067 Oct 15 21:07:23.369332 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2783/3897] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:23 2022] POST /compute/v2.1/servers/ef793b11-b893-401e-a870-c605e545fc36/action => generated 111 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:23.372437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV247Test-server-294120653", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": "none"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:23.428933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:23.429392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:23.429872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:23.430286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:23.430704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:23.440194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.440673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.441138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.441867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.442323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.442747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.460166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Getting quotas for project 8c7c33c41d0a4f96bd720b6235d0acad. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:23.465487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Getting quotas for user 508ea6505b99421d8d65a42f1b6b3876 and project 8c7c33c41d0a4f96bd720b6235d0acad. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:23.477779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:07:23.480070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:07:23.538832 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7d77606-4f3b-44a9-982d-a440249b9877 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.170380 Oct 15 21:07:23.539486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2784/3898] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:07:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 172 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:07:23.544100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:23.544719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:23.551345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.551830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.552271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.553059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.553457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.553888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.740955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:23.749134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47a3a0ca-95ad-420e-832b-589199c8290f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.208600 Oct 15 21:07:23.749835 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2785/3899] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:23 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:23.770056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71b95af7-1f60-48fb-a232-3a72887b597b tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-171646432", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 0}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71b95af7-1f60-48fb-a232-3a72887b597b tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Request method failure captured: Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 167 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJ0Ey1zwtIFRCzHLOJZfu3qZOQpLy-6bWkh59xU59ZZ8C8NqEdAp3VuA80hLmFCcCOnYkntS5vJQ9HqKTRdVNb8EuII_af2p90LwstS0DaWqsk3c-RYF6N4i9rn908gqIT_bsYiwVCLJIe1d6Tkqhri5zSP3WzjC7V4ZKOG8lYN2U Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5d4c5f85b7e4a09b56a423e42274a79 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-171646432", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 0}} Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-171646432', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'min_count': 0}}} Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['server']['properties']['min_count']: Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minimum': 1, Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 0 Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:07:23.776604 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Oct 15 21:07:23.780684 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:23.785434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71b95af7-1f60-48fb-a232-3a72887b597b tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Returning 400 to user: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:23.786479 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71b95af7-1f60-48fb-a232-3a72887b597b tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 130 microversion: 2.1 time: 0.023586 Oct 15 21:07:23.787231 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2786/3900] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:23 2022] POST /compute/v2.1/servers => generated 130 bytes in 37 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:07:23.791669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fcecc84-2a70-4607-b0b9-0ca89d1c01e3 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Action: 'update', calling method: >, body: {"server": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:23.794098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9fcecc84-2a70-4607-b0b9-0ca89d1c01e3 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:23.854434 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fcecc84-2a70-4607-b0b9-0ca89d1c01e3 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "PUT /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 991 microversion: 2.47 time: 0.066233 Oct 15 21:07:23.855446 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2787/3901] 10.222.0.16 () {62 vars in 1297 bytes} [Sat Oct 15 21:07:23 2022] PUT /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 991 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:07:23.868949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:23.870250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:23.879433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.880107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.880705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:23.881637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:23.882105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:23.882545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.023103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.031732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.033700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.034879 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2e873f5-6b4c-428c-bf65-db747b4b4f73 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.172220 Oct 15 21:07:24.035473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2788/3902] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:23 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:24.040200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:24.040813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:24.046740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.047382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.048089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.048872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.049365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.049835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.182004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.190846 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4857f730-00af-49cf-81d5-1e69a150545e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.154420 Oct 15 21:07:24.191414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2789/3903] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:24 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:24.196261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:24.196867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:24.207218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.207718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.208165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.208896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.209371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.209808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.274962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.283277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.284769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.285933 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e708611-ee33-4bf2-8402-6869346500d2 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1281 microversion: 2.47 time: 0.093345 Oct 15 21:07:24.286501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2790/3904] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:24 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1281 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:24.290842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:24.291468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:24.297115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.298213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.298662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.299361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.299825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.300257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.367269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.375069 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd54c492-49ec-4076-ad5f-3b91df539227 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1273 microversion: 2.1 time: 0.087577 Oct 15 21:07:24.376248 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2791/3905] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:24 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1273 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:24.381609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6e43480-2524-486b-97d0-a030998a2e42 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1386365289", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 2.5}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6e43480-2524-486b-97d0-a030998a2e42 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Request method failure captured: Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 170 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJ0Ey1zwtIFRCzHLOJZfu3qZOQpLy-6bWkh59xU59ZZ8C8NqEdAp3VuA80hLmFCcCOnYkntS5vJQ9HqKTRdVNb8EuII_af2p90LwstS0DaWqsk3c-RYF6N4i9rn908gqIT_bsYiwVCLJIe1d6Tkqhri5zSP3WzjC7V4ZKOG8lYN2U Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8fbc8a431f1249c3a7fa3b33e2990c39"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8fbc8a431f1249c3a7fa3b33e2990c39 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-655419863 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c5d4c5f85b7e4a09b56a423e42274a79 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-655419863-project Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1386365289", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "min_count": 2.5}} Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-1386365289', 'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9', 'flavorRef': '452', 'min_count': 2.5}}} Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 2.5 is not of type 'integer', 'string' Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['min_count']: Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'minimum': 1, Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 2.5 Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:07:24.385557 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6e43480-2524-486b-97d0-a030998a2e42 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] Returning 400 to user: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6e43480-2524-486b-97d0-a030998a2e42 tempest-MultipleCreateNegativeTestJSON-655419863 tempest-MultipleCreateNegativeTestJSON-655419863-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.008903 Oct 15 21:07:24.389249 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2792/3906] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:24 2022] POST /compute/v2.1/servers => generated 139 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:07:24.394882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:24.395444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:24.402492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.402989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.403430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.404135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.404601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.405108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.429436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lazy-loading 'flavor' on Instance uuid 5f318670-01e5-4565-8e32-ac7ef73ba33c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:07:24.502712 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7f1ebce-884b-41d7-a309-a525ab90cdb5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 202 len: 0 microversion: 2.1 time: 0.114157 Oct 15 21:07:24.503513 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2793/3907] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:24 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 0 bytes in 115 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:07:24.508792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:24.509517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:24.515552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.516544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.516999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.517707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.518160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.518583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.845449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:24.856600 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3a31299-cdfc-4bff-b3d2-84fa8c4c376f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1317 microversion: 2.1 time: 0.350303 Oct 15 21:07:24.870657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2794/3908] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:24 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1317 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:24.870657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:24.870657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:24.888654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.889338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.889866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:24.890584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:24.891260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:24.891985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.158673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.166243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:25.169663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ff4b9380-e266-4f43-960a-e556f594b997 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1513 microversion: 2.1 time: 0.310871 Oct 15 21:07:25.170571 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2795/3909] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:24 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1513 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:25.176874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:25.177913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:25.186380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.187071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.187666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.189122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.190057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.190651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.771498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.787305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.788704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.789918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b622a3f-6950-4088-9d7f-b1c02e92cdbb tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.618137 Oct 15 21:07:25.805500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2796/3910] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:25 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:25.814490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:25.815177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:25.821867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.822024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.822465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.823234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.823703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.824131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.913397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.926891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.928917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:25.930922 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfd0ce74-ea76-4741-bea7-eacb0b9df3cd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1335 microversion: 2.47 time: 0.124347 Oct 15 21:07:25.932467 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2797/3911] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:25 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1335 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:25.942283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-844fb488-1a68-4b6e-b1e0-68b1802e3a04 req-c71acae4-8378-49b2-832f-36d64e245718 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "18be360f-e339-42ec-89ff-72acf428932a", "name": "network-vif-plugged", "status": "completed", "tag": "83f1bf1e-ab89-427f-af3d-df7b7e6537b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:25.953986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-844fb488-1a68-4b6e-b1e0-68b1802e3a04 req-c71acae4-8378-49b2-832f-36d64e245718 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.955109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-844fb488-1a68-4b6e-b1e0-68b1802e3a04 req-c71acae4-8378-49b2-832f-36d64e245718 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.955575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-844fb488-1a68-4b6e-b1e0-68b1802e3a04 req-c71acae4-8378-49b2-832f-36d64e245718 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.967676 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-844fb488-1a68-4b6e-b1e0-68b1802e3a04 req-c71acae4-8378-49b2-832f-36d64e245718 service nova] Creating event network-vif-plugged:83f1bf1e-ab89-427f-af3d-df7b7e6537b8 for instance 18be360f-e339-42ec-89ff-72acf428932a on nch1-master-1 Oct 15 21:07:25.974106 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-844fb488-1a68-4b6e-b1e0-68b1802e3a04 req-c71acae4-8378-49b2-832f-36d64e245718 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040054 Oct 15 21:07:25.975449 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2798/3912] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:25.981089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:25.981554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:25.988770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.989288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.989732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:25.990757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:25.991135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:25.991565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.079064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:26.088147 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5985f55c-a07b-4cfe-b134-f6327183872a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1271 microversion: 2.1 time: 0.111504 Oct 15 21:07:26.101493 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2799/3913] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:25 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1271 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:26.101493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:26.101493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:26.109863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.110368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.110812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.111530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.111985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.112592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.434141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:26.445444 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46390b7a-4c08-4c9b-b1cd-67cc007d0dc6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1317 microversion: 2.1 time: 0.355226 Oct 15 21:07:26.446164 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2800/3914] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:26 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1317 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:26.451154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:26.451983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:26.463306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.463908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.464537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.465283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.465948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.466307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.621871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:26.631057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-830e9e18-083a-4702-9e93-88f8e6741b48 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1496 microversion: 2.1 time: 0.183614 Oct 15 21:07:26.633399 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2801/3915] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:26 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1496 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:26.656201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:26.657581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:26.664647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.665255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.665759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.666729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.667263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.668482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.700358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lazy-loading 'flavor' on Instance uuid 5f318670-01e5-4565-8e32-ac7ef73ba33c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:07:26.713871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'suspend' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state suspended Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: {"suspend": {}} Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'suspend' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state suspended Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'suspend': {}}} Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 40, in _suspend Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.suspend(context, server) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 259, in inner Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state suspended. Cannot suspend while the instance is in this state. Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 44, in _suspend Oct 15 21:07:26.715714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'suspend' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state suspended Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'suspend' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state suspended Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'suspend' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state suspended {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-188d6d52-7adf-4ea6-adb6-662036440664 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 148 microversion: 2.1 time: 0.065194 Oct 15 21:07:26.718964 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2802/3916] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:26 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 148 bytes in 66 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:07:26.726709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:26.727317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:26.733443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.734162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.735184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.735966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.736680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.737251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.818172 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6075ea9-34c3-4e1c-abf8-60570bdfa0fe tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 202 len: 0 microversion: 2.1 time: 0.094926 Oct 15 21:07:26.818667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2803/3917] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:26 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 0 bytes in 96 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:07:26.823803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:26.824381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:26.831505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.831754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.832187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:26.832885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:26.833486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:26.833781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.027406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.037199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.038883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.040153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-beaae756-a182-4797-b922-656bcea075e7 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.220588 Oct 15 21:07:27.040888 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2804/3918] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:26 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:27.045647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:27.046255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:27.052857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.053371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.053951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.054865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.055389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.055814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.271282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.281067 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-730f2c73-eed9-4c50-ba3a-f5e31e677cdc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1502 microversion: 2.1 time: 0.239093 Oct 15 21:07:27.282787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2805/3919] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:27 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1502 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:27.288395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:27.289286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:27.302173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.302442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.303213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.304004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.304749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.305384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.406721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.426361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.427995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.429588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f63ebb9e-34fa-42a9-9451-19aab0defa10 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1335 microversion: 2.47 time: 0.145405 Oct 15 21:07:27.430176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2806/3920] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:27 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1335 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:27.440906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:27.442078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:27.454281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.455137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.455744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.456529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.457004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.457659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.696199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:27.710533 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-594fcbb5-5c74-475c-ae07-fce494069c1b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.278966 Oct 15 21:07:27.713114 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2807/3921] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:27 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:27.721477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:27.722080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:27.738068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.738714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.739170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:27.740162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:27.740623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:27.741073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.035481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.043537 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-88d3a0f4-f393-494f-8760-88d7f9b200b7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1506 microversion: 2.1 time: 0.327471 Oct 15 21:07:28.047850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2808/3922] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:27 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1506 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:28.069107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:28.069842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:28.076429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.077007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.077610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.079339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.079906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.080413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.272060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.287714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.289801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.291452 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea1f7c42-02f6-4222-b3a9-2b41c3057383 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.226146 Oct 15 21:07:28.301165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2809/3923] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:28 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:07:28.306219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:28.306836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:28.314516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.315410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.315859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.316603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.317086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.317558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.476731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.486695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0fe0f76b-8383-4dc8-ad8d-ceb87f53254e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1506 microversion: 2.1 time: 0.193602 Oct 15 21:07:28.487397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2810/3924] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:28 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1506 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:28.505278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:28.506140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:28.516849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.517829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.518401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.519404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.520057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.520491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.676491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.686257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e0fe38d-2474-4aec-a5ab-a9b89f80f1c3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1502 microversion: 2.1 time: 0.196901 Oct 15 21:07:28.691422 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2811/3925] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:28 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1502 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:28.696504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:28.697231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:28.704417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.704909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.705675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.706130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.706583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.707327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.715955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:28.784819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.801286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.803017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:28.804342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4434218-ad64-4c61-a222-0c4f903068bf tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1335 microversion: 2.47 time: 0.111880 Oct 15 21:07:28.805158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2812/3926] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:28 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1335 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:28.810828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "OS-DCF:diskConfig": "AUTO"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:28.816689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:28.826676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.827174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.827615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.828845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:28.829756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:28.831609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:28.968534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:28.993562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:28.993958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:28.997567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:28.998068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:28.998503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:29.160649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:29.169829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:29.170569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:29.171224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:29.172838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:29.174274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:29.174931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:29.759623 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fca7e862-7e40-4bd9-b494-80f259cf66a3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "POST /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a/action" status: 202 len: 1108 microversion: 2.1 time: 0.953088 Oct 15 21:07:29.765913 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2813/3927] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:28 2022] POST /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a/action => generated 1108 bytes in 960 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Oct 15 21:07:29.771116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:29.771651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:29.781428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:29.781960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:29.782410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:29.783370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:29.783776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:29.784205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:30.043510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:30.051923 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d462d844-3fcd-4dc9-a48c-bff681c6d8a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.285041 Oct 15 21:07:30.052537 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2814/3928] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:29 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:30.070597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:30.071076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:30.078488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:30.079405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:30.079856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:30.080624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:30.081122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:30.081558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:30.359423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:30.369289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:30.373817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:30.377345 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fefeb8cc-4271-4840-82a6-c46910fde7bf tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.311307 Oct 15 21:07:30.378959 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2815/3929] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:30 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:30.389569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:30.390583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:30.398143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:30.398881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:30.399486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:30.400376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:30.401006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:30.401985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:30.719143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:30.739198 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16e41657-c06f-415a-856d-c901356f36e1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.357697 Oct 15 21:07:30.740203 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2816/3930] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:30 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:30.755975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:30.761597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:30.779256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:30.782555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:30.784531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:30.785983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:30.786624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:30.787091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.029948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:31.040355 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d16f9bf8-14fa-428d-9d47-a2fbe4a7532f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.297775 Oct 15 21:07:31.062916 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2817/3931] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:30 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:31.081248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:31.081886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:31.108590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.109837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.117700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.119411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.120418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.121482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.221265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:31.239474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:31.241579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:31.243710 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6abc82e5-e0fc-4aa0-9457-38abe7f71aca tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1356 microversion: 2.47 time: 0.179060 Oct 15 21:07:31.244857 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2818/3932] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:31 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1356 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:31.250960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-00c1d687-22fa-491f-a32d-6bd1ccf50aaa req-f4197387-5ce6-4bb8-98c2-e793ef2d13b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "name": "network-vif-plugged", "status": "completed", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:31.267134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-00c1d687-22fa-491f-a32d-6bd1ccf50aaa req-f4197387-5ce6-4bb8-98c2-e793ef2d13b1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.268785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-00c1d687-22fa-491f-a32d-6bd1ccf50aaa req-f4197387-5ce6-4bb8-98c2-e793ef2d13b1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.269505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-00c1d687-22fa-491f-a32d-6bd1ccf50aaa req-f4197387-5ce6-4bb8-98c2-e793ef2d13b1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.293669 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-00c1d687-22fa-491f-a32d-6bd1ccf50aaa req-f4197387-5ce6-4bb8-98c2-e793ef2d13b1 service nova] Creating event network-vif-plugged:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:07:31.301421 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-00c1d687-22fa-491f-a32d-6bd1ccf50aaa req-f4197387-5ce6-4bb8-98c2-e793ef2d13b1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054810 Oct 15 21:07:31.302696 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2819/3933] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:31.308150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-da121599-9311-47d7-bfb6-b0ff18c214bc req-2d081053-a953-49f1-b5fc-5fdc875effeb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a621f033-1287-458c-aa82-a651da088114", "tag": "abf8f7e6-9012-4b3c-b0df-1c78ab33ae1b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:31.320888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-da121599-9311-47d7-bfb6-b0ff18c214bc req-2d081053-a953-49f1-b5fc-5fdc875effeb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.322224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-da121599-9311-47d7-bfb6-b0ff18c214bc req-2d081053-a953-49f1-b5fc-5fdc875effeb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.322873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-da121599-9311-47d7-bfb6-b0ff18c214bc req-2d081053-a953-49f1-b5fc-5fdc875effeb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.339641 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-da121599-9311-47d7-bfb6-b0ff18c214bc req-2d081053-a953-49f1-b5fc-5fdc875effeb service nova] Creating event network-changed:abf8f7e6-9012-4b3c-b0df-1c78ab33ae1b for instance a621f033-1287-458c-aa82-a651da088114 on nch2-master-1 Oct 15 21:07:31.355496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-da121599-9311-47d7-bfb6-b0ff18c214bc req-2d081053-a953-49f1-b5fc-5fdc875effeb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051215 Oct 15 21:07:31.361250 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2820/3934] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:31.367639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:31.369344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:31.377295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.378362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.379124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.380082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.381326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.382018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.558797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:31.573790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25adb012-5379-4adb-a0a4-2f4a0603209b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.210526 Oct 15 21:07:31.574493 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2821/3935] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:31 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:31.580426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:31.581158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:31.587500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.588116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.588630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.589627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.590107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.590546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.724584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:31.732661 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a291e401-f5bb-401c-b474-548eaa4253c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.156907 Oct 15 21:07:31.733271 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2822/3936] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:31 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:31.739055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:31.742510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:31.748402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.749291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.749401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.750116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.751227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.751664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.833355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:31.833789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:31.834273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:31.834693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:31.835117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:31.897917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:31.903936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.904492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.904937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.905972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.906165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.906668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.942856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae3344f1-fe33-4efd-8a60-493237304c07 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "POST /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969/action" status: 202 len: 1078 microversion: 2.47 time: 0.207289 Oct 15 21:07:31.942856 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2823/3937] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Oct 15 21:07:31 2022] POST /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969/action => generated 1078 bytes in 209 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Oct 15 21:07:31.961072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:31.961839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:31.968192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.968845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.969545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:31.970378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:31.970852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:31.971295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.116818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.134314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.134314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.135383 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4f14f26a-671f-40cd-8cf3-3177348e29aa tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.186476 Oct 15 21:07:32.145730 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2824/3938] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:31 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:32.287776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22696a15-7b8c-4076-a3cf-92c7b4f8b283 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Action: 'create', calling method: .version_select at 0x7fe74e2518b0>, body: {"server_group": {"name": "tempest-ServerGroupTestJSON-Server-Group-1308050460", "policies": ["affinity"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:32.306359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-22696a15-7b8c-4076-a3cf-92c7b4f8b283 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:32.311773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-22696a15-7b8c-4076-a3cf-92c7b4f8b283 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:32.354363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-22696a15-7b8c-4076-a3cf-92c7b4f8b283 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:32.360066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-22696a15-7b8c-4076-a3cf-92c7b4f8b283 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:32.373517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22696a15-7b8c-4076-a3cf-92c7b4f8b283 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 184 microversion: 2.1 time: 0.224133 Oct 15 21:07:32.380949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2825/3939] 10.222.0.16 () {60 vars in 1174 bytes} [Sat Oct 15 21:07:32 2022] POST /compute/v2.1/os-server-groups => generated 184 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:32.380949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5b80bd6-fbf4-49fe-aef0-d9986dc11adc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:32.380949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5b80bd6-fbf4-49fe-aef0-d9986dc11adc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 027e8ae6-2c14-4421-b3f9-b29c09fe9f51] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:32.386071 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f5b80bd6-fbf4-49fe-aef0-d9986dc11adc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 027e8ae6-2c14-4421-b3f9-b29c09fe9f51 could not be found. Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5b80bd6-fbf4-49fe-aef0-d9986dc11adc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/027e8ae6-2c14-4421-b3f9-b29c09fe9f51/action HTTP/1.1 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: {"unpause": {}} Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 027e8ae6-2c14-4421-b3f9-b29c09fe9f51 could not be found. Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '027e8ae6-2c14-4421-b3f9-b29c09fe9f51', 'body': {'unpause': {}}} Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 027e8ae6-2c14-4421-b3f9-b29c09fe9f51 could not be found. Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 58, in _unpause Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, ctxt, id) Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 027e8ae6-2c14-4421-b3f9-b29c09fe9f51 could not be found. Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:32.387644 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 027e8ae6-2c14-4421-b3f9-b29c09fe9f51 could not be found. Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5b80bd6-fbf4-49fe-aef0-d9986dc11adc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 027e8ae6-2c14-4421-b3f9-b29c09fe9f51 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5b80bd6-fbf4-49fe-aef0-d9986dc11adc tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/027e8ae6-2c14-4421-b3f9-b29c09fe9f51/action" status: 404 len: 111 microversion: 2.1 time: 0.013996 Oct 15 21:07:32.392458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2826/3940] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:32 2022] POST /compute/v2.1/servers/027e8ae6-2c14-4421-b3f9-b29c09fe9f51/action => generated 111 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:32.396070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:32.396671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:32.402829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.403315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.403753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.404467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.404926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.405385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.471562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.480175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.481677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.482872 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-84125848-ec64-4fab-a7c6-7b46f346f8a5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.092061 Oct 15 21:07:32.483688 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2827/3941] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:32 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:32.489427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:32.490240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:32.496452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.496951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.497417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.498126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.498588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.499022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.504758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:32.660291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.668555 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d62aaf7-0cec-49c3-ae1f-d140924ecc0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.184073 Oct 15 21:07:32.669357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2828/3942] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:32 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:32.675916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de45fa8a-26db-416d-84b7-128a9cbe32f7 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Action: 'create', calling method: .version_select at 0x7fe7584a9820>, body: {"server_group": {"name": "tempest-server-group-926073483", "policies": ["affinity"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:32.704577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-de45fa8a-26db-416d-84b7-128a9cbe32f7 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:32.712824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-de45fa8a-26db-416d-84b7-128a9cbe32f7 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:32.767474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-de45fa8a-26db-416d-84b7-128a9cbe32f7 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:32.773992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-de45fa8a-26db-416d-84b7-128a9cbe32f7 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:32.795446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de45fa8a-26db-416d-84b7-128a9cbe32f7 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 163 microversion: 2.1 time: 0.123681 Oct 15 21:07:32.796056 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2829/3943] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Oct 15 21:07:32 2022] POST /compute/v2.1/os-server-groups => generated 163 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:32.805934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:32.806793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:32.814230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.815201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.815843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.816775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.817280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.817749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.975402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:32.984795 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4aa8da41-fd8a-4f45-89cb-dc99d0299b22 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.187566 Oct 15 21:07:32.985459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2830/3944] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:32 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:32.990751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:32.991375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:32.998204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:32.998597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:32.999074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:32.999814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.000950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.001517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.160784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:33.172967 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ec00e0a-5e8a-4eb3-8f1b-ce9b6b8bd161 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.186242 Oct 15 21:07:33.173765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2831/3945] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:32 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:33.195012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4809bf60-ca72-4118-a228-b51c2b76a7c0 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Action: 'create', calling method: .version_select at 0x7fe74e309160>, body: {"server_group": {"name": "tempest-server-group-926073483", "policies": ["affinity"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:33.223796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4809bf60-ca72-4118-a228-b51c2b76a7c0 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:33.226318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:33.230770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4809bf60-ca72-4118-a228-b51c2b76a7c0 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:33.284134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4809bf60-ca72-4118-a228-b51c2b76a7c0 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:33.290313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4809bf60-ca72-4118-a228-b51c2b76a7c0 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:33.305652 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4809bf60-ca72-4118-a228-b51c2b76a7c0 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 163 microversion: 2.1 time: 0.127478 Oct 15 21:07:33.311842 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2832/3946] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Oct 15 21:07:33 2022] POST /compute/v2.1/os-server-groups => generated 163 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:33.311842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:33.312121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:33.319638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.319982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.320589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.321321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.321780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.322205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.450803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:33.611897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:33.623021 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-db6fc6be-33e7-4c29-a689-1409a689fed1 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.314779 Oct 15 21:07:33.639175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2833/3947] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:33 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:33.645053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:33.645965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:33.652341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.652991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.653538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.654400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.654948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.655453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.829738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:33.838008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:33.839466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:33.840643 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45f7726b-a289-4813-adce-2d65a0ca587c tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.200249 Oct 15 21:07:33.841228 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2834/3948] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:33 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:33.846203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ceb97a86-f902-4dc3-8053-7666ab2b8762 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:33.871405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ceb97a86-f902-4dc3-8053-7666ab2b8762 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/54f5373f-b98b-4b64-b863-848ac26febb4" status: 204 len: 0 microversion: 2.1 time: 0.029090 Oct 15 21:07:33.872157 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2835/3949] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:07:33 2022] DELETE /compute/v2.1/os-server-groups/54f5373f-b98b-4b64-b863-848ac26febb4 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:33.877174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-630ac8e4-dc03-47b3-9758-07c02c5e6301 req-e597bd40-8f3e-4a4e-b5dd-a1bc6c6feceb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-plugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:33.887860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-630ac8e4-dc03-47b3-9758-07c02c5e6301 req-e597bd40-8f3e-4a4e-b5dd-a1bc6c6feceb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.888352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-630ac8e4-dc03-47b3-9758-07c02c5e6301 req-e597bd40-8f3e-4a4e-b5dd-a1bc6c6feceb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.888790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-630ac8e4-dc03-47b3-9758-07c02c5e6301 req-e597bd40-8f3e-4a4e-b5dd-a1bc6c6feceb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.900706 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-630ac8e4-dc03-47b3-9758-07c02c5e6301 req-e597bd40-8f3e-4a4e-b5dd-a1bc6c6feceb service nova] Creating event network-vif-plugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:07:33.905932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-630ac8e4-dc03-47b3-9758-07c02c5e6301 req-e597bd40-8f3e-4a4e-b5dd-a1bc6c6feceb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032652 Oct 15 21:07:33.907037 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2836/3950] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:33.912147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:33.912849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:33.919272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.919768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.920222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.920926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:33.921426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:33.922565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:33.994410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.004311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.005871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.007051 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4c0863f-18d9-4a8f-a146-42a9e18663a6 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.098937 Oct 15 21:07:34.007609 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2837/3951] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:33 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:34.012219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:34.012894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:34.022845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.023327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.023761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.024462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.024917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.025380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.049446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'unpause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: {"unpause": {}} Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'unpause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'unpause': {}}} Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 62, in _unpause Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.unpause(ctxt, server) Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state active. Cannot unpause while the instance is in this state. Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 66, in _unpause Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'unpause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:34.051077 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'unpause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'unpause' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c09d52c0-b7f2-4d01-a4ed-78f850f5f106 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 145 microversion: 2.1 time: 0.043950 Oct 15 21:07:34.055216 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2838/3952] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:34 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 145 bytes in 45 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:07:34.057346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.057954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:34.064051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.064541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.064983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.065708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.066165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.066591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.285305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.295492 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de79d614-70b6-4a66-957f-1c727de4264e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.241461 Oct 15 21:07:34.296407 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2839/3953] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:34.313401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a50c3774-5d2f-4d47-94c1-dec5ecc4ee5f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.329995 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a50c3774-5d2f-4d47-94c1-dec5ecc4ee5f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 334 microversion: 2.1 time: 0.026300 Oct 15 21:07:34.330964 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2840/3954] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/os-server-groups => generated 334 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:34.335376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.336082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:34.342124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.343592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.343592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.344604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.345487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.345708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.481113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.489236 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea5952e6-dba5-4257-bb50-ed1cc63f6176 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.157331 Oct 15 21:07:34.490065 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2841/3955] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:34.494555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.495197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:34.501845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.502347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.502795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.503542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.504012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.504448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.636775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.644494 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9833cbd-2bc9-477b-959b-5599e6cb48c0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.153445 Oct 15 21:07:34.645081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2842/3956] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:34.653158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-29affeb1-1773-4688-bd1c-528b04738a22 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.676374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-29affeb1-1773-4688-bd1c-528b04738a22 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/79596233-46e5-4faf-b348-d70898f9386f" status: 204 len: 0 microversion: 2.1 time: 0.030245 Oct 15 21:07:34.680317 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2843/3957] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:07:34 2022] DELETE /compute/v2.1/os-server-groups/79596233-46e5-4faf-b348-d70898f9386f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:34.682267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.682894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:34.690569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.691072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.691650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.693606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.693606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.693606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.847083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:34.856354 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9e637d4-118b-43e1-a1b6-0a2e83f75230 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.178137 Oct 15 21:07:34.857576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2844/3958] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:34.862847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-475f4970-ec00-41fc-aaf7-2a90567606a8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.873758 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-475f4970-ec00-41fc-aaf7-2a90567606a8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.014860 Oct 15 21:07:34.882958 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2845/3959] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:34.888651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:34.889813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:34.897147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.898299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.898961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:34.900035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:34.900716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:34.901388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:35.369475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.380036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.381959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.383359 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a164ba6-2b54-4d2a-a089-d42d128f0568 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1729 microversion: 2.20 time: 0.498922 Oct 15 21:07:35.384714 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2846/3960] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:34 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1729 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:35.394213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abe3ea82-9542-4577-9878-e07bd50f239b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:35.395596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abe3ea82-9542-4577-9878-e07bd50f239b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:35.403970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-abe3ea82-9542-4577-9878-e07bd50f239b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5 could not be found. Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abe3ea82-9542-4577-9878-e07bd50f239b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/282f350e-8a3e-4d97-ae78-3da5f0ab2bf5/action HTTP/1.1 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 18 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: {"unshelve": null} Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5 could not be found. Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '282f350e-8a3e-4d97-ae78-3da5f0ab2bf5', 'body': {'unshelve': None}} Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5 could not be found. Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 102, in _unshelve Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5 could not be found. Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:35.405812 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5 could not be found. Oct 15 21:07:35.409997 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:35.410499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abe3ea82-9542-4577-9878-e07bd50f239b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 282f350e-8a3e-4d97-ae78-3da5f0ab2bf5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:35.411793 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abe3ea82-9542-4577-9878-e07bd50f239b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/282f350e-8a3e-4d97-ae78-3da5f0ab2bf5/action" status: 404 len: 111 microversion: 2.1 time: 0.025826 Oct 15 21:07:35.412454 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2847/3961] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:35 2022] POST /compute/v2.1/servers/282f350e-8a3e-4d97-ae78-3da5f0ab2bf5/action => generated 111 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:35.417636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1dfd0699-9fba-4f4a-8521-5c306891fb6b tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Action: 'create', calling method: .version_select at 0x7fe74e3094c0>, body: {"server_group": {"name": "tempest-server-group-1201119832", "policies": ["affinity"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:35.439286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1dfd0699-9fba-4f4a-8521-5c306891fb6b tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:35.444624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1dfd0699-9fba-4f4a-8521-5c306891fb6b tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:35.495800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1dfd0699-9fba-4f4a-8521-5c306891fb6b tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:35.501702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-1dfd0699-9fba-4f4a-8521-5c306891fb6b tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:35.516886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1dfd0699-9fba-4f4a-8521-5c306891fb6b tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 164 microversion: 2.1 time: 0.103128 Oct 15 21:07:35.517829 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2848/3962] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Oct 15 21:07:35 2022] POST /compute/v2.1/os-server-groups => generated 164 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:35.522454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:35.523494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:35.529886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:35.530372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:35.530813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:35.532125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:35.532759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:35.533433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:35.605809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.616093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.617635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.618892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-94a20190-d2ba-40d4-9141-dfc9a65c5aa9 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.100038 Oct 15 21:07:35.619550 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2849/3963] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:35 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:35.624759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:35.626261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:35.642711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:35.643319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:35.643961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:35.644682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:35.645393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:35.645828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:35.809161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:35.825742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d415ca90-b1f9-477b-b6a9-4354ddb0b158 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.205240 Oct 15 21:07:35.835620 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2850/3964] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:35 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:35.845018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:35.845018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:35.850187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:35.851100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:35.851904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:35.853013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:35.853351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:35.853842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.025568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:36.035908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-132751d1-87bc-4135-be71-42a5c642a0f3 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.208101 Oct 15 21:07:36.037390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2851/3965] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:35 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:36.042007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da44626c-8b6d-4c4e-b161-583ddf8169e1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:36.058920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=4,nova_api:DELETE=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:36.081800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da44626c-8b6d-4c4e-b161-583ddf8169e1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/c424b219-6066-43ff-8a02-5aef1d628ad3" status: 204 len: 0 microversion: 2.1 time: 0.043308 Oct 15 21:07:36.082590 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2852/3966] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:07:36 2022] DELETE /compute/v2.1/os-server-groups/c424b219-6066-43ff-8a02-5aef1d628ad3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:36.091905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:36.093004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:36.102030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.102933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.103541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.104363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.104851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.105363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.323117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:36.333301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a12dc995-c7e7-4120-bcde-1c40bbfee9cc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.246476 Oct 15 21:07:36.334093 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2853/3967] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:36 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:36.338806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:36.339424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:36.355358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.355852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.356301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.357029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.357677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.358017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.537989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:36.541533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:36.548050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-472b7d33-4e40-4de4-a991-94282d60da8f tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.212468 Oct 15 21:07:36.548823 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2854/3968] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:36 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:36.565957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e24d9199-190a-42af-86d9-9e5105801cba tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:36.581728 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e24d9199-190a-42af-86d9-9e5105801cba tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.030379 Oct 15 21:07:36.582459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2855/3969] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:07:36 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:36.587496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:36.588381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:36.595996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.596802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.597516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.602952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.603648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.604259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.796448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:36.806997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:36.808420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:36.813361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4093112-082c-4ed4-9e57-cf9983ebb01e tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1716 microversion: 2.20 time: 0.229571 Oct 15 21:07:36.821653 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2856/3970] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:36 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1716 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:36.836975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:36.839959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:36.851287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.851928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.852383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.853138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:36.853607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:36.854039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:36.884941 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Cannot 'unshelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action HTTP/1.1 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 18 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: {"unshelve": null} Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'unshelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'unshelve': None}} Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 124, in _unshelve Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.unshelve( Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c in vm_state active. Cannot unshelve while the instance is in this state. Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 137, in _unshelve Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state( Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:07:36.886634 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'unshelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'unshelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 409 to user: Cannot 'unshelve' instance 5f318670-01e5-4565-8e32-ac7ef73ba33c while it is in vm_state active {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e252b13c-063e-40fe-8532-17eb9ab2565c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action" status: 409 len: 146 microversion: 2.1 time: 0.065062 Oct 15 21:07:36.890602 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2857/3971] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:36 2022] POST /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c/action => generated 146 bytes in 66 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:07:36.895277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-211ca92f-7a6b-4aaa-b7bf-3c954f2aa1cf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Action: 'create', calling method: .version_select at 0x7fe74ebe6d30>, body: {"server_group": {"name": "tempest-server-group-599006132", "policies": ["anti-affinity"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:36.919474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-211ca92f-7a6b-4aaa-b7bf-3c954f2aa1cf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:36.932098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-211ca92f-7a6b-4aaa-b7bf-3c954f2aa1cf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:36.983466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-211ca92f-7a6b-4aaa-b7bf-3c954f2aa1cf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:36.990133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-211ca92f-7a6b-4aaa-b7bf-3c954f2aa1cf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:37.004933 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-211ca92f-7a6b-4aaa-b7bf-3c954f2aa1cf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 168 microversion: 2.1 time: 0.115145 Oct 15 21:07:37.011185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2858/3972] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Oct 15 21:07:36 2022] POST /compute/v2.1/os-server-groups => generated 168 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:37.011185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:37.011750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:37.019168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.019788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.020321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.021204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.021810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.022655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.118332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.128699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.130813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.132033 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b48d72fd-25ff-42c6-9f0b-b05be4ee9ede tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.124831 Oct 15 21:07:37.132903 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2859/3973] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:37 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:37.148403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:37.149501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:37.156116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.157566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.158295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.159138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.159713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.160255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.332930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.344989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5acb65c1-1570-4ae0-b9a8-4c0c2a70bf30 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.209630 Oct 15 21:07:37.346708 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2860/3974] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:37 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:37.365997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:37.366619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:37.378380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.379437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.380595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.381841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.382528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.383059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.570397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.579907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c676ac5-3f9b-4c76-aea1-2931c25f3e65 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.231898 Oct 15 21:07:37.580911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2861/3975] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:37 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:37.585772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82585484-d45d-4aa1-8db6-df13eb948cf4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:37.629522 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82585484-d45d-4aa1-8db6-df13eb948cf4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/e50bbe48-eb3c-410a-bf9b-a324595a6739" status: 204 len: 0 microversion: 2.1 time: 0.047195 Oct 15 21:07:37.630852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2862/3976] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:07:37 2022] DELETE /compute/v2.1/os-server-groups/e50bbe48-eb3c-410a-bf9b-a324595a6739 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:37.637498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:37.638187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:37.645450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.646271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.646929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.647923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.648929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.649586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.816884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.824988 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f48d29c2-2972-4763-b6c3-b9616b7160a9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.192823 Oct 15 21:07:37.826320 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2863/3977] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:37 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:37.833146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:37.833146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:37.838870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.839474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.839994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.840779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:37.841361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:37.841866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:37.985988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:37.994783 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20fd0107-daa6-49d3-b1c4-b87818a8be2c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.167501 Oct 15 21:07:38.009064 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2864/3978] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:37 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:38.013665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22c83348-aa69-45cf-9488-ee54ba16e116 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:38.025403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22c83348-aa69-45cf-9488-ee54ba16e116 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.028353 Oct 15 21:07:38.026582 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2865/3979] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:07:37 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:38.031821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a9f878e-bdd4-4f49-bbc4-ea40381f50fa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1478736221_updated"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:38.034565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a9f878e-bdd4-4f49-bbc4-ea40381f50fa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 60963161-2f9a-4724-850d-5d3b2dca8e56] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:38.044237 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8a9f878e-bdd4-4f49-bbc4-ea40381f50fa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 60963161-2f9a-4724-850d-5d3b2dca8e56 could not be found. Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a9f878e-bdd4-4f49-bbc4-ea40381f50fa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/servers/60963161-2f9a-4724-850d-5d3b2dca8e56 HTTP/1.1 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 81 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1478736221_updated"}} Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 60963161-2f9a-4724-850d-5d3b2dca8e56 could not be found. Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '60963161-2f9a-4724-850d-5d3b2dca8e56', 'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1478736221_updated'}}} Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 60963161-2f9a-4724-850d-5d3b2dca8e56 could not be found. Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 915, in update Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(ctxt, req, id, is_detail=True) Oct 15 21:07:38.045903 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 60963161-2f9a-4724-850d-5d3b2dca8e56 could not be found. Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 60963161-2f9a-4724-850d-5d3b2dca8e56 could not be found. Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a9f878e-bdd4-4f49-bbc4-ea40381f50fa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 60963161-2f9a-4724-850d-5d3b2dca8e56 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:38.050750 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a9f878e-bdd4-4f49-bbc4-ea40381f50fa tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "PUT /compute/v2.1/servers/60963161-2f9a-4724-850d-5d3b2dca8e56" status: 404 len: 111 microversion: 2.1 time: 0.020247 Oct 15 21:07:38.052119 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2866/3980] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:07:38 2022] PUT /compute/v2.1/servers/60963161-2f9a-4724-850d-5d3b2dca8e56 => generated 111 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:38.058100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:38.058650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:38.065620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:38.066116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:38.066615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:38.067600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:38.068761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:38.069748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:38.225002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:38.234411 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6f00e8e8-cc51-43d3-a41c-2c69474c267e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.181154 Oct 15 21:07:38.235021 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2867/3981] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:38 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:38.240343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerGroupTestJSON-server-1027470325", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "f0b246e9-a714-4157-b9fe-76eaff6c282c"}]}, "os:scheduler_hints": {"group": "77476bf5-16bd-4c39-991d-fdae00279b28"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:38.301864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] validate_networks() for [('f0b246e9-a714-4157-b9fe-76eaff6c282c', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:07:38.752601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:07:38.753733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:07:38.754374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:07:38.754947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:07:38.755549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:07:38.775857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:38.775857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:38.775857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:38.776177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:38.776698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:38.776963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:38.797082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:38.802676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:38.822298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:07:38.824730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:07:38.901305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:38.913285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:38.950793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:DELETE=6,nova_api:SELECT=88,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:38.959947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:07:38.969208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Getting quotas for user 1d78f5fa0ab549cfba0a91e02d3379b4 and project 6edf9c4fb3cc4276999855482ca29697. Resources: {'server_group_members'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:07:39.002151 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e62534f-7527-4406-ad0e-0676f663a5bf tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.765908 Oct 15 21:07:39.003013 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2868/3982] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:07:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 767 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:07:39.014409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:39.014866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:39.021657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.022162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.022615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.023357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.023830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.024272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.126569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.138736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.140302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.142420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d5ae3d72-f789-4c51-aa93-98645eb3fabb tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.132437 Oct 15 21:07:39.143468 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2869/3983] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:39 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:39.149193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:39.149939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:39.159008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.159560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.160099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.161969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.162826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.163360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.368245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.378623 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f2b1517-feb1-4ecd-b7f9-225bac567b5c tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.233099 Oct 15 21:07:39.379168 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2870/3984] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:39 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:39.383874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:39.384509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:39.391375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.391879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.392326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.393074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.393545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.393983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.554166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.562506 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-90c1f51e-51ee-47f9-8572-b366ca56b80f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.182198 Oct 15 21:07:39.563457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2871/3985] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:39 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:39.568400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:39.569030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:39.575577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.576078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.576596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.577349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.578569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.579691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.770844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.788529 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf13baeb-d93b-4791-a592-c5629e836569 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.223810 Oct 15 21:07:39.802252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2872/3986] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:39 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:39.812095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:39.813090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:39.821191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.821284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.821773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.822525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.822991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.823448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.897223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:39.905721 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-305fe3a1-e6bc-43d3-bb45-99c9f3b88391 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1217 microversion: 2.1 time: 0.102386 Oct 15 21:07:39.917189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2873/3987] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:39 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1217 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:39.917189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-14992342-8250-4664-b34c-47df45ca39fa req-6c33491d-411a-4523-a01a-9d13744e0b4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a621f033-1287-458c-aa82-a651da088114", "name": "network-vif-plugged", "status": "completed", "tag": "abf8f7e6-9012-4b3c-b0df-1c78ab33ae1b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:39.924484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-14992342-8250-4664-b34c-47df45ca39fa req-6c33491d-411a-4523-a01a-9d13744e0b4e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.924980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-14992342-8250-4664-b34c-47df45ca39fa req-6c33491d-411a-4523-a01a-9d13744e0b4e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.925448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-14992342-8250-4664-b34c-47df45ca39fa req-6c33491d-411a-4523-a01a-9d13744e0b4e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.937920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-14992342-8250-4664-b34c-47df45ca39fa req-6c33491d-411a-4523-a01a-9d13744e0b4e service nova] Creating event network-vif-plugged:abf8f7e6-9012-4b3c-b0df-1c78ab33ae1b for instance a621f033-1287-458c-aa82-a651da088114 on nch2-master-1 Oct 15 21:07:39.943742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-14992342-8250-4664-b34c-47df45ca39fa req-6c33491d-411a-4523-a01a-9d13744e0b4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035012 Oct 15 21:07:39.944471 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2874/3988] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:39.949764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c56223f9-5689-45dc-87d8-75cf0366a459 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'update', calling method: >, body: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c56223f9-5689-45dc-87d8-75cf0366a459 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c HTTP/1.1 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 280 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'server': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['name']: Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.956092 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c56223f9-5689-45dc-87d8-75cf0366a459 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c56223f9-5689-45dc-87d8-75cf0366a459 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "PUT /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 400 len: 619 microversion: 2.1 time: 0.012062 Oct 15 21:07:39.958963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2875/3989] 10.222.0.16 () {60 vars in 1257 bytes} [Sat Oct 15 21:07:39 2022] PUT /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 619 bytes in 13 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:07:39.972979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:39.974046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:39.983402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.984919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.985571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:39.986464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:39.990183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:39.990802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.278394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:40.287626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7efbac07-2c7e-4e73-b2a9-10efe469745b tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.321905 Oct 15 21:07:40.315069 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2876/3990] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:39 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:40.321922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:40.324138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:40.332557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:40.336373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:40.336900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.338127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:40.338207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:40.338952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.419294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:40.430074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:40.432008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:40.433234 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-549fde4d-cef1-4aae-a5f7-477fbbc0ddde tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.115426 Oct 15 21:07:40.433795 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2877/3991] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:40 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:40.457238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:40.457866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:40.464066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:40.464734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:40.465341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.466219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:40.466842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:40.467273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.615939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:40.626626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ebb14f3-b266-465d-a4a1-2122fa550aa6 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.180290 Oct 15 21:07:40.627492 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2878/3992] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:40 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:40.633883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:40.634817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:40.642834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:40.643608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:40.644259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.645218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:40.645864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:40.647307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:40.663394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:40.957428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:40.968618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39dd360d-a8ba-4b25-b971-66fd29a3f7b1 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.339774 Oct 15 21:07:40.969584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2879/3993] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:40 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:40.974283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a424402f-2359-400c-acfa-e4f1356515c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Action: 'update', calling method: >, body: {"server": {"name": ""}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a424402f-2359-400c-acfa-e4f1356515c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: request: PUT /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c HTTP/1.1 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 24 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: {"server": {"name": ""}} Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute name. Value: . '' is too short Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c', 'body': {'server': {'name': ''}}} Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['server']['properties']['name']: Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['server']: Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '' Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:07:40.980135 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a424402f-2359-400c-acfa-e4f1356515c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:40.985588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a424402f-2359-400c-acfa-e4f1356515c7 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "PUT /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 400 len: 108 microversion: 2.1 time: 0.010936 Oct 15 21:07:40.986948 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2880/3994] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:07:40 2022] PUT /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 108 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:07:40.991868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:40.992504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:41.000657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.001256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.001719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.002447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.004790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.005282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.244730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:41.258971 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98ffc676-e58d-42e1-9e15-dde8e2235836 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1313 microversion: 2.1 time: 0.269279 Oct 15 21:07:41.259759 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2881/3995] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:40 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1313 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:41.284730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:41.285339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:41.295079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.295942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.297477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.298526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.302527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.304685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.390157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:41.398711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8587c803-b606-4fae-9530-853646803978 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1271 microversion: 2.1 time: 0.118466 Oct 15 21:07:41.410363 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2882/3996] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:41 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1271 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:41.422271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:41.423356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:41.435285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.436028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.436615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.437778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.438675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.439258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.686111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:41.697514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cbd43701-ca60-4f1b-983e-b072d18738a5 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1505 microversion: 2.1 time: 0.285382 Oct 15 21:07:41.698328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2883/3997] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:41 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1505 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:41.705901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:41.707086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:41.718848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.719586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.720392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.722008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:41.741634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:41.745146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:41.907982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:41.930354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:41.934178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:41.936272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61e5961d-cf46-4e06-a87a-271f91d3abfe tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.235417 Oct 15 21:07:41.937280 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2884/3998] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:41 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:41.997712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:41.998617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:42.015169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:42.016064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:42.016599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:42.017442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:42.018854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:42.019920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:42.335947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:42.348060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc80d6b6-004f-4116-a7ad-4dffeda31a47 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.356326 Oct 15 21:07:42.348842 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2885/3999] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:41 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:42.371594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:42.372194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:42.404103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:42.404854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:42.406924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:42.408077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:42.410190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:42.410859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:42.731766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:42.744438 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a6f9823-fe47-47bb-8fda-e474d100d5ed tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1499 microversion: 2.1 time: 0.377997 Oct 15 21:07:42.745175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2886/4000] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:42 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1499 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:42.754582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:42.755288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:42.763596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:42.764485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:42.765185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:42.766305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:42.766968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:42.767486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.052747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:43.061357 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6759c6b-941c-409c-8060-481595cd3f63 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.315019 Oct 15 21:07:43.078428 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2887/4001] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:42 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:43.085675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:43.088109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:43.105069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.105069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.105069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.105069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.105069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.105069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.131714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lazy-loading 'flavor' on Instance uuid a621f033-1287-458c-aa82-a651da088114 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:07:43.268447 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-973d37f8-04bb-4d77-a491-4524d652e83e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "POST /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114/action" status: 200 len: 32 microversion: 2.1 time: 0.188638 Oct 15 21:07:43.269527 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2888/4002] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:43 2022] POST /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114/action => generated 32 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:07:43.280635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:43.283128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:43.299375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.299934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.300426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.301190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.301668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.302172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.477602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:43.494624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:43.496782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:43.498350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe3515da-4cf4-42d5-a94f-c6c76811b9b4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.227425 Oct 15 21:07:43.500175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2889/4003] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:43 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:43.506400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:43.507540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:43.521903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.523085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.523670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.524662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.525363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.525996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.914034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:43.922798 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-74ce33d2-965c-4d6a-98e2-641a66c645c8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.420170 Oct 15 21:07:43.923387 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2890/4004] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:43 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:43.931559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-171d9c2e-8070-4bfd-9fa7-f03c33453191 req-a4b2bdf7-bccd-49a5-9b80-af264e7ce3e8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "feb638c6-e091-4e85-97fe-a8371d2afca5", "tag": "0e68c56a-3507-401b-8ef7-9cd8ce9b4ba9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:43.945419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-171d9c2e-8070-4bfd-9fa7-f03c33453191 req-a4b2bdf7-bccd-49a5-9b80-af264e7ce3e8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.945914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-171d9c2e-8070-4bfd-9fa7-f03c33453191 req-a4b2bdf7-bccd-49a5-9b80-af264e7ce3e8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.946374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-171d9c2e-8070-4bfd-9fa7-f03c33453191 req-a4b2bdf7-bccd-49a5-9b80-af264e7ce3e8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.959596 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-171d9c2e-8070-4bfd-9fa7-f03c33453191 req-a4b2bdf7-bccd-49a5-9b80-af264e7ce3e8 service nova] Creating event network-changed:0e68c56a-3507-401b-8ef7-9cd8ce9b4ba9 for instance feb638c6-e091-4e85-97fe-a8371d2afca5 on nch1-master-1 Oct 15 21:07:43.965327 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-171d9c2e-8070-4bfd-9fa7-f03c33453191 req-a4b2bdf7-bccd-49a5-9b80-af264e7ce3e8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040723 Oct 15 21:07:43.966599 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2891/4005] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:43.972813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:43.973658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:43.981371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.981882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.982230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:43.982928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:43.983393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:43.983821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:44.728102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:44.738067 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44eb2ed2-07c8-466b-923c-0e8c00e1eb23 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.770272 Oct 15 21:07:44.756269 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2892/4006] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:43 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:44.762062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:44.762900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:44.771747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:44.772475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:44.773259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:44.774583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:44.775214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:44.776220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:44.963516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:44.972513 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1662c36-7fa1-4c99-b3b7-1850b1014578 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.214809 Oct 15 21:07:44.973583 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2893/4007] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:44 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:44.978581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:44.979092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:44.985910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:44.986411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:44.986857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:44.988307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:44.988307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:44.988487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.005388 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 HTTP/1.1 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '14e27a99-2936-474b-85fd-21419ba8f525'} Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:45.007097 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be684f93-735a-4155-aee6-c02f49f01c23 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "DELETE /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 404 len: 111 microversion: 2.1 time: 0.033993 Oct 15 21:07:45.013030 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2894/4008] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:07:44 2022] DELETE /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:45.014512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:45.015166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:45.021538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.022035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.022481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.023196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.023662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.024102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.103039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:45.113690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:45.114620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:45.116093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0ae04993-97f2-416f-97db-847e8bbe1eee tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.105508 Oct 15 21:07:45.118383 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2895/4009] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:45 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:45.123238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:45.123845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:45.134780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.135283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.138090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.138827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.139296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.139732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.144725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:45.295583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:45.304639 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f65d442-87e3-4fe3-b653-5c408b586dc6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.185189 Oct 15 21:07:45.313151 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2896/4010] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:45 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:45.318305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:45.319533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 14e27a99-2936-474b-85fd-21419ba8f525] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:45.325909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.326570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.327177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.328565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.329084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.329680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.344271 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 HTTP/1.1 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '14e27a99-2936-474b-85fd-21419ba8f525'} Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:45.345879 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 14e27a99-2936-474b-85fd-21419ba8f525 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:45.350060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-484eb064-282f-4b97-a4b7-3272df9db7da tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525" status: 404 len: 111 microversion: 2.1 time: 0.032906 Oct 15 21:07:45.351122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2897/4011] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:45 2022] GET /compute/v2.1/servers/14e27a99-2936-474b-85fd-21419ba8f525 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:45.355666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:45.356138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:45.363264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.363765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.364205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.365202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.365663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.366086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.394158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:07:45.487525 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b73e61b7-2543-4eab-8c12-b6e3250bc09e tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 204 len: 0 microversion: 2.1 time: 0.135165 Oct 15 21:07:45.488302 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2898/4012] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:07:45 2022] DELETE /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:45.497196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:45.497901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:45.504799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.505354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.506078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.506797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.507263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.507703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.655068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:45.664715 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ee03c8c-e9ee-4e52-88c5-1f3a171b7651 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.171934 Oct 15 21:07:45.665336 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2899/4013] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:45 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:45.764419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:45.765125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:45.772082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.772771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.773416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.774273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:45.774974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:45.775994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:45.930553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:45.941374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-244e7fbc-8e37-46c0-bf93-d9cd35b8cabf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.181640 Oct 15 21:07:45.941962 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2900/4014] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:45 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:46.000540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:46.001818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:46.014779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.015454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.016013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.016764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.017326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.017831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.174785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:46.184157 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4d88199-57d8-4e25-a9ce-65e273db0c4f tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.188147 Oct 15 21:07:46.184758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2901/4015] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:45 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:46.194468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:46.194468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:46.200550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.201666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.202343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.203601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.204259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.204931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.287077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:46.296004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:46.297755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:46.299223 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd0930e0-95aa-4254-9e3c-d84482136d30 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.113480 Oct 15 21:07:46.300318 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2902/4016] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:46 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:46.330423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:46.331061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:46.337429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.337914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.338360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.339252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.339601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.340059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.489609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:46.498268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d52d6b31-6d91-4ae6-b5ac-cd0fba95432a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.172354 Oct 15 21:07:46.498845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2903/4017] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:46 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:46.690013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:46.692350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:46.697150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.697674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.698244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.698953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.699409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.699851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.854263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:46.863445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5f4d5f1-c46c-4818-97d5-ce2567bb4236 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.178180 Oct 15 21:07:46.864414 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2904/4018] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:46 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:46.966379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:46.966912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:46.973412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.974005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.974717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:46.975628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:46.976170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:46.976606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.128693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:47.136733 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e39ad323-6a4f-40b8-9bd4-52cea5c3f5d3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.174747 Oct 15 21:07:47.137734 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2905/4019] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:46 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:47.211497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:47.212131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:47.219413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.219804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.220258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.220995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.221492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.221932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.362906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:47.371862 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8347b2be-f2e0-40f8-89f4-4d66eeabecec tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.164368 Oct 15 21:07:47.372556 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2906/4020] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:47 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:47.377707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:47.378321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:47.384312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.384803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.385293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.385993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.386454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.386887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.455328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:47.463577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:47.465068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:47.466260 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-41628943-2faf-4542-9bc5-6fa1847afbce tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.092726 Oct 15 21:07:47.466832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2907/4021] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:47 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:47.521965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:47.522505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:47.528523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.529008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.529479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.530369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.531558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.532029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.706680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:47.714470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b736231-ed34-44f6-b1a2-c4a71a9f1001 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.196264 Oct 15 21:07:47.715036 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2908/4022] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:47 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:47.720076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:47.720682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:47.729167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.729665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.730112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.730823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:47.731284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:47.731718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:47.997298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.005500 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-954f2371-cbd6-4cbf-a613-8c4f65119403 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.289400 Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2909/4023] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:47 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.021079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.349899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90e8754a100240a498e422de07c76bbf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '172114b9a46e4df8ae8076ed582f0b5f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.357765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abe55f60-dab3-4002-940d-f0c73727e731 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 200 len: 1511 microversion: 2.1 time: 0.350119 Oct 15 21:07:48.358345 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2910/4024] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:48 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 1511 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:48.363661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:48.364287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:48.371709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.372204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.372696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.373499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.374194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.375455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.590038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.594451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:48.599810 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d8972609-f829-432e-8f51-866e7509f0f3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.239926 Oct 15 21:07:48.600394 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2911/4025] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:48 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:48.605524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:48.606139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:48.612172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.612659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.613133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.613847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.614314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.615007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.786711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:48.818602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.826470 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b045f0e-0c44-4e74-a10c-02818db09b9c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.225080 Oct 15 21:07:48.827080 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2912/4026] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:48 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:48.832497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:48.833711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:48.840445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.840878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.841386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.842301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.842791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.843391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.914683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.923923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.925576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:48.926755 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b60b7f8-5447-4339-8540-ec58384ff799 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.098015 Oct 15 21:07:48.927324 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2913/4027] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:48 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:48.931812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:48.932432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:48.937965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:48.941143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.941677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.942120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.942830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:48.943294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:48.943725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:48.979470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=72 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:49.086899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:49.095757 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e929213-de5d-4df4-9a9b-7619eea08cc9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.167295 Oct 15 21:07:49.096270 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2914/4028] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:48 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:49.100785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6c211114-8474-4f19-bab4-6c0459a6bb97 req-756d2bc2-ab17-447d-b56d-e5e0809337f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f318670-01e5-4565-8e32-ac7ef73ba33c", "name": "network-vif-deleted", "tag": "b779e022-9b5e-4fc4-8c92-ae2e834e6c81"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:49.110986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c211114-8474-4f19-bab4-6c0459a6bb97 req-756d2bc2-ab17-447d-b56d-e5e0809337f5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.111473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c211114-8474-4f19-bab4-6c0459a6bb97 req-756d2bc2-ab17-447d-b56d-e5e0809337f5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.111908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6c211114-8474-4f19-bab4-6c0459a6bb97 req-756d2bc2-ab17-447d-b56d-e5e0809337f5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.124641 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6c211114-8474-4f19-bab4-6c0459a6bb97 req-756d2bc2-ab17-447d-b56d-e5e0809337f5 service nova] Creating event network-vif-deleted:b779e022-9b5e-4fc4-8c92-ae2e834e6c81 for instance 5f318670-01e5-4565-8e32-ac7ef73ba33c on nch1-master-1 Oct 15 21:07:49.129871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6c211114-8474-4f19-bab4-6c0459a6bb97 req-756d2bc2-ab17-447d-b56d-e5e0809337f5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032348 Oct 15 21:07:49.130757 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2915/4029] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:49.135980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:49.136694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:49.143931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.144429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.144875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.145628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.146091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.146526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.285768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:49.294031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eac07b7d-d3da-49d6-8d9d-53bd8b810ffd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.162077 Oct 15 21:07:49.294604 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2916/4030] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:49 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:49.317236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-2-1428011824"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:49.319751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:49.325854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.326352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.326801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.328039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.328607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.329016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.520989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5926a0e4-e4ce-479d-859c-727c892482c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.207505 Oct 15 21:07:49.521716 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2917/4031] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:07:49 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 209 msecs (HTTP/1.1 202) 10 headers in 456 bytes (2 switches on core 0) Oct 15 21:07:49.527191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:49.527899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] [instance: 5f318670-01e5-4565-8e32-ac7ef73ba33c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:49.535135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.535335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.535751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.536464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.536956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.537580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.552772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] HTTP exception thrown: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c could not be found. Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Request method failure captured: Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c HTTP/1.1 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyA6qowpyk9suhRXjJEQCv9aUpH2WwkLcNC_oJauindHF0L6iG_d2iHGZYonU5lTGXtilp4pnx4XV-R_qUb10BI7T0WIupj8dZLCAzlPFMD0lqUE_n0fREQ3LMzi8VHVAYw3WXBBQOwYajS46zeivJS7OHlLCCjLYmu5ik88N0qkHy8 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/172114b9a46e4df8ae8076ed582f0b5f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 172114b9a46e4df8ae8076ed582f0b5f Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-469731530 Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 90e8754a100240a498e422de07c76bbf Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-469731530-project Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c could not be found. Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '5f318670-01e5-4565-8e32-ac7ef73ba33c'} Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c could not be found. Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c could not be found. Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:07:49.554476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c could not be found. Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] Returning 404 to user: Instance 5f318670-01e5-4565-8e32-ac7ef73ba33c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0cef106-6ef9-46e6-a9e0-bb70e90692c0 tempest-ServersNegativeTestJSON-469731530 tempest-ServersNegativeTestJSON-469731530-project] 10.222.0.16 "GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c" status: 404 len: 111 microversion: 2.1 time: 0.032927 Oct 15 21:07:49.558771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2918/4032] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:49 2022] GET /compute/v2.1/servers/5f318670-01e5-4565-8e32-ac7ef73ba33c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:07:49.623928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:49.625497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:49.637406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.638288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.638852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.639670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.640872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.642781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.809770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:49.818871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-109634d3-cbd8-4505-add7-01df0bf78975 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.198987 Oct 15 21:07:49.819454 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2919/4033] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:49 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:49.849997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:49.850616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:49.856725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.857330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.857865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.859412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:49.859902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:49.860334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:49.992650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:50.001295 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-077d5009-3a56-4848-a140-e708a0902412 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.155040 Oct 15 21:07:50.001875 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2920/4034] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:49 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:50.006465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:50.007077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:50.012888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:50.013412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:50.013858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:50.014559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:50.015372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:50.016080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:50.084323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:50.092637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:50.095067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:50.096296 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c0bb2af-c9fd-4d24-bc57-b2bb33160328 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.093466 Oct 15 21:07:50.096850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2921/4035] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:50 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:50.119911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:50.120643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:50.126995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:50.127495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:50.127946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:50.128657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:50.129152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:50.129591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:50.273586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:50.282565 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67ee411c-3b45-42c5-ad22-e2a9de98310c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.166420 Oct 15 21:07:50.295904 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2922/4036] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:50 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:50.842467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:50.844100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:50.850468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:50.851051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:50.851578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:50.852450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:50.853181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:50.853698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.011303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:51.021343 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-696be5c2-1f23-4027-be26-7e04141628b3 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.182708 Oct 15 21:07:51.025606 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2923/4037] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:50 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:51.047339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:51.049427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:51.056265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.056756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.057237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.058052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.058523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.058960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.236257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:51.246127 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-879caa2c-a84a-46f3-818c-fafdc418d508 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.217974 Oct 15 21:07:51.247426 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2924/4038] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:51 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:51.253723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:51.254683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:51.262305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.263106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.263915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.265412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.266494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.267158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.349016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:51.361258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:51.363260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:51.364321 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-234bf56f-a89d-407e-a84d-51f6de52f587 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.115478 Oct 15 21:07:51.365841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2925/4039] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:51 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:51.380448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:51.381375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:51.389551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.391373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.391884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.392787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.393361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.393783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.570213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:51.598507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b09f6ef4-c47b-45a7-bba7-30cca8acbb1c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.229087 Oct 15 21:07:51.602107 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2926/4040] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:51 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:51.725296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-19ac1c5b-cfd3-477d-ba8e-8b6805c8ad5b req-ea45eacf-473f-4d00-86d5-f6d84573ad48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "feb638c6-e091-4e85-97fe-a8371d2afca5", "name": "network-vif-plugged", "status": "completed", "tag": "0e68c56a-3507-401b-8ef7-9cd8ce9b4ba9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:51.736106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-19ac1c5b-cfd3-477d-ba8e-8b6805c8ad5b req-ea45eacf-473f-4d00-86d5-f6d84573ad48 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:51.736743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-19ac1c5b-cfd3-477d-ba8e-8b6805c8ad5b req-ea45eacf-473f-4d00-86d5-f6d84573ad48 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:51.737301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-19ac1c5b-cfd3-477d-ba8e-8b6805c8ad5b req-ea45eacf-473f-4d00-86d5-f6d84573ad48 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:51.751660 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-19ac1c5b-cfd3-477d-ba8e-8b6805c8ad5b req-ea45eacf-473f-4d00-86d5-f6d84573ad48 service nova] Creating event network-vif-plugged:0e68c56a-3507-401b-8ef7-9cd8ce9b4ba9 for instance feb638c6-e091-4e85-97fe-a8371d2afca5 on nch1-master-1 Oct 15 21:07:51.758427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-19ac1c5b-cfd3-477d-ba8e-8b6805c8ad5b req-ea45eacf-473f-4d00-86d5-f6d84573ad48 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036931 Oct 15 21:07:51.759611 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2927/4041] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:07:52.045430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:52.046575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:52.052886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.053433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.053919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.054632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.055097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.055538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.209918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:52.223621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8404b18f-40d8-4218-9558-d378641754f2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.183170 Oct 15 21:07:52.224977 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2928/4042] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:52 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:52.275464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:52.276149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:52.282461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.283162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.283762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.284630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.285116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.285544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.290045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:52.514820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:52.522733 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18bdc42d-1540-4fd3-a503-06bb8814fd49 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1313 microversion: 2.1 time: 0.251874 Oct 15 21:07:52.523315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2929/4043] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:52 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1313 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:52.527895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:52.528814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:52.535243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.535737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.536173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.536868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.537590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.538027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.615782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:52.631338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:52.633596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:52.635052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2fe7e3f3-c291-4eef-8e3b-9c076a5f8e96 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.110694 Oct 15 21:07:52.649126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2930/4044] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:52 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:52.658161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:52.659085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:52.665527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.666038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.666493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.667216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:52.667690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:52.668130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:52.926803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:52.939839 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e56ed54-139e-44f3-a5c0-567d2e795b5a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.289105 Oct 15 21:07:52.953189 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2931/4045] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:52 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:53.145112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:53.147765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:53.155180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.156169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.156620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.157388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.157859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.158291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.199739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd5567e4-adf7-46f6-a4e6-d4ec070a797d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053196 Oct 15 21:07:53.199739 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2932/4046] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:07:53 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 21:07:53.248087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:53.249733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:53.260971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.261721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.262162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.263203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.264306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.264776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.421412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:53.430746 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86166908-1d71-45c3-993a-08060294cf9f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.186734 Oct 15 21:07:53.431514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2933/4047] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:53 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:53.436191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:53.437297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:53.445901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.445901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.445901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.445901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.446173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.446591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.473795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:07:53.557609 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24c7248d-8a25-4135-8107-6b2476118995 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 204 len: 0 microversion: 2.1 time: 0.124749 Oct 15 21:07:53.558558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2934/4048] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:07:53 2022] DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:53.564748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:53.565521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:53.571735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.572233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.572715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.573498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.573931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.574364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.713723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:53.722557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-afe82a74-ca54-4c0e-8ad7-5fc410215dcd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1498 microversion: 2.1 time: 0.163026 Oct 15 21:07:53.723213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2935/4049] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:53 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1498 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:53.731480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:53.732190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:53.738719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.739214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.739661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.740735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.741304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.741742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.879749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:53.888092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba5e7ca3-5f12-4b67-8d21-cc880e11e814 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1673 microversion: 2.1 time: 0.163683 Oct 15 21:07:53.888669 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2936/4050] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:53 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1673 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:53.893786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:53.894399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:53.900123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.900608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.901071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.901764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:53.902222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:53.902651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:53.970885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:53.979348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:53.980900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:53.982037 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d2c8820-86fc-4166-a08f-2ab6a7bd3425 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.092215 Oct 15 21:07:53.982614 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2937/4051] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:53 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:53.988075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d224a938-7bd0-4826-8695-1bc495fd4c94 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:54.003117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_groups [None req-d224a938-7bd0-4826-8695-1bc495fd4c94 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Querying cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) for 1 instances {{(pid=138688) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Oct 15 21:07:54.003840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d224a938-7bd0-4826-8695-1bc495fd4c94 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.004328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d224a938-7bd0-4826-8695-1bc495fd4c94 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.004772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d224a938-7bd0-4826-8695-1bc495fd4c94 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.021922 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d224a938-7bd0-4826-8695-1bc495fd4c94 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/77476bf5-16bd-4c39-991d-fdae00279b28" status: 200 len: 222 microversion: 2.1 time: 0.038292 Oct 15 21:07:54.022534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2938/4052] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:07:53 2022] GET /compute/v2.1/os-server-groups/77476bf5-16bd-4c39-991d-fdae00279b28 => generated 222 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:54.026874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bf097543-8be3-4025-97fc-b4835abade65 req-3c74dade-42b4-45d3-9273-58228f8b683a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-unplugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:54.036814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bf097543-8be3-4025-97fc-b4835abade65 req-3c74dade-42b4-45d3-9273-58228f8b683a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.037354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bf097543-8be3-4025-97fc-b4835abade65 req-3c74dade-42b4-45d3-9273-58228f8b683a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.037777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bf097543-8be3-4025-97fc-b4835abade65 req-3c74dade-42b4-45d3-9273-58228f8b683a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.048866 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-bf097543-8be3-4025-97fc-b4835abade65 req-3c74dade-42b4-45d3-9273-58228f8b683a service nova] Creating event network-vif-unplugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:07:54.053983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bf097543-8be3-4025-97fc-b4835abade65 req-3c74dade-42b4-45d3-9273-58228f8b683a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030325 Oct 15 21:07:54.054581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2939/4053] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:54.059647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:54.060354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:54.067037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.067537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.068068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.068789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.069384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.069831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.208266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:54.216130 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c506260d-7224-4910-95d1-94fe85c4aa46 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.160364 Oct 15 21:07:54.216709 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2940/4054] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:54 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:54.221472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:54.221975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:54.227732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.228225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.228659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.229460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.229930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.230361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.259746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:07:54.342656 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98d1dfd2-e965-462b-b4b6-0e6c627d53f4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 204 len: 0 microversion: 2.1 time: 0.124394 Oct 15 21:07:54.343505 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2941/4055] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:07:54 2022] DELETE /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:07:54.352205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:54.352840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:54.359500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.359996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.360438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.361211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.361665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.362098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.502745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:54.510656 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35309c00-bd61-49e5-8796-264a2deea9c6 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1504 microversion: 2.1 time: 0.162766 Oct 15 21:07:54.511242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2942/4056] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:54 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1504 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:54.516977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:54.517653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:54.524123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.524618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.525087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.525800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.526260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.526690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.545266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:54.667570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:54.676407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d6ec49d-8475-4f06-8381-180e95600554 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.164041 Oct 15 21:07:54.677126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2943/4057] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:54 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:54.912742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:54.913398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:54.920787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.921617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.922282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:54.923140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:54.924335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:54.924875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.077067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.086493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-916ee59f-52e8-4f4d-b180-8664348324ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1673 microversion: 2.1 time: 0.177609 Oct 15 21:07:55.087274 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2944/4058] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:54 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1673 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:55.111279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:55.111908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:55.119402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.119604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.119916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.120632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.121122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.121621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.205503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.219097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.221073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.222629 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5666da40-2b47-4c59-8662-43f35fb20d86 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.118978 Oct 15 21:07:55.225086 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2945/4059] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:55 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:55.242978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:55.244028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:55.252339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.252847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.253373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.254255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.254751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.255191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.405606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.417395 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5efb0c22-90db-460b-ac6b-be114d13fd7e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.179740 Oct 15 21:07:55.418455 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2946/4060] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:55 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:55.535537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:55.536859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:55.551541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.553411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.554013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.555857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.556339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.556785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.728083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.738486 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f02d8d9b-90f7-49c5-8237-2aa013cbc83c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1504 microversion: 2.1 time: 0.208145 Oct 15 21:07:55.739081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2947/4061] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:55 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1504 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:55.749458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:55.749458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:55.760169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.760679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.761150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.761905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:55.762373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:55.762991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:55.913269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:55.922321 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31b65d19-12f8-45d7-a9d5-8e51b4eabf05 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.182182 Oct 15 21:07:55.922988 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2948/4062] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:55 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:55.971689 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:07:55.972206 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:07:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:07:56.116972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:56.117965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:56.147848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.148421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.150679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.150679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.155884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.156559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.323153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:56.331576 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24ee0a25-6325-4893-8861-0c3cbf3a3bbe tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1673 microversion: 2.1 time: 0.219558 Oct 15 21:07:56.332434 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2949/4063] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:56 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1673 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:07:56.335750 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:07:56.336132 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:07:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:07:56.345747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:56.346652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:56.352766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.353298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.353734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.354434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.354888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.355315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.423721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:56.431896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:56.433384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:56.434743 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-91f1c900-1b9a-4bdb-bb0c-2ebd2c0a4245 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.101001 Oct 15 21:07:56.435119 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2950/4064] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:56 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:56.440137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:56.440739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:56.446555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.447037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.447487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.448181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.448644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.449094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.592146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:56.601420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c32c2391-cb93-4464-9593-1c43227b5756 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.165122 Oct 15 21:07:56.602222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2951/4065] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:56 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:56.764097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:56.765606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:56.772811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.773406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.774047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.774784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.775262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.775850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.951127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:56.961963 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1903f890-9ff9-413c-9143-6ad845f141f3 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1504 microversion: 2.1 time: 0.201868 Oct 15 21:07:56.962503 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2952/4066] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:56 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1504 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:56.968488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:56.969534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:56.992915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.993657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.994479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:56.995718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:56.997604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:56.998075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:57.415731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:57.429364 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d633806a-0b26-42b2-bee7-3fe92f444eaf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.465679 Oct 15 21:07:57.430167 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2953/4067] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:56 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:57.450455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:57.451392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:57.459092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:57.460318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:57.460987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:57.461932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:57.462580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:57.463192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:58.076375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:58.091854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7dc481c5-342d-4d5e-9996-3f4fab399820 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1673 microversion: 2.1 time: 0.660317 Oct 15 21:07:58.096116 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2954/4068] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:57 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1673 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:58.105957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:58.112433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:58.124648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:58.125540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:58.126410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:58.127436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:58.128722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:58.130191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:58.598246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:58.608425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:58.610437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:58.612026 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63142270-9518-4c6a-ad03-a0213caaee89 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.513619 Oct 15 21:07:58.612510 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2955/4069] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:07:58 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:07:58.618501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d64a3930-6f72-435c-a48b-00b247490cd4 req-83b47695-22e9-4c96-a390-aca9cbd5059a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-plugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:07:58.636642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d64a3930-6f72-435c-a48b-00b247490cd4 req-83b47695-22e9-4c96-a390-aca9cbd5059a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:58.637848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d64a3930-6f72-435c-a48b-00b247490cd4 req-83b47695-22e9-4c96-a390-aca9cbd5059a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:58.638486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d64a3930-6f72-435c-a48b-00b247490cd4 req-83b47695-22e9-4c96-a390-aca9cbd5059a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:58.653627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d64a3930-6f72-435c-a48b-00b247490cd4 req-83b47695-22e9-4c96-a390-aca9cbd5059a service nova] Creating event network-vif-plugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:07:58.659733 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d64a3930-6f72-435c-a48b-00b247490cd4 req-83b47695-22e9-4c96-a390-aca9cbd5059a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045620 Oct 15 21:07:58.660945 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2956/4070] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:07:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:07:58.669628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:58.670588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:58.682625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:07:58.688027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:58.689015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:58.689659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:58.690614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:58.691234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:58.691820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:59.171140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:59.182583 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-721f7332-34c2-4287-96ca-5c821f0fae9c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.519096 Oct 15 21:07:59.193143 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2957/4071] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:58 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:59.202749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:59.202749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:59.211778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:59.212474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:59.216731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:59.218028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:59.218880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:59.219309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:59.421636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:59.431244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-180ec171-338b-4a7d-9ff2-38b1f950e914 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1462 microversion: 2.1 time: 0.237623 Oct 15 21:07:59.445062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2958/4072] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:59 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1462 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:59.445062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:59.445062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:59.447544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:59.454071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:59.454071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:59.454071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:59.454071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:59.454071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:59.795637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:07:59.818199 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60a0182d-c13a-4ef2-baf0-cfff0999eeef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.383951 Oct 15 21:07:59.818812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2959/4073] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:59 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:07:59.850627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:07:59.852630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:07:59.882900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:59.887968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:59.888744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:07:59.889851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:07:59.890963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:07:59.891835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.027071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:00.040761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:00.051471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c170a63-b034-42db-8477-f370bd30942d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1606 microversion: 2.1 time: 0.231560 Oct 15 21:08:00.052390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2960/4074] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:07:59 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1606 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:00.057131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-90013aed-f75b-4c1c-9126-9a37cd1c14d0 req-759046ca-f3e7-47a1-a415-f5ea464ea408 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "feb638c6-e091-4e85-97fe-a8371d2afca5", "name": "network-vif-deleted", "tag": "0e68c56a-3507-401b-8ef7-9cd8ce9b4ba9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:00.070282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-90013aed-f75b-4c1c-9126-9a37cd1c14d0 req-759046ca-f3e7-47a1-a415-f5ea464ea408 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.070781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-90013aed-f75b-4c1c-9126-9a37cd1c14d0 req-759046ca-f3e7-47a1-a415-f5ea464ea408 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.071226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-90013aed-f75b-4c1c-9126-9a37cd1c14d0 req-759046ca-f3e7-47a1-a415-f5ea464ea408 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.090764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-90013aed-f75b-4c1c-9126-9a37cd1c14d0 req-759046ca-f3e7-47a1-a415-f5ea464ea408 service nova] Creating event network-vif-deleted:0e68c56a-3507-401b-8ef7-9cd8ce9b4ba9 for instance feb638c6-e091-4e85-97fe-a8371d2afca5 on nch1-master-1 Oct 15 21:08:00.097336 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-90013aed-f75b-4c1c-9126-9a37cd1c14d0 req-759046ca-f3e7-47a1-a415-f5ea464ea408 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043675 Oct 15 21:08:00.099112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2961/4075] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:00.105492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:00.106199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:00.112554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.113225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.113666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.114567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.115288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.115797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.194435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:00.213162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:00.215181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:00.218820 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.118089 Oct 15 21:08:00.219907 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2962/4076] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:00 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:00.222197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:00.223671 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad858978-f6e4-44f2-ab68-11d3b5306bcd tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001773 Oct 15 21:08:00.224241 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2963/4077] 10.222.0.16 () {56 vars in 886 bytes} [Sat Oct 15 21:08:00 2022] GET /compute/ => generated 377 bytes in 4 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Oct 15 21:08:00.235169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:00.236217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:00.244429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.244938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.246173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.247026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.247493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.248227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.451370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:00.463633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f02a4dfa-151b-4470-8536-76d83cabbdc1 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.238114 Oct 15 21:08:00.464219 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2964/4078] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:00 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:00.469150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:00.469763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:00.476100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.476672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.477227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.478099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.478548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.478985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.513521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.514011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.514967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.515755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lazy-loading 'fault' on Instance uuid feb638c6-e091-4e85-97fe-a8371d2afca5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:08:00.598358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d78f5fa0ab549cfba0a91e02d3379b4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6edf9c4fb3cc4276999855482ca29697', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:00.606742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b71a023d-5a02-492a-aafc-9cae789a0597 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 200 len: 1303 microversion: 2.1 time: 0.141465 Oct 15 21:08:00.621730 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2965/4079] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:00 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 1303 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:00.845075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:00.845915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:00.860904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.862947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.863410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:00.864140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:00.864607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:00.869408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.216429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:01.245740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c9f5816-0351-4793-8ece-1c3e6d98c6f9 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1513 microversion: 2.1 time: 0.389955 Oct 15 21:08:01.245740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2966/4080] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:00 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1513 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:08:01.245740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:01.245740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:01.259293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.260845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.261411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.265107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.267567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.270206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.367453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.372475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.372911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.378100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lazy-loading 'fault' on Instance uuid d4ea7154-0623-44f0-8bad-076461f12dd5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:08:01.471377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:01.479633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-497b1bb2-ba93-40f3-83b3-f9578d4219d5 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 200 len: 1327 microversion: 2.1 time: 0.246912 Oct 15 21:08:01.480149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2967/4081] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:01 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 1327 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:01.507006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:01.509117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:01.533190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.534596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.535764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.538026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.539619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.540022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.737107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:01.754072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:01.760602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:01.764923 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f68e2fa3-8648-429f-9036-4cef3a21a37f tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.280559 Oct 15 21:08:01.767096 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2968/4082] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:01 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:01.779265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:01.781498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:01.800579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.801104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.803746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:01.804508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:01.804992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:01.810318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:02.244250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:02.253169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-252c8e8f-f78e-4b4f-90ac-8eab69407f03 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.483698 Oct 15 21:08:02.265504 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2969/4083] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:01 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:02.278189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:02.278754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:02.300241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:02.300735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:02.301209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:02.301924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:02.302382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:02.302809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:02.326758 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 HTTP/1.1 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'feb638c6-e091-4e85-97fe-a8371d2afca5'} Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:02.340693 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22367f7c-974a-465e-85c9-ecbf8d6fa9cd tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 404 len: 111 microversion: 2.1 time: 0.075815 Oct 15 21:08:02.345173 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2970/4084] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:02 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:02.354169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-79ee2b46-1fea-46eb-bc76-10d90b0a3054 req-eb19d19d-ff4a-4d0b-ab3b-33ba7bb41f88 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d4ea7154-0623-44f0-8bad-076461f12dd5", "name": "network-vif-deleted", "tag": "c039980d-116d-4619-8adf-362015a1de1c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:02.376478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-79ee2b46-1fea-46eb-bc76-10d90b0a3054 req-eb19d19d-ff4a-4d0b-ab3b-33ba7bb41f88 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:02.378833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-79ee2b46-1fea-46eb-bc76-10d90b0a3054 req-eb19d19d-ff4a-4d0b-ab3b-33ba7bb41f88 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:02.378833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-79ee2b46-1fea-46eb-bc76-10d90b0a3054 req-eb19d19d-ff4a-4d0b-ab3b-33ba7bb41f88 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:02.404005 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-79ee2b46-1fea-46eb-bc76-10d90b0a3054 req-eb19d19d-ff4a-4d0b-ab3b-33ba7bb41f88 service nova] Creating event network-vif-deleted:c039980d-116d-4619-8adf-362015a1de1c for instance d4ea7154-0623-44f0-8bad-076461f12dd5 on nch1-master-1 Oct 15 21:08:02.420704 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-79ee2b46-1fea-46eb-bc76-10d90b0a3054 req-eb19d19d-ff4a-4d0b-ab3b-33ba7bb41f88 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076309 Oct 15 21:08:02.421775 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2971/4085] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:02.437222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:02.437938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:02.446371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:02.446865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:02.447306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:02.448019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:02.448477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:02.448908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.133052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:03.161555 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6c49a3e-9186-4a88-bf8e-44d04f303b0c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1519 microversion: 2.1 time: 0.730747 Oct 15 21:08:03.164678 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2972/4086] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:02 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1519 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:03.174013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5cc9193a-df9a-420b-b58a-41b8221710d8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.226901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_groups [None req-5cc9193a-df9a-420b-b58a-41b8221710d8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Querying cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) for 1 instances {{(pid=138688) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Oct 15 21:08:03.229854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc9193a-df9a-420b-b58a-41b8221710d8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.230517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc9193a-df9a-420b-b58a-41b8221710d8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.230996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cc9193a-df9a-420b-b58a-41b8221710d8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.255532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5cc9193a-df9a-420b-b58a-41b8221710d8 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.086873 Oct 15 21:08:03.263624 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2973/4087] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:08:03 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:03.269985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.270640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:03.278405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.280228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.280842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.281644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.282230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.282704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.316404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] HTTP exception thrown: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Request method failure captured: Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 HTTP/1.1 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyB4RquiCnYmgp3soXHTxPzpZGgu0GC4FJjNmBkyJLbR7XR-EuLkOyVrYSEJIWELDgGGJ4liNiIb-CZIzbEllaNfkI-nG3QKBEFn61aO-uS1NViXHjKT6dW1_WcVD-lpOc_ivykZy27VcldLmm5gwwTaP1sA39wITsqmIqUEZPXWwfE Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2f4e4fc2f97f4c1a832c6348a97fd155 Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd4ea7154-0623-44f0-8bad-076461f12dd5'} Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:03.321339 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Returning 404 to user: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-28634593-edc5-45a3-a573-8356ead8c6b9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 404 len: 111 microversion: 2.1 time: 0.058175 Oct 15 21:08:03.325184 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2974/4088] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:03 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:03.340368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.340999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:03.363135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.363640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.364095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.370006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.375452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.377730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.549080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:03.558775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:03.560502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:03.562654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-156bae88-8ba2-4bde-8513-104edf517419 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.233552 Oct 15 21:08:03.569383 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2975/4089] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:03 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:03.581696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70db11a6-056b-4298-a6f6-1a058b8e048c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.617604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_groups [None req-70db11a6-056b-4298-a6f6-1a058b8e048c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Querying cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) for 1 instances {{(pid=138688) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Oct 15 21:08:03.618626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db11a6-056b-4298-a6f6-1a058b8e048c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.630717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db11a6-056b-4298-a6f6-1a058b8e048c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.631181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db11a6-056b-4298-a6f6-1a058b8e048c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.650592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70db11a6-056b-4298-a6f6-1a058b8e048c tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/77476bf5-16bd-4c39-991d-fdae00279b28" status: 200 len: 184 microversion: 2.1 time: 0.079755 Oct 15 21:08:03.651554 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2976/4090] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:08:03 2022] GET /compute/v2.1/os-server-groups/77476bf5-16bd-4c39-991d-fdae00279b28 => generated 184 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:03.659014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.659656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:03.672603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.673156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.673607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.674334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.674804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.675241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.875872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:03.886251 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f19bc51d-2e1d-4e48-af8c-5b6a6a27de61 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.233486 Oct 15 21:08:03.902318 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2977/4091] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:03 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:03.913018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfaea401-38a1-43f1-8c82-37d6ce157154 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '.version_select at 0x7fe74eba0c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.948852 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bfaea401-38a1-43f1-8c82-37d6ce157154 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-808352224" status: 202 len: 0 microversion: 2.1 time: 0.045251 Oct 15 21:08:03.949722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2978/4092] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 21:08:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-808352224 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:08:03.969777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:03.970329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:03.984317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.987601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.991452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:03.992319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:03.992812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:03.995328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.026920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 HTTP/1.1 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'feb638c6-e091-4e85-97fe-a8371d2afca5'} Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:04.028547 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.032063 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.032953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:04.033850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1e65f56-2cc9-42a3-be8d-809be49913e2 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 404 len: 111 microversion: 2.1 time: 0.082874 Oct 15 21:08:04.035442 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2979/4093] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:08:03 2022] DELETE /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 111 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:04.048501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.050909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] [instance: feb638c6-e091-4e85-97fe-a8371d2afca5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:04.070138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.070643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.071090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.071824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.072291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.072726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.104925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 HTTP/1.1 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'feb638c6-e091-4e85-97fe-a8371d2afca5'} Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.106579 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance feb638c6-e091-4e85-97fe-a8371d2afca5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:04.111330 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb79ccc0-4e7a-4232-8b9c-a1f1076ca6a4 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5" status: 404 len: 111 microversion: 2.1 time: 0.069504 Oct 15 21:08:04.115852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2980/4094] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:04 2022] GET /compute/v2.1/servers/feb638c6-e091-4e85-97fe-a8371d2afca5 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:04.129097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff8bf15c-547e-45e5-b574-7aaa4bfbcad1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.142179 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ff8bf15c-547e-45e5-b574-7aaa4bfbcad1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance group e50bbe48-eb3c-410a-bf9b-a324595a6739 could not be found. Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff8bf15c-547e-45e5-b574-7aaa4bfbcad1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-server-groups/e50bbe48-eb3c-410a-bf9b-a324595a6739 HTTP/1.1 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance group e50bbe48-eb3c-410a-bf9b-a324595a6739 could not be found. Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e50bbe48-eb3c-410a-bf9b-a324595a6739'} Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group e50bbe48-eb3c-410a-bf9b-a324595a6739 could not be found. Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group e50bbe48-eb3c-410a-bf9b-a324595a6739 could not be found. Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.154116 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.158968 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.158968 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.158968 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:04.158968 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:04.158968 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group e50bbe48-eb3c-410a-bf9b-a324595a6739 could not be found. Oct 15 21:08:04.158968 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.161252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff8bf15c-547e-45e5-b574-7aaa4bfbcad1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance group e50bbe48-eb3c-410a-bf9b-a324595a6739 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:04.162905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ff8bf15c-547e-45e5-b574-7aaa4bfbcad1 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/e50bbe48-eb3c-410a-bf9b-a324595a6739" status: 404 len: 117 microversion: 2.1 time: 0.046018 Oct 15 21:08:04.167701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2981/4095] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:04 2022] DELETE /compute/v2.1/os-server-groups/e50bbe48-eb3c-410a-bf9b-a324595a6739 => generated 117 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:04.173529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1dae5e3a-e5eb-445c-8c3d-2e66c5fb8e67 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.183673 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-1dae5e3a-e5eb-445c-8c3d-2e66c5fb8e67 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance group c424b219-6066-43ff-8a02-5aef1d628ad3 could not be found. Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1dae5e3a-e5eb-445c-8c3d-2e66c5fb8e67 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-server-groups/c424b219-6066-43ff-8a02-5aef1d628ad3 HTTP/1.1 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance group c424b219-6066-43ff-8a02-5aef1d628ad3 could not be found. Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c424b219-6066-43ff-8a02-5aef1d628ad3'} Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group c424b219-6066-43ff-8a02-5aef1d628ad3 could not be found. Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group c424b219-6066-43ff-8a02-5aef1d628ad3 could not be found. Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.187563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.191670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.191670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.191670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:04.191670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:04.191670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group c424b219-6066-43ff-8a02-5aef1d628ad3 could not be found. Oct 15 21:08:04.191670 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.193306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1dae5e3a-e5eb-445c-8c3d-2e66c5fb8e67 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance group c424b219-6066-43ff-8a02-5aef1d628ad3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:04.194351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1dae5e3a-e5eb-445c-8c3d-2e66c5fb8e67 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/c424b219-6066-43ff-8a02-5aef1d628ad3" status: 404 len: 117 microversion: 2.1 time: 0.025460 Oct 15 21:08:04.195057 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2982/4096] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:04 2022] DELETE /compute/v2.1/os-server-groups/c424b219-6066-43ff-8a02-5aef1d628ad3 => generated 117 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:04.206162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.207060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:04.221918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.222660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.223292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.224228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.224851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.225506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.529545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:04.548941 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f73138b5-b424-4626-b75e-e11a48705c6b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1519 microversion: 2.1 time: 0.351871 Oct 15 21:08:04.550363 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2983/4097] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:04 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1519 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:04.557986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad911402-735e-46a5-9aa3-ef8022230703 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.566012 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ad911402-735e-46a5-9aa3-ef8022230703 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance group 79596233-46e5-4faf-b348-d70898f9386f could not be found. Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad911402-735e-46a5-9aa3-ef8022230703 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-server-groups/79596233-46e5-4faf-b348-d70898f9386f HTTP/1.1 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance group 79596233-46e5-4faf-b348-d70898f9386f could not be found. Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '79596233-46e5-4faf-b348-d70898f9386f'} Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 79596233-46e5-4faf-b348-d70898f9386f could not be found. Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 79596233-46e5-4faf-b348-d70898f9386f could not be found. Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.567770 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.571309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.571309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.571309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:04.571309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:04.571309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 79596233-46e5-4faf-b348-d70898f9386f could not be found. Oct 15 21:08:04.571309 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.574339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad911402-735e-46a5-9aa3-ef8022230703 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance group 79596233-46e5-4faf-b348-d70898f9386f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:04.575403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad911402-735e-46a5-9aa3-ef8022230703 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/79596233-46e5-4faf-b348-d70898f9386f" status: 404 len: 117 microversion: 2.1 time: 0.022815 Oct 15 21:08:04.576432 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2984/4098] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:04 2022] DELETE /compute/v2.1/os-server-groups/79596233-46e5-4faf-b348-d70898f9386f => generated 117 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:04.591959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-728fc1ed-4f21-424e-9f2e-07e86c443025 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.601019 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-728fc1ed-4f21-424e-9f2e-07e86c443025 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] HTTP exception thrown: Instance group 54f5373f-b98b-4b64-b863-848ac26febb4 could not be found. Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-728fc1ed-4f21-424e-9f2e-07e86c443025 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Request method failure captured: Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-server-groups/54f5373f-b98b-4b64-b863-848ac26febb4 HTTP/1.1 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyET3mQDcA1Z5gb1rizmBSur8FctVENznpdqIVVbP9_TCJA63CoouxKNE-eB6b9brlhJ8rp9gYdgpQxl6gH7AM9l98b7bBlMFy0XpOLCInlPFGh6-bhnbfYskByu8vmano_O_y6OWvYE5ZWHCwupDCwxFo7p1QYSIofj7s2Lv5EvYZI Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6edf9c4fb3cc4276999855482ca29697"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6edf9c4fb3cc4276999855482ca29697 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroupTestJSON-1475526334 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d78f5fa0ab549cfba0a91e02d3379b4 Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroupTestJSON-1475526334-project Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance group 54f5373f-b98b-4b64-b863-848ac26febb4 could not be found. Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '54f5373f-b98b-4b64-b863-848ac26febb4'} Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 54f5373f-b98b-4b64-b863-848ac26febb4 could not be found. Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 54f5373f-b98b-4b64-b863-848ac26febb4 could not be found. Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.603103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:04.606923 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:04.606923 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:04.606923 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:04.606923 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:04.606923 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 54f5373f-b98b-4b64-b863-848ac26febb4 could not be found. Oct 15 21:08:04.606923 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:04.610381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-728fc1ed-4f21-424e-9f2e-07e86c443025 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Returning 404 to user: Instance group 54f5373f-b98b-4b64-b863-848ac26febb4 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:04.611633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-728fc1ed-4f21-424e-9f2e-07e86c443025 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/54f5373f-b98b-4b64-b863-848ac26febb4" status: 404 len: 117 microversion: 2.1 time: 0.028188 Oct 15 21:08:04.613003 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2985/4099] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:04 2022] DELETE /compute/v2.1/os-server-groups/54f5373f-b98b-4b64-b863-848ac26febb4 => generated 117 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:04.625890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.626720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:04.633618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.634320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.635071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.636029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.636635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.637263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.744686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:04.757620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:04.760129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:04.761542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4cfa9aed-457f-4f42-8a37-e8a63e6b4f18 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1365 microversion: 2.47 time: 0.147346 Oct 15 21:08:04.778328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2986/4100] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:04 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1365 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:04.786975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37287c6c-0c48-4340-aa44-72bc0db79e08 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.841148 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37287c6c-0c48-4340-aa44-72bc0db79e08 tempest-ServerGroupTestJSON-1475526334 tempest-ServerGroupTestJSON-1475526334-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/77476bf5-16bd-4c39-991d-fdae00279b28" status: 204 len: 0 microversion: 2.1 time: 0.061369 Oct 15 21:08:04.842439 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2987/4101] 10.222.0.16 () {58 vars in 1266 bytes} [Sat Oct 15 21:08:04 2022] DELETE /compute/v2.1/os-server-groups/77476bf5-16bd-4c39-991d-fdae00279b28 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:08:04.915101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:04.916062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:04.931123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.931926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.934180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:04.936771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:04.939464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:04.940471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:05.362095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:05.373406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6bcc92ac-80b2-462a-b1c4-8abaa7ea0926 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.461647 Oct 15 21:08:05.407863 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2988/4102] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:04 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:08:05.559008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-00c73ab2-8c07-40eb-90cc-4c6c9fef8d54 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: .version_select at 0x7fe74ff37e50>, body: {"keypair": {"name": "tempest-keypair-1950741633", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJwJ6lcTm+RRsYiGts30zS9Cee+EDZ9rlcKRnNVjV8C51YqseBWp3lSFdK2xuTZ68KvTV/q7yyzmiUDUogog43pFxf7rSkiVkMTGfKA/8LMJ8XW4Yw+2mHkvv88b50kjOQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:05.601723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-00c73ab2-8c07-40eb-90cc-4c6c9fef8d54 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:08:05.609639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-00c73ab2-8c07-40eb-90cc-4c6c9fef8d54 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for user 2f4e4fc2f97f4c1a832c6348a97fd155 and project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:08:05.650328 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-00c73ab2-8c07-40eb-90cc-4c6c9fef8d54 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.094737 Oct 15 21:08:05.660579 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2989/4103] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:08:05 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:08:05.677919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:05.678823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:05.691251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:05.698296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:05.699086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:05.700125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:05.701218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:05.701946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:05.741731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:06.190817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:06.199742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-85381356-67cd-4a1b-9a9f-fcf52f8a9dc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1519 microversion: 2.1 time: 0.537517 Oct 15 21:08:06.209458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2990/4104] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:05 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1519 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:06.223313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3b20c55e-c465-4965-a2b0-27939509ef53 req-7a6c1921-ea32-446e-b17d-d83cb4b02159 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "18be360f-e339-42ec-89ff-72acf428932a", "name": "network-vif-unplugged", "status": "completed", "tag": "83f1bf1e-ab89-427f-af3d-df7b7e6537b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:06.239961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3b20c55e-c465-4965-a2b0-27939509ef53 req-7a6c1921-ea32-446e-b17d-d83cb4b02159 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:06.241379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3b20c55e-c465-4965-a2b0-27939509ef53 req-7a6c1921-ea32-446e-b17d-d83cb4b02159 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:06.242013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3b20c55e-c465-4965-a2b0-27939509ef53 req-7a6c1921-ea32-446e-b17d-d83cb4b02159 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:06.256109 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3b20c55e-c465-4965-a2b0-27939509ef53 req-7a6c1921-ea32-446e-b17d-d83cb4b02159 service nova] Creating event network-vif-unplugged:83f1bf1e-ab89-427f-af3d-df7b7e6537b8 for instance 18be360f-e339-42ec-89ff-72acf428932a on nch1-master-1 Oct 15 21:08:06.262544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3b20c55e-c465-4965-a2b0-27939509ef53 req-7a6c1921-ea32-446e-b17d-d83cb4b02159 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.051369 Oct 15 21:08:06.263735 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2991/4105] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:06 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:06.270796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:06.271749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:06.279015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:06.279808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:06.281749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:06.283639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:06.284268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:06.284858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:06.365858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:06.379333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:06.380799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:06.384160 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad15616e-a338-437e-a638-f9135b21f29c tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1371 microversion: 2.47 time: 0.118014 Oct 15 21:08:06.385921 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2992/4106] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:06 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1371 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:06.418376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:06.424200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:06.438414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:06.443738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:06.444584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:06.445588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:06.446653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:06.448361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:06.973357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:06.992387 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fc1a131b-081b-4722-aaf2-b0be3f3872b5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.580434 Oct 15 21:08:06.993811 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2993/4107] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:06 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:07.234763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:07.235403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:07.254386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:07.255126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:07.255807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:07.256937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:07.257891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:07.258796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:07.991557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:08.000727 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-305fe1b5-3c8f-4740-8a42-6f33faac7bbc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1519 microversion: 2.1 time: 0.771564 Oct 15 21:08:08.008602 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2994/4108] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:07 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1519 bytes in 780 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:08:08.018972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:08.020096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:08.026965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=4,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:08.032630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:08.033770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:08.035337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:08.036975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:08.038450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:08.039400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:08.192421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:08.212641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:08.215698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:08.217973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b2bb3e5-939c-4e27-8e69-44f4793e4e8a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1371 microversion: 2.47 time: 0.203161 Oct 15 21:08:08.235215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2995/4109] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:08 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1371 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:08.241552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:08.242194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:08.256918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:08.257771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:08.258540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:08.259558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:08.260200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:08.260780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:09.251922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:09.262405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bdfcebf9-1eaf-43ae-adfd-8f45fe47d496 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 1.033663 Oct 15 21:08:09.280738 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2996/4110] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:08 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 1046 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:09.287473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7f1ecaec-ce43-4435-8fc4-bee039b6aba9 req-73a43480-735d-4cdd-bd0a-3e283d30ca95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "18be360f-e339-42ec-89ff-72acf428932a", "name": "network-vif-plugged", "status": "completed", "tag": "83f1bf1e-ab89-427f-af3d-df7b7e6537b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:09.301811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f1ecaec-ce43-4435-8fc4-bee039b6aba9 req-73a43480-735d-4cdd-bd0a-3e283d30ca95 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:09.304289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f1ecaec-ce43-4435-8fc4-bee039b6aba9 req-73a43480-735d-4cdd-bd0a-3e283d30ca95 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:09.306113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f1ecaec-ce43-4435-8fc4-bee039b6aba9 req-73a43480-735d-4cdd-bd0a-3e283d30ca95 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:09.324961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7f1ecaec-ce43-4435-8fc4-bee039b6aba9 req-73a43480-735d-4cdd-bd0a-3e283d30ca95 service nova] Creating event network-vif-plugged:83f1bf1e-ab89-427f-af3d-df7b7e6537b8 for instance 18be360f-e339-42ec-89ff-72acf428932a on nch1-master-1 Oct 15 21:08:09.361100 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7f1ecaec-ce43-4435-8fc4-bee039b6aba9 req-73a43480-735d-4cdd-bd0a-3e283d30ca95 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058410 Oct 15 21:08:09.372925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2997/4111] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:09.387655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:09.389097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:09.399645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:09.400759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:09.401611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:09.402647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:09.403349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:09.403968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:10.041268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:10.051647 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01993b46-d556-4f14-9557-dcbc30925fef tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1519 microversion: 2.1 time: 0.671533 Oct 15 21:08:10.059777 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2998/4112] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:09 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1519 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:10.070195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:10.072192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:10.085137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:10.089908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:10.090740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:10.091644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:10.092314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:10.092903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:10.288895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:10.307044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:10.308505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:10.309728 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f41e1523-9fb5-4d57-8a7e-ef8bdf279a5b tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1356 microversion: 2.47 time: 0.248749 Oct 15 21:08:10.331135 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 2999/4113] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:10 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1356 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:10.346357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2046172559", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e8da6ca7-0893-4850-8148-25e3a241b376"}], "security_groups": [{"name": "tempest-securitygroup--1138332832"}], "key_name": "tempest-keypair-1950741633", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:10.487611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] validate_networks() for [('e8da6ca7-0893-4850-8148-25e3a241b376', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:08:11.232070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:08:11.232773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:08:11.235209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:08:11.235867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:08:11.237240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:08:11.254046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:11.255407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:11.256187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:11.257733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:11.259446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:11.260037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:11.300341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:08:11.309817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Getting quotas for user 2f4e4fc2f97f4c1a832c6348a97fd155 and project cfe55a7ec21b4e6ebbf53684a480e535. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:08:11.331557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:08:11.340122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:08:11.409490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-858ff164-7aa9-4162-907c-f8797e65ff96 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.074576 Oct 15 21:08:11.412474 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3000/4114] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:10 2022] POST /compute/v2.1/servers => generated 402 bytes in 1077 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:08:11.415192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:11.415949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:11.423007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:11.423741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:11.424571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:11.425478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:11.425961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:11.426426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:11.704009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:11.720740 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30ef723c-8801-454c-8c01-c5991a0ea054 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.309566 Oct 15 21:08:11.723300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3001/4115] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:11 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:11.728253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:11.735897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:11.737256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Searching by: {'deleted': False, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:08:11.752184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:11.752935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:11.765959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:11.774774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:11.775560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:11.776014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:11.972940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:08:12.163384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:12.195462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:12.202553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:12.204393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:12.247929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:12.251196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:12.409119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f431a4bc-a33d-472b-85b4-77af1582419d tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 2706 microversion: 2.47 time: 0.684452 Oct 15 21:08:12.409722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3002/4116] 10.222.0.16 () {60 vars in 1187 bytes} [Sat Oct 15 21:08:11 2022] GET /compute/v2.1/servers/detail => generated 2706 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:12.415250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:12.415888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:12.427761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:12.428262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:12.428705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:12.429461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:12.429925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:12.430364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:12.692075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:12.731231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f422e828-8c8b-46ea-99a7-a3917c168db8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1504 microversion: 2.1 time: 0.320427 Oct 15 21:08:12.734292 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3003/4117] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:12 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1504 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:12.782370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:12.784689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:12.800647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:12.804806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:12.805608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:12.806599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:12.812091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:12.812831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:12.984637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:12.994084 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f2a4de7-40b9-4046-a159-e5c449c93e1c tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1290 microversion: 2.1 time: 0.234087 Oct 15 21:08:12.995187 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3004/4118] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:12 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1290 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:13.004282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:13.004573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:13.022781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:13.023284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:13.023800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:13.024425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:13.025224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:13.025360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:13.069551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:08:13.251706 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72c7a3e8-d039-4a40-9386-34dbab38eb4a tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 204 len: 0 microversion: 2.47 time: 0.254629 Oct 15 21:08:13.252740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3005/4119] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:08:12 2022] DELETE /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:08:13.260163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:13.260883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:13.270115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:13.270549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:13.270997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:13.271711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:13.273767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:13.274224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:13.297739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:13.685357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:13.694638 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40942ae8-ad4e-488c-8ca4-8acb28a78aaa tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.440124 Oct 15 21:08:13.706290 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3006/4120] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:13 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:13.725230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:13.726261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:13.734640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:13.735280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:13.735887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:13.736823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:13.737440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:13.737984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.071922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:14.085862 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ee61597-daa4-468b-b85a-4e2a7d8a44b0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 200 len: 1504 microversion: 2.1 time: 0.377702 Oct 15 21:08:14.088153 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3007/4121] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:13 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 1504 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:14.104429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:14.105254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:14.114148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.115370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.115870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.117480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.117856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.118511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.215337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:14.226977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:14.228464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:14.231972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83831780-186b-4a62-a250-272378bac2e5 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1362 microversion: 2.47 time: 0.142382 Oct 15 21:08:14.232557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3008/4122] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:14 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1362 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:14.244409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:14.245029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:14.260740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.273443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.274222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.275165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.275816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.276405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.398576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:14.407951 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-34c51547-884e-4515-8e7f-988bca70ab13 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1296 microversion: 2.1 time: 0.169407 Oct 15 21:08:14.413480 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3009/4123] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:14 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1296 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:14.431889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:14.432837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:14.444806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.445461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.447755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.447755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.447755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.447755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.479919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:08:14.659659 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8ffe960-e410-4936-9eeb-ca80c6c42b2e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "DELETE /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 204 len: 0 microversion: 2.1 time: 0.244484 Oct 15 21:08:14.661790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3010/4124] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:08:14 2022] DELETE /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:08:14.738603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:14.739321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:14.752648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.753178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.753627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:14.754346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:14.754810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:14.760829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:15.072033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:15.080874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f8e7937-42f0-42a0-9b5f-44b3019ffafb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.346876 Oct 15 21:08:15.085804 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3011/4125] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:14 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:15.111493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-2144044583", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7cae7909-1e8f-4b21-a25f-4a74daa4a2c1"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:15.264131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] validate_networks() for [('7cae7909-1e8f-4b21-a25f-4a74daa4a2c1', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:08:16.104073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:08:16.104718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:08:16.105733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:08:16.106291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:08:16.106715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:08:16.125626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.126387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.127014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:16.128104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.128577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.129002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:16.174619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Getting quotas for project 96091c3dac03410dbb4f59faa1482781. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:08:16.185360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Getting quotas for user dd615ca74887465b8e21b7f4011fb53f and project 96091c3dac03410dbb4f59faa1482781. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:08:16.221562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:08:16.228134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:08:16.339244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54e448d4-5e19-4e85-b681-21ac60af91fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.252294 Oct 15 21:08:16.346815 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3012/4126] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 1260 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:08:16.358710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:16.358922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:16.375730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.376755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.379457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:16.381005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.382594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.383204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:16.442657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.443339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.443943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:16.445344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lazy-loading 'fault' on Instance uuid e277f250-cc5f-464c-84c3-68fa8122d969 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:08:16.799576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:16.809408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:16.813401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:16.826360 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d1af33a-41eb-4c28-81cb-2060aedd3ccc tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 200 len: 1367 microversion: 2.47 time: 0.478338 Oct 15 21:08:16.833378 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3013/4127] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:16 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 1367 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:16.847576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:16.847576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:16.866750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.867254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.867690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:16.881418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:16.881944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:16.882335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:17.619922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:17.631714 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e275097a-37f0-49c7-868d-404944f64b6b tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1364 microversion: 2.1 time: 0.794355 Oct 15 21:08:17.632887 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3014/4128] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:16 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1364 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:17.646779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:17.647638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:17.659926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:17.660717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:17.661380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:17.662282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:17.666892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:17.667831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:18.411713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:18.423257 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a654177-5228-4d16-8375-ad3a38cb98c6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1505 microversion: 2.1 time: 0.788888 Oct 15 21:08:18.423830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3015/4129] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:17 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1505 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:18.446404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:18.447053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:18.472003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:18.472517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:18.472973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:18.486483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:18.486717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:18.487326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:18.861818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:18.878611 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19d6753a-6657-436d-9d52-0f3581f0d234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1288 microversion: 2.1 time: 0.428124 Oct 15 21:08:18.879564 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3016/4130] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:18 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1288 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:18.892152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:18.893144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: e277f250-cc5f-464c-84c3-68fa8122d969] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:18.909597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:18.912224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:18.912904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:18.915151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:18.916763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:18.917488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:18.953370 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] HTTP exception thrown: Instance e277f250-cc5f-464c-84c3-68fa8122d969 could not be found. Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Request method failure captured: Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 HTTP/1.1 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJBql13Mz-CocpG0jhcEYl0wgtzTFzQwullFMfswjFra3sH8A7kCYZ4NRsDVRvUR68PQGKIY4-JVGV3VmUg_sewocg04UjZnNTqzKmQvfuprm3tWYbGOhLMeFoE2cp4InNrTyD5_QTnbWNw0Mh1Uts9tDmCYd83BIV8x2NF0Uy6AY Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.47 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8c7c33c41d0a4f96bd720b6235d0acad Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerShowV247Test-294118727 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerShowV247Test-294118727 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8c7c33c41d0a4f96bd720b6235d0acad Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerShowV247Test-294118727 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerShowV247Test-294118727-project Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 508ea6505b99421d8d65a42f1b6b3876 Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerShowV247Test-294118727-project Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e277f250-cc5f-464c-84c3-68fa8122d969 could not be found. Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e277f250-cc5f-464c-84c3-68fa8122d969'} Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e277f250-cc5f-464c-84c3-68fa8122d969 could not be found. Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e277f250-cc5f-464c-84c3-68fa8122d969 could not be found. Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:18.960151 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e277f250-cc5f-464c-84c3-68fa8122d969 could not be found. Oct 15 21:08:18.964712 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:18.972674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Returning 404 to user: Instance e277f250-cc5f-464c-84c3-68fa8122d969 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:18.975494 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a82798f4-df36-4d44-a4c5-e8a143e2e7f7 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969" status: 404 len: 111 microversion: 2.47 time: 0.093278 Oct 15 21:08:18.977741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3017/4131] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:18 2022] GET /compute/v2.1/servers/e277f250-cc5f-464c-84c3-68fa8122d969 => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:08:18.988091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:18.993563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:19.010335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:19.011240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:19.011880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:19.014081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:19.014754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:19.018234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:19.662034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:19.674456 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba5e960d-ecc0-44aa-8b7f-a292747b04de tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1364 microversion: 2.1 time: 0.695190 Oct 15 21:08:19.675952 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3018/4132] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:18 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1364 bytes in 697 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:19.689795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5cb7b144-f85d-4d82-824b-76bf96912947 req-665bf111-742b-4c1b-b965-b916c6bc61ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "18be360f-e339-42ec-89ff-72acf428932a", "name": "network-vif-deleted", "tag": "83f1bf1e-ab89-427f-af3d-df7b7e6537b8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:19.726589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5cb7b144-f85d-4d82-824b-76bf96912947 req-665bf111-742b-4c1b-b965-b916c6bc61ff service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:19.727076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5cb7b144-f85d-4d82-824b-76bf96912947 req-665bf111-742b-4c1b-b965-b916c6bc61ff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:19.727512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5cb7b144-f85d-4d82-824b-76bf96912947 req-665bf111-742b-4c1b-b965-b916c6bc61ff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:19.754095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5cb7b144-f85d-4d82-824b-76bf96912947 req-665bf111-742b-4c1b-b965-b916c6bc61ff service nova] Creating event network-vif-deleted:83f1bf1e-ab89-427f-af3d-df7b7e6537b8 for instance 18be360f-e339-42ec-89ff-72acf428932a on nch1-master-1 Oct 15 21:08:19.771552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5cb7b144-f85d-4d82-824b-76bf96912947 req-665bf111-742b-4c1b-b965-b916c6bc61ff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094235 Oct 15 21:08:19.775317 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3019/4133] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:19.784962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:19.795083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: 1d88df21-cddf-4cef-a308-055d6727397d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:19.821074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:19.821074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:19.827324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:19.849500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:19.850141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:19.850665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:20.013812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: 1d88df21-cddf-4cef-a308-055d6727397d] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:08:20.197862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=6,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:20.205861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2af83d3f-cdd4-4da7-b134-03cd432848a4 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d" status: 204 len: 0 microversion: 2.47 time: 0.428866 Oct 15 21:08:20.206987 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3020/4134] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:08:19 2022] DELETE /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:08:20.216302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:20.218569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:20.239534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:20.244558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:20.245533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:20.246788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:20.247656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:20.249213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:20.304765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:21.043528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:21.064698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eca05cca-8d25-4f8a-9a1b-730c731182e3 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1485 microversion: 2.1 time: 0.856161 Oct 15 21:08:21.071070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3021/4135] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:20 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1485 bytes in 863 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:21.081582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:21.096131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:21.127135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:21.127900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:21.137641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:21.145312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:21.145794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:21.146254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:21.897184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:21.912889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5019fd88-a845-41c9-bcc7-7ed2bee2ceac tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1318 microversion: 2.1 time: 0.835193 Oct 15 21:08:21.921656 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3022/4136] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:21 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1318 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:21.939156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:21.948948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: 1d88df21-cddf-4cef-a308-055d6727397d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:21.964105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:21.964846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:21.965462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:21.966324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:21.967550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:21.968131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:22.345275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:22.355070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:22.362928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '508ea6505b99421d8d65a42f1b6b3876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c7c33c41d0a4f96bd720b6235d0acad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:22.364430 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9df0b59b-27a1-4d1a-9b84-242d435a64c1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d" status: 200 len: 1363 microversion: 2.47 time: 0.441532 Oct 15 21:08:22.369657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3023/4137] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:21 2022] GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d => generated 1363 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:22.375428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:22.384712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:22.406207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:22.406713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:22.407158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:22.407874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:22.408334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:22.408763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:23.073396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:23.082985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f1898a3-5101-4038-80d3-bd0499febf92 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1364 microversion: 2.1 time: 0.711810 Oct 15 21:08:23.088351 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3024/4138] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:22 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1364 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:23.093883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSON-server-20668927", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:23.211345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] validate_networks() for [('114bda92-662d-4788-82b0-9f1053954aa2', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:08:24.705964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:08:24.707035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:08:24.708032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:08:24.708631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:08:24.711153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:08:24.730719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:24.731253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:24.731664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:24.734021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:24.734514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:24.734962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:24.775974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Getting quotas for project 5572b130042c482384a1a06e6a91c784. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:08:24.786476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Getting quotas for user 461a5c1e334a47eb9d70908227cca0b2 and project 5572b130042c482384a1a06e6a91c784. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:08:24.822138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:08:24.825466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:08:24.937775 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dcae2b3d-a0a7-4827-826f-9e678045bef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.847986 Oct 15 21:08:24.945534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3025/4139] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:23 2022] POST /compute/v2.1/servers => generated 383 bytes in 1856 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:08:24.951521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-21727b27-6d24-469c-a951-161aec9cd065 req-c183cb26-33ca-4ef6-94b8-3835a3ded980 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b381ebc7-0aed-4382-90c2-832cc391b53e", "tag": "3d83816b-d4d9-4bb6-ad20-6c0105588621"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:24.968634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-21727b27-6d24-469c-a951-161aec9cd065 req-c183cb26-33ca-4ef6-94b8-3835a3ded980 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:24.969750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-21727b27-6d24-469c-a951-161aec9cd065 req-c183cb26-33ca-4ef6-94b8-3835a3ded980 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:24.970385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-21727b27-6d24-469c-a951-161aec9cd065 req-c183cb26-33ca-4ef6-94b8-3835a3ded980 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:24.994008 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-21727b27-6d24-469c-a951-161aec9cd065 req-c183cb26-33ca-4ef6-94b8-3835a3ded980 service nova] Creating event network-changed:3d83816b-d4d9-4bb6-ad20-6c0105588621 for instance b381ebc7-0aed-4382-90c2-832cc391b53e on nch1-master-1 Oct 15 21:08:25.005485 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-21727b27-6d24-469c-a951-161aec9cd065 req-c183cb26-33ca-4ef6-94b8-3835a3ded980 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058278 Oct 15 21:08:25.014195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3026/4140] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:25.028934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:25.029845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:25.047898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:25.051392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:25.052005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:25.052874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:25.057539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:25.058246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:25.730989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:25.744377 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bed63f73-4741-4576-8009-3dde20a21d64 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1318 microversion: 2.1 time: 0.725960 Oct 15 21:08:25.745640 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3027/4141] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:25 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1318 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:25.783470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:25.784873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] [instance: 1d88df21-cddf-4cef-a308-055d6727397d] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:25.804389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:25.808519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:25.808999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:25.812237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:25.814080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:25.814528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:25.874598 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] HTTP exception thrown: Instance 1d88df21-cddf-4cef-a308-055d6727397d could not be found. Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Request method failure captured: Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d HTTP/1.1 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyEJBql13Mz-CocpG0jhcEYl0wgtzTFzQwullFMfswjFra3sH8A7kCYZ4NRsDVRvUR68PQGKIY4-JVGV3VmUg_sewocg04UjZnNTqzKmQvfuprm3tWYbGOhLMeFoE2cp4InNrTyD5_QTnbWNw0Mh1Uts9tDmCYd83BIV8x2NF0Uy6AY Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.47 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8c7c33c41d0a4f96bd720b6235d0acad Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerShowV247Test-294118727 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8c7c33c41d0a4f96bd720b6235d0acad"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerShowV247Test-294118727 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8c7c33c41d0a4f96bd720b6235d0acad Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerShowV247Test-294118727 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerShowV247Test-294118727-project Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 508ea6505b99421d8d65a42f1b6b3876 Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerShowV247Test-294118727-project Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1d88df21-cddf-4cef-a308-055d6727397d could not be found. Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1d88df21-cddf-4cef-a308-055d6727397d'} Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d88df21-cddf-4cef-a308-055d6727397d could not be found. Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d88df21-cddf-4cef-a308-055d6727397d could not be found. Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:25.876537 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d88df21-cddf-4cef-a308-055d6727397d could not be found. Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:25.880154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] Returning 404 to user: Instance 1d88df21-cddf-4cef-a308-055d6727397d could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:25.888295 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c11367b2-ac1e-4622-b185-053ae20eb0e1 tempest-ServerShowV247Test-294118727 tempest-ServerShowV247Test-294118727-project] 10.222.0.16 "GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d" status: 404 len: 111 microversion: 2.47 time: 0.127731 Oct 15 21:08:25.889512 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3028/4142] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:25 2022] GET /compute/v2.1/servers/1d88df21-cddf-4cef-a308-055d6727397d => generated 111 bytes in 140 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:08:25.904721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:25.905363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:25.922114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:25.922610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:25.923046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:25.929390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:25.932231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:25.932870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:26.285125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:26.296422 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a7ad26c-85e2-481f-a39c-34ea85b6d0ea tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1364 microversion: 2.1 time: 0.405613 Oct 15 21:08:26.297396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3029/4143] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:25 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1364 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:26.303024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:26.303816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:26.311837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:26.312523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:26.313164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:26.314056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:26.314558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:26.314991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:26.531522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:26.540199 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43f40c9e-a0ad-43a6-99a7-ecd9e9a78a23 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.241372 Oct 15 21:08:26.540782 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3030/4144] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:26 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:26.549190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:26.549971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:26.557060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:26.557559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:26.558164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:26.559084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:26.559598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:26.560252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:26.645026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:26.654304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24f2f617-e32e-4547-aba1-378ffc4b8e4a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1264 microversion: 2.1 time: 0.112334 Oct 15 21:08:26.655072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3031/4145] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:26 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1264 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:26.971398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerPasswordTestJSON-server-1336311403", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7360494f-8695-4e54-8ffa-1567d5087ace"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:27.067016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] validate_networks() for [('7360494f-8695-4e54-8ffa-1567d5087ace', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:08:27.981442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:08:27.982644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:08:27.983571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:08:27.984378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:08:27.985359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:08:28.007234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:28.009165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:28.010970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:28.013985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:28.017125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:28.017790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:28.067729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Getting quotas for project 93dac014c11c45029f5b8569b7f509d0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:08:28.083810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Getting quotas for user 06e2c573568846d095c2a42a1df4e918 and project 93dac014c11c45029f5b8569b7f509d0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:08:28.115601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:08:28.118674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:08:28.233918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c4215a81-153d-4037-9f95-115f1d597f35 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.576622 Oct 15 21:08:28.235501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3032/4146] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:08:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 1580 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:08:28.242476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-3-481936647"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:28.245487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:28.253021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:28.253815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:28.254414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:28.255289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:28.256246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:28.256919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:28.727695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cac88fad-2b66-4cb1-aa53-2f178498e560 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.489334 Oct 15 21:08:28.729749 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3033/4147] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:08:28 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 493 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Oct 15 21:08:28.740662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:28.741787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:28.753372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:28.754125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:28.754768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:28.755807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:28.757218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:28.757949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:29.118553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:29.129299 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd5494e1-b56a-47d5-88a3-2286fb8e89e8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1318 microversion: 2.1 time: 0.397141 Oct 15 21:08:29.132529 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3034/4148] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:28 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1318 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:29.137780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ec8a12e2-8b5b-4e8c-816e-83147a76ff06 req-001a5c23-64d1-4dc6-ab7e-e5b3f8a3ee0c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9f1f541f-e2cd-4527-b301-15fc8d6b34e9", "tag": "cdfb68d1-a92c-47c1-b12e-066c2ab0987e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:29.160291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec8a12e2-8b5b-4e8c-816e-83147a76ff06 req-001a5c23-64d1-4dc6-ab7e-e5b3f8a3ee0c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:29.160947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec8a12e2-8b5b-4e8c-816e-83147a76ff06 req-001a5c23-64d1-4dc6-ab7e-e5b3f8a3ee0c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:29.161642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec8a12e2-8b5b-4e8c-816e-83147a76ff06 req-001a5c23-64d1-4dc6-ab7e-e5b3f8a3ee0c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:29.190626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ec8a12e2-8b5b-4e8c-816e-83147a76ff06 req-001a5c23-64d1-4dc6-ab7e-e5b3f8a3ee0c service nova] Creating event network-changed:cdfb68d1-a92c-47c1-b12e-066c2ab0987e for instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 on nch1-master-1 Oct 15 21:08:29.196364 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ec8a12e2-8b5b-4e8c-816e-83147a76ff06 req-001a5c23-64d1-4dc6-ab7e-e5b3f8a3ee0c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062109 Oct 15 21:08:29.197617 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3035/4149] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:29.218922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:29.221942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:29.231532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:29.232371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:29.233357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:29.234475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:29.235098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:29.235713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:30.045490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:30.061123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a13b7cd5-c496-44c3-97b5-f084dcf384d0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1364 microversion: 2.1 time: 0.861528 Oct 15 21:08:30.063769 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3036/4150] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:29 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1364 bytes in 865 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:30.072678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:30.073766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:30.093375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:30.106516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:30.107378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:30.109741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:30.110759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:30.111383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:30.610301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:30.619949 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce1527b0-4da2-4c74-8a44-1e4e3a78815c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.554869 Oct 15 21:08:30.620955 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3037/4151] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:30 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:30.633165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0a6d46f2-1421-43e3-a8e7-88e3fd8cf115 req-d014fc7b-0c84-477d-9988-a2f4e3a8e609 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b381ebc7-0aed-4382-90c2-832cc391b53e", "name": "network-vif-plugged", "status": "completed", "tag": "3d83816b-d4d9-4bb6-ad20-6c0105588621"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:30.650676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a6d46f2-1421-43e3-a8e7-88e3fd8cf115 req-d014fc7b-0c84-477d-9988-a2f4e3a8e609 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:30.651569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a6d46f2-1421-43e3-a8e7-88e3fd8cf115 req-d014fc7b-0c84-477d-9988-a2f4e3a8e609 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:30.652102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0a6d46f2-1421-43e3-a8e7-88e3fd8cf115 req-d014fc7b-0c84-477d-9988-a2f4e3a8e609 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:30.667827 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0a6d46f2-1421-43e3-a8e7-88e3fd8cf115 req-d014fc7b-0c84-477d-9988-a2f4e3a8e609 service nova] Creating event network-vif-plugged:3d83816b-d4d9-4bb6-ad20-6c0105588621 for instance b381ebc7-0aed-4382-90c2-832cc391b53e on nch1-master-1 Oct 15 21:08:30.675655 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0a6d46f2-1421-43e3-a8e7-88e3fd8cf115 req-d014fc7b-0c84-477d-9988-a2f4e3a8e609 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053174 Oct 15 21:08:30.677369 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3038/4152] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:30.685468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:30.686927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:30.698271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:30.699667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:30.700557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:30.704004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:30.704534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:30.704982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:30.977179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:31.008752 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-44887afd-cbad-465d-b5f1-1ec9d5cfa6f3 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1274 microversion: 2.1 time: 0.329927 Oct 15 21:08:31.012065 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3039/4153] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:30 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1274 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:31.383199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-351370ef-8315-4515-aacc-97a9f8a2aac9 tempest-ExtensionsTest-2147119447 tempest-ExtensionsTest-2147119447-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:31.385751 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-351370ef-8315-4515-aacc-97a9f8a2aac9 tempest-ExtensionsTest-2147119447 tempest-ExtensionsTest-2147119447-project] 10.222.0.16 "GET /compute/v2.1/extensions/os-consoles" status: 200 len: 217 microversion: 2.1 time: 0.372494 Oct 15 21:08:31.386299 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3040/4154] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 21:08:31 2022] GET /compute/v2.1/extensions/os-consoles => generated 217 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:31.398793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:31.399668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:31.415094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:31.415777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:31.416228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:31.416962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:31.417515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:31.417957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:31.832755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:31.837448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:31.845693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40ec428c-38f3-46b8-8a87-99cd77ba6917 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.458099 Oct 15 21:08:31.846529 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3041/4155] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:31 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:31.851303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:31.852281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:31.868820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:31.869514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:31.870025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:31.871399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:31.871935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:31.872520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.201682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:32.212861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-baa12871-ea70-42ad-88d5-ec08e07e6379 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1318 microversion: 2.1 time: 0.365029 Oct 15 21:08:32.213494 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3042/4156] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:31 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1318 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:32.227534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:32.228455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:32.259286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.262261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.262965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.263660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.265174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.267777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.585511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:32.600559 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57848b0f-ec93-4e29-9637-5237dfee14e7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1550 microversion: 2.1 time: 0.378698 Oct 15 21:08:32.601709 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3043/4157] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:32 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1550 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:32.606042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8f86911c-1b8f-41a6-83be-9363d1c59ca9 req-b2a6c888-45b2-466f-9921-81abd84e0e98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "91f53d83-e9a4-425b-ba6c-5685483898e5", "tag": "0a05acd5-32b4-4c32-a416-77c9c17a3aec"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:32.620971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8f86911c-1b8f-41a6-83be-9363d1c59ca9 req-b2a6c888-45b2-466f-9921-81abd84e0e98 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.621419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8f86911c-1b8f-41a6-83be-9363d1c59ca9 req-b2a6c888-45b2-466f-9921-81abd84e0e98 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.621860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8f86911c-1b8f-41a6-83be-9363d1c59ca9 req-b2a6c888-45b2-466f-9921-81abd84e0e98 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.642216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8f86911c-1b8f-41a6-83be-9363d1c59ca9 req-b2a6c888-45b2-466f-9921-81abd84e0e98 service nova] Creating event network-changed:0a05acd5-32b4-4c32-a416-77c9c17a3aec for instance 91f53d83-e9a4-425b-ba6c-5685483898e5 on nch2-master-1 Oct 15 21:08:32.648006 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8f86911c-1b8f-41a6-83be-9363d1c59ca9 req-b2a6c888-45b2-466f-9921-81abd84e0e98 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045443 Oct 15 21:08:32.648638 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3044/4158] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:32.655097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e166070-4a65-4f26-85a8-c6bd4adce6e7 tempest-ExtensionsTest-2147119447 tempest-ExtensionsTest-2147119447-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:32.657152 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e166070-4a65-4f26-85a8-c6bd4adce6e7 tempest-ExtensionsTest-2147119447 tempest-ExtensionsTest-2147119447-project] 10.222.0.16 "GET /compute/v2.1/extensions" status: 200 len: 22700 microversion: 2.1 time: 0.007095 Oct 15 21:08:32.658041 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3045/4159] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 21:08:32 2022] GET /compute/v2.1/extensions => generated 22700 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:08:32.663381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:32.664100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:32.670436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.671104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.671548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.672244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.672854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.673314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.931317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:32.941532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fea9765e-0839-4810-b06a-9b39801d275a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.282313 Oct 15 21:08:32.954012 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3046/4160] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:32 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:32.965507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:32.967149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:32.973378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.974055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.975079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:32.976075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:32.976687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:32.977312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.255859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:33.267485 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab13dca8-054d-42f9-b6a1-a3081f845e7d tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.311059 Oct 15 21:08:33.269370 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3047/4161] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:32 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:33.274808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:33.275495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:33.286865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.288077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.288711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.290766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.291394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.291998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.395239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-565cd13a-9849-4e19-8a3c-30dde58d6a88 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.124308 Oct 15 21:08:33.395956 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3048/4162] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:08:33 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-interface => generated 284 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:33.408071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-unplugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}, {"server_uuid": "9f1f541f-e2cd-4527-b301-15fc8d6b34e9", "name": "network-vif-plugged", "status": "completed", "tag": "cdfb68d1-a92c-47c1-b12e-066c2ab0987e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:33.421351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.422049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.422690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.441317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] Creating event network-vif-unplugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:08:33.442390 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] Creating event network-vif-plugged:cdfb68d1-a92c-47c1-b12e-066c2ab0987e for instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 on nch1-master-1 Oct 15 21:08:33.455390 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4f089d7f-4117-4c24-97e5-a6e987f946a1 req-4a464fcd-5cbf-4973-901e-f384581d7f0f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.057806 Oct 15 21:08:33.461564 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3049/4163] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:33 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:33.469903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:33.470796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:33.478949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.479515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.479978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.480831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.481332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.481789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.844904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:33.853733 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27bcb522-3b22-411a-83c1-95b40ddb2b9b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1689 microversion: 2.1 time: 0.390031 Oct 15 21:08:33.854425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3050/4164] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:33 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1689 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:33.860712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:33.862016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:33.868765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.869939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.870469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:33.871255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:33.871732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:33.872171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.120940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:34.130475 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4cfdc79-8421-4a38-84c3-4bd313bcd31c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1318 microversion: 2.1 time: 0.274973 Oct 15 21:08:34.131066 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3051/4165] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:33 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1318 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:34.136617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:34.137271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:34.145065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.146310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.146764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.147489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.147958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.148397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.410967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:34.419694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e16c1973-5708-4dc3-8c85-eb66c612c728 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.287190 Oct 15 21:08:34.431032 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3052/4166] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:34 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:34.445072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:34.446165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:34.454120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.454680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.455131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.456071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.456693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.457444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.718508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:34.728826 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffb38ef3-a7b5-49dd-be68-ff265d02e94a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.287599 Oct 15 21:08:34.729467 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3053/4167] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:34 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:34.736329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-105d19bf-f3bb-476b-8324-15951c7fca24 req-d257e2d8-b731-40f5-abf8-19e743cb7cfd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "66b37c78-8e50-4dde-80af-ca3fcb2c67ae", "tag": "edfad690-e63a-423d-95b3-5b28b2bda4e1"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:34.760494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-105d19bf-f3bb-476b-8324-15951c7fca24 req-d257e2d8-b731-40f5-abf8-19e743cb7cfd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.760908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-105d19bf-f3bb-476b-8324-15951c7fca24 req-d257e2d8-b731-40f5-abf8-19e743cb7cfd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.761375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-105d19bf-f3bb-476b-8324-15951c7fca24 req-d257e2d8-b731-40f5-abf8-19e743cb7cfd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.785632 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-105d19bf-f3bb-476b-8324-15951c7fca24 req-d257e2d8-b731-40f5-abf8-19e743cb7cfd service nova] Creating event network-changed:edfad690-e63a-423d-95b3-5b28b2bda4e1 for instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae on nch1-master-1 Oct 15 21:08:34.803305 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-105d19bf-f3bb-476b-8324-15951c7fca24 req-d257e2d8-b731-40f5-abf8-19e743cb7cfd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061492 Oct 15 21:08:34.804017 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3054/4168] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:34.884116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:34.884963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:34.909473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.909992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.910441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:34.911188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:34.911661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:34.912096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:35.807295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:35.821882 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52daf90a-bc74-4072-9b17-c0c212e3908c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1689 microversion: 2.1 time: 0.946192 Oct 15 21:08:35.825111 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3055/4169] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:34 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1689 bytes in 950 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:35.848886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:35.849893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:35.863692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:35.865186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:35.866312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:35.867479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:35.871641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:35.875514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:36.180251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:36.191727 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18c9440d-59b3-4ef3-8e3a-df06ee235abc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1507 microversion: 2.1 time: 0.364555 Oct 15 21:08:36.209430 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3056/4170] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:35 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1507 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:36.231077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9fbb8e74-192f-46c0-a63f-b2b493242cf3 req-11d915a6-130c-4d0e-ba99-0c7850580c77 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b381ebc7-0aed-4382-90c2-832cc391b53e", "tag": "3d83816b-d4d9-4bb6-ad20-6c0105588621"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:36.259182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:UPDATE=1,nova_api:SELECT=93,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:36.270263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9fbb8e74-192f-46c0-a63f-b2b493242cf3 req-11d915a6-130c-4d0e-ba99-0c7850580c77 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:36.271108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9fbb8e74-192f-46c0-a63f-b2b493242cf3 req-11d915a6-130c-4d0e-ba99-0c7850580c77 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:36.271671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9fbb8e74-192f-46c0-a63f-b2b493242cf3 req-11d915a6-130c-4d0e-ba99-0c7850580c77 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:36.313631 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9fbb8e74-192f-46c0-a63f-b2b493242cf3 req-11d915a6-130c-4d0e-ba99-0c7850580c77 service nova] Creating event network-changed:3d83816b-d4d9-4bb6-ad20-6c0105588621 for instance b381ebc7-0aed-4382-90c2-832cc391b53e on nch1-master-1 Oct 15 21:08:36.319595 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9fbb8e74-192f-46c0-a63f-b2b493242cf3 req-11d915a6-130c-4d0e-ba99-0c7850580c77 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106371 Oct 15 21:08:36.327519 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3057/4171] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:36.334149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:36.334804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:36.342537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:36.343084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:36.343531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:36.344241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:36.344702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:36.345179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:36.492829 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:08:36.493918 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:08:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:08:36.860348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:36.871261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e36642a-8b6f-4cc7-888f-3e2cd0c9bdef tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.542285 Oct 15 21:08:36.873391 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3058/4172] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:36 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:36.880576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:36.881245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:36.897800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:36.898470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:36.898992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:36.899659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:36.900121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:36.901392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:37.276860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:37.296294 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b966347-e468-4dc1-a2d4-59a613cb0cae tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.419752 Oct 15 21:08:37.297677 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3059/4173] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:36 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:37.303570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:37.307063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:37.315039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:37.315620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:37.316265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:37.318208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:37.318837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:37.323456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:37.670495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:37.681938 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1377a53c-0c44-486b-9cf8-7b975cd0be2b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1507 microversion: 2.1 time: 0.381706 Oct 15 21:08:37.683211 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3060/4174] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:37 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1507 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:37.691800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:37.692770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:37.702031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:37.702667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:37.703384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:37.705018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:37.706541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:37.707368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:37.924159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:37.932736 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-520bd015-b52f-4bd6-8453-e527881ad83e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1689 microversion: 2.1 time: 0.248096 Oct 15 21:08:37.933346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3061/4175] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:37 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1689 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:37.938648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb6fa004-767b-4b0b-bee1-396acd28023f tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe7585765e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:37.991072 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb6fa004-767b-4b0b-bee1-396acd28023f tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4aa48e67-a448-47fe-bdb1-900c59844bdb" status: 200 len: 343 microversion: 2.1 time: 0.056661 Oct 15 21:08:37.991825 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3062/4176] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:08:37 2022] GET /compute/v2.1/os-volumes/4aa48e67-a448-47fe-bdb1-900c59844bdb => generated 343 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:38.045181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=10 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:38.102292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=30 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:38.194553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6397a935-8e40-44e4-86d7-d877dc158d15 req-2bca5c34-7784-4f24-878c-e4fab75d7c3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-plugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:38.206219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6397a935-8e40-44e4-86d7-d877dc158d15 req-2bca5c34-7784-4f24-878c-e4fab75d7c3a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.206959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6397a935-8e40-44e4-86d7-d877dc158d15 req-2bca5c34-7784-4f24-878c-e4fab75d7c3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.207461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6397a935-8e40-44e4-86d7-d877dc158d15 req-2bca5c34-7784-4f24-878c-e4fab75d7c3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.224477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6397a935-8e40-44e4-86d7-d877dc158d15 req-2bca5c34-7784-4f24-878c-e4fab75d7c3a service nova] Creating event network-vif-plugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:08:38.230958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6397a935-8e40-44e4-86d7-d877dc158d15 req-2bca5c34-7784-4f24-878c-e4fab75d7c3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.236013 Oct 15 21:08:38.231594 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3063/4177] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:38.238571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:38.241411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:38.249189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.249861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.250308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.251019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.251483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.251922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.256791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:38.337706 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4f1279f2-39db-4ad2-aff8-ff1ebee73096 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "PUT /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1079 microversion: 2.1 time: 0.103492 Oct 15 21:08:38.339058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3064/4178] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:08:38 2022] PUT /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1079 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:38.347553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:38.348378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:38.356337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.357350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.357860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.358821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.359415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.359985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.528721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:38.538930 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c4bc552a-ae15-466b-8ea2-11de5ac5e6a0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.198523 Oct 15 21:08:38.539604 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3065/4179] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:38 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:38.545087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:38.546063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:38.552218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.552780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.553399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.554381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.554958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.555626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.697918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:38.706362 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b273d67-6771-42b7-bf16-552a988c0496 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.165701 Oct 15 21:08:38.706934 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3066/4180] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:38 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:38.711552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:38.712158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:38.718701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.719189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.719635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.720348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.720809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.721277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.858386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:38.866915 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1ff007c-ff97-41d5-93ba-6527091b6589 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1505 microversion: 2.1 time: 0.158987 Oct 15 21:08:38.867784 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3067/4181] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:38 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1505 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:38.890583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:38.891132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:38.897470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.897949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.898394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:38.899132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:38.899620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:38.900054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.056803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:39.070717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-143221b9-092a-42e6-a600-1be89b16fdb7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1505 microversion: 2.1 time: 0.184486 Oct 15 21:08:39.084242 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3068/4182] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:38 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1505 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:39.091704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:39.092650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:39.099718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.100201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.100668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.102157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.102677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.103271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.299839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:39.309469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b914347-a346-440c-9eab-137a9c57a08c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.236428 Oct 15 21:08:39.310581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3069/4183] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:39 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:39.316268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "OS-DCF:diskConfig": "MANUAL"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:39.320358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:39.329498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.330537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.330986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.331709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.332166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.332779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.460609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:08:39.461087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:08:39.461579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:08:39.463040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:08:39.463488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:08:39.563830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:39.583362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.584316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.584860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.593229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.594724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.596153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.632939 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1563ff0c-06f5-4af9-ab78-72ae0fa151eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "POST /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9/action" status: 202 len: 1111 microversion: 2.1 time: 0.321017 Oct 15 21:08:39.634845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3070/4184] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:08:39 2022] POST /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9/action => generated 1111 bytes in 323 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Oct 15 21:08:39.642292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc920049-57e9-4fa7-8030-92930bd747b1 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe7584fc280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:39.728673 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc920049-57e9-4fa7-8030-92930bd747b1 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/e42c7dad-e876-4ee6-8e70-b93b4704449b" status: 200 len: 342 microversion: 2.1 time: 0.092079 Oct 15 21:08:39.733104 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3071/4185] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:08:39 2022] GET /compute/v2.1/os-volumes/e42c7dad-e876-4ee6-8e70-b93b4704449b => generated 342 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:39.735913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:39.736528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:39.775269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.775802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.781793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:39.784623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:39.785182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:39.785638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.062730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:40.086698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-68e29d4b-74c9-452d-b669-c8a29e00d2e7 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.353934 Oct 15 21:08:40.087630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3072/4186] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:39 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:40.094107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:40.094728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:40.100948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.101477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.101916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.102703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.103174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.103719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.344002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:40.357183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6134c713-39dd-41ac-90b4-3217394165fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.267407 Oct 15 21:08:40.358963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3073/4187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:40 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:40.365136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:40.365945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:40.372031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.382230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.382928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.383908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.384765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.385251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.583424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:40.593064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee962571-b657-46af-839b-c6f85feeda1a tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.232833 Oct 15 21:08:40.594906 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3074/4188] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:40 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:40.600218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a31f7766-a22e-4847-ae73-decd9069d5cf req-23e88e6c-399b-4216-9de8-66a269f7f6e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91f53d83-e9a4-425b-ba6c-5685483898e5", "name": "network-vif-plugged", "status": "completed", "tag": "0a05acd5-32b4-4c32-a416-77c9c17a3aec"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:40.619062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a31f7766-a22e-4847-ae73-decd9069d5cf req-23e88e6c-399b-4216-9de8-66a269f7f6e3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.623768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a31f7766-a22e-4847-ae73-decd9069d5cf req-23e88e6c-399b-4216-9de8-66a269f7f6e3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.624244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a31f7766-a22e-4847-ae73-decd9069d5cf req-23e88e6c-399b-4216-9de8-66a269f7f6e3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.647965 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a31f7766-a22e-4847-ae73-decd9069d5cf req-23e88e6c-399b-4216-9de8-66a269f7f6e3 service nova] Creating event network-vif-plugged:0a05acd5-32b4-4c32-a416-77c9c17a3aec for instance 91f53d83-e9a4-425b-ba6c-5685483898e5 on nch2-master-1 Oct 15 21:08:40.654365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a31f7766-a22e-4847-ae73-decd9069d5cf req-23e88e6c-399b-4216-9de8-66a269f7f6e3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057404 Oct 15 21:08:40.655197 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3075/4189] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:40.662244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:40.663186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:40.687951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.688458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.688904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.692061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:40.692550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:40.692996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:40.951796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:40.962780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-803facdd-9a7a-497d-8b5f-f227469231cb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.305763 Oct 15 21:08:40.963856 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3076/4190] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:40 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:41.114387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:41.116320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:41.129079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:41.130459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:41.131368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:41.132253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:41.133280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:41.133905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:41.321589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:41.332964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c0e4023-5daf-4f08-ae51-4a4a035e0991 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.222837 Oct 15 21:08:41.333581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3077/4191] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:41 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:08:41.384857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:41.385488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:41.396385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:41.398458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:41.401587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:41.402497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:41.403144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:41.413935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:41.699774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:41.715133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-204967e0-4855-4fbd-a44d-aead19f8aedc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.332266 Oct 15 21:08:41.717326 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3078/4192] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:41 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:08:41.726079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cca5ca87-ff22-4b8a-ae31-6bda8d8e55cc tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe75439b790>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:41.808404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cca5ca87-ff22-4b8a-ae31-6bda8d8e55cc tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/8ba2a78e-17d9-4660-9456-63938fc30899" status: 200 len: 344 microversion: 2.1 time: 0.088562 Oct 15 21:08:41.808986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3079/4193] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:08:41 2022] GET /compute/v2.1/os-volumes/8ba2a78e-17d9-4660-9456-63938fc30899 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:41.814718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:41.816570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:41.829315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:41.830794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:41.831281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:41.832015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:41.832489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:41.833399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.006099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:42.017365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:42.021187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39767223-5688-488c-bcec-493ba935c107 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.210458 Oct 15 21:08:42.022442 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3080/4194] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:41 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:42.028854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6f0fd03-422c-4036-834c-c2a9c3ac92c4 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe754590550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:42.125186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6f0fd03-422c-4036-834c-c2a9c3ac92c4 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1012 microversion: 2.1 time: 0.100014 Oct 15 21:08:42.127041 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3081/4195] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 21:08:42 2022] GET /compute/v2.1/os-volumes => generated 1012 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:42.132669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:42.133405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:42.142168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.142743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.143361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.144262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.145023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.145497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.319662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:42.328414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86cd6c51-f708-4654-8a28-1613deb89f80 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.200089 Oct 15 21:08:42.329394 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3082/4196] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:42 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:42.343613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3983947-b7ad-416b-8ccf-56d267c2a264 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe758462160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:42.473149 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3983947-b7ad-416b-8ccf-56d267c2a264 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 679 microversion: 2.1 time: 0.142395 Oct 15 21:08:42.473783 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3083/4197] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 21:08:42 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 679 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:42.480714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:42.483239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:42.495861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.497961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.498613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.499739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.500361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.500931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.660055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:42.669961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af76f6c2-b900-485c-a154-38e539b8c110 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1312 microversion: 2.1 time: 0.194020 Oct 15 21:08:42.688887 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3084/4198] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:42 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1312 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:42.694110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e885f9b-4f08-4347-b94c-0e12e4584a8e tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe754a85310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:42.779373 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e885f9b-4f08-4347-b94c-0e12e4584a8e tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1012 microversion: 2.1 time: 0.089266 Oct 15 21:08:42.779953 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3085/4199] 10.222.0.16 () {58 vars in 1134 bytes} [Sat Oct 15 21:08:42 2022] GET /compute/v2.1/os-volumes => generated 1012 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:42.784416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7f37e48c-75a6-4a97-96b5-fc2834ef9ec7 req-da58c68c-3ae7-4f5e-a09f-c923eb326e54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "66b37c78-8e50-4dde-80af-ca3fcb2c67ae", "name": "network-vif-plugged", "status": "completed", "tag": "edfad690-e63a-423d-95b3-5b28b2bda4e1"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:42.799539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f37e48c-75a6-4a97-96b5-fc2834ef9ec7 req-da58c68c-3ae7-4f5e-a09f-c923eb326e54 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.799943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f37e48c-75a6-4a97-96b5-fc2834ef9ec7 req-da58c68c-3ae7-4f5e-a09f-c923eb326e54 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.802773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7f37e48c-75a6-4a97-96b5-fc2834ef9ec7 req-da58c68c-3ae7-4f5e-a09f-c923eb326e54 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.814905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7f37e48c-75a6-4a97-96b5-fc2834ef9ec7 req-da58c68c-3ae7-4f5e-a09f-c923eb326e54 service nova] Creating event network-vif-plugged:edfad690-e63a-423d-95b3-5b28b2bda4e1 for instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae on nch1-master-1 Oct 15 21:08:42.820026 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7f37e48c-75a6-4a97-96b5-fc2834ef9ec7 req-da58c68c-3ae7-4f5e-a09f-c923eb326e54 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038796 Oct 15 21:08:42.820741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3086/4200] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:42.826490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:42.827207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:42.833778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.834225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.834696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:42.835379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:42.835847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:42.836283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:43.064562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:43.076401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfdcc2a1-04a0-4ef7-893e-79f8a7a1f83f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.251355 Oct 15 21:08:43.077399 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3087/4201] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:42 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:43.085593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e5eb00e-02a4-4697-9cc0-00f791c846a8 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe754a85310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:43.210610 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e5eb00e-02a4-4697-9cc0-00f791c846a8 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 345 microversion: 2.1 time: 0.130859 Oct 15 21:08:43.211798 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3088/4202] 10.222.0.16 () {58 vars in 1167 bytes} [Sat Oct 15 21:08:43 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 345 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:43.218982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:43.219603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:43.227745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:43.228265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:43.228699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:43.229594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:43.230242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:43.230851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:43.413027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:43.422242 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-286dfc6a-afe9-4699-93e0-6e23bf7b5b89 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1316 microversion: 2.1 time: 0.208730 Oct 15 21:08:43.423408 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3089/4203] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:43 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1316 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:43.428226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24c53943-f45d-4ede-b50e-19aacb116600 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe74faa0700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:43.532285 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24c53943-f45d-4ede-b50e-19aacb116600 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 679 microversion: 2.1 time: 0.107485 Oct 15 21:08:43.533123 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3090/4204] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Oct 15 21:08:43 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 679 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:43.537786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:43.538592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:43.552032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:43.554306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:43.554765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:43.555690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:43.556210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:43.556736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:43.732835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:43.742898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3539c99f-88ff-4df5-a18a-66e38f1bfcc3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.208567 Oct 15 21:08:43.743772 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3091/4205] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:43 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:43.749748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c83c7ee-b745-451a-b5b8-b0b352fa4ee0 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe74fa3c430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:43.857569 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c83c7ee-b745-451a-b5b8-b0b352fa4ee0 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1012 microversion: 2.1 time: 0.112573 Oct 15 21:08:43.858150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3092/4206] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 21:08:43 2022] GET /compute/v2.1/os-volumes/detail => generated 1012 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:43.880725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:43.881701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:43.893703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:43.894201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:43.894644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:43.895354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:43.895979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:43.896481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:44.063905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:44.073662 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a348bb3-1670-4018-8709-e69cfff43bd5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1497 microversion: 2.1 time: 0.211481 Oct 15 21:08:44.076514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3093/4207] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:43 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1497 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:44.102460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1eaa4a78-cf7e-428b-8a11-8d9ff8374c2a tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe75846e430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:44.206820 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1eaa4a78-cf7e-428b-8a11-8d9ff8374c2a tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 345 microversion: 2.1 time: 0.115807 Oct 15 21:08:44.208057 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3094/4208] 10.222.0.16 () {58 vars in 1188 bytes} [Sat Oct 15 21:08:44 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 345 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:44.230665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:44.233900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:44.240203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:44.240702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:44.241176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:44.241895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:44.242354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:44.243629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:44.437370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:44.446953 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb58a54e-cf9e-4136-a599-a5896435158d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.220843 Oct 15 21:08:44.447689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3095/4209] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:44 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:44.453833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:44.456615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:44.463266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:44.463857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:44.464400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:44.465469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:44.466718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:44.467229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:44.496013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lazy-loading 'flavor' on Instance uuid 91f53d83-e9a4-425b-ba6c-5685483898e5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:08:44.603255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7a52839-dbb4-4c04-877a-9573eae13642 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "POST /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5/action" status: 200 len: 32 microversion: 2.1 time: 0.153983 Oct 15 21:08:44.604785 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3096/4210] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:08:44 2022] POST /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5/action => generated 32 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:08:44.612263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8abca644-8803-42ae-a274-f6272c48d596 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] Calling method '.version_select at 0x7fe754706940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:44.728809 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8abca644-8803-42ae-a274-f6272c48d596 tempest-VolumesTestJSON-1578423100 tempest-VolumesTestJSON-1578423100-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1012 microversion: 2.1 time: 0.122640 Oct 15 21:08:44.730133 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3097/4211] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 21:08:44 2022] GET /compute/v2.1/os-volumes/detail => generated 1012 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:44.735463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:44.736187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:44.755341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:44.755852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:44.756300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:44.757052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:44.757523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:44.757961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.099541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:45.110982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6bf07fe4-081d-4388-bdfa-838128f0e184 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1503 microversion: 2.1 time: 0.379753 Oct 15 21:08:45.128609 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3098/4212] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:44 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1503 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:45.138436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:45.139070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:45.150019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.150512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.151529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.152263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.154448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.154576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.504659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:45.517142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9661033c-a78d-4a4b-aeac-e141d0e6b715 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.387120 Oct 15 21:08:45.536917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3099/4213] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:45 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:45.545230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:45.546138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:45.554155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.554982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.555489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.556603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.557491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.558103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.768514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:45.779282 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33315895-d285-4f3d-b0dd-92040ed3ba4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.240113 Oct 15 21:08:45.789294 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3100/4214] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:45 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:45.789294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:45.789294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:45.792143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.797093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.797367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.798111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.798584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.799036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.909608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0eb0926-8f0d-479a-92f0-a6866f3d8913 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "DELETE /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae/os-server-password" status: 204 len: 0 microversion: 2.1 time: 0.128597 Oct 15 21:08:45.911740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3101/4215] 10.222.0.16 () {58 vars in 1296 bytes} [Sat Oct 15 21:08:45 2022] DELETE /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae/os-server-password => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:08:45.919676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:45.920602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:45.928760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.929542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.930532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:45.931292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:45.931755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:45.932186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.247940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:46.254956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=3,nova_cell1:INSERT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:46.258602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d280fb5-67ba-454b-aa01-8c657319c134 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.344838 Oct 15 21:08:46.259393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3102/4216] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:45 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:46.270543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:46.271614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:46.284211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.285683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.286139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.287032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.287655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.288287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.327403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c72297c-af90-4c30-b186-006170dcf9cc tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae/os-server-password" status: 200 len: 16 microversion: 2.1 time: 0.066772 Oct 15 21:08:46.327993 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3103/4217] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:08:46 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae/os-server-password => generated 16 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:08:46.351944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:46.352456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:46.360073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.363984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.364697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.366641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.367124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.368120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.406501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:08:46.565056 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2e52a76-0792-4386-b475-62bad14c0d79 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "DELETE /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 204 len: 0 microversion: 2.1 time: 0.228217 Oct 15 21:08:46.567852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3104/4218] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:08:46 2022] DELETE /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 0 bytes in 232 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:08:46.578050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:46.579204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:46.586991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.588284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.589283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.590090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.590746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.591729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.847796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:46.859535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c02775b4-6118-4897-a8e5-c50a15317afc tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.287745 Oct 15 21:08:46.860500 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3105/4219] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:46 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:46.868398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:46.869021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:46.879527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.880056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.880873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:46.881849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:46.882222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:46.885264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:47.060209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:47.070707 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58fd136a-548e-4386-9b08-30cacaa2e325 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1509 microversion: 2.1 time: 0.209124 Oct 15 21:08:47.071303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3106/4220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:46 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1509 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:47.080874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:47.081905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:47.088820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:47.089566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:47.090447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:47.092021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:47.092491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:47.092928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:47.249895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:47.261083 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07277252-384f-4c37-a964-362fd1505e52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.188660 Oct 15 21:08:47.284177 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3107/4221] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:47 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:47.287843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:47.288579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:47.300338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:47.300847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:47.301327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:47.302317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:47.303301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:47.304189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:47.531503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:47.546066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f4cd27f-f911-4d05-a285-6f7eabdfe76a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.262066 Oct 15 21:08:47.546999 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3108/4222] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:47 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:47.890382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:47.891111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:47.902468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:47.905339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:47.906540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:47.908847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:47.909974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:47.910446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:48.724341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:48.739925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a459d358-2d25-493f-aee5-2c96bf93888d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.856639 Oct 15 21:08:48.749421 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3109/4223] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:47 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:48.761660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:48.762675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:48.769220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:48.770507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:48.771323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:48.772140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:48.772688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:48.773370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:49.008368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06e2c573568846d095c2a42a1df4e918', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '93dac014c11c45029f5b8569b7f509d0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:49.018616 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c92a3b6-5f57-4fba-8c94-2c6fb38d3606 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 200 len: 1509 microversion: 2.1 time: 0.267860 Oct 15 21:08:49.033971 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3110/4224] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:48 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 1509 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:49.045725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:49.046692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:49.055521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:49.056116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:49.056661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:49.057593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:49.058168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:49.061877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:49.285642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:49.298025 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fcefe85f-1bec-4d56-b38c-3a89282fdec3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.262224 Oct 15 21:08:49.319311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3111/4225] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:49 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:49.332056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:49.332950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:49.355706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:49.356710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:49.357270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:49.359801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:49.360283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:49.360734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:49.806877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:49.818527 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21e2b7d6-23bb-48d2-b77d-e0908596fdf8 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.497956 Oct 15 21:08:49.819523 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3112/4226] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:49 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:50.204510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d937295f-2005-485f-af8b-51cc4283a0e7 req-f315d5be-86e0-42b2-a2d0-a72d72025f38 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "66b37c78-8e50-4dde-80af-ca3fcb2c67ae", "name": "network-vif-deleted", "tag": "edfad690-e63a-423d-95b3-5b28b2bda4e1"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:50.217985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:50.220527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d937295f-2005-485f-af8b-51cc4283a0e7 req-f315d5be-86e0-42b2-a2d0-a72d72025f38 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.221401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d937295f-2005-485f-af8b-51cc4283a0e7 req-f315d5be-86e0-42b2-a2d0-a72d72025f38 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.222110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d937295f-2005-485f-af8b-51cc4283a0e7 req-f315d5be-86e0-42b2-a2d0-a72d72025f38 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.240659 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d937295f-2005-485f-af8b-51cc4283a0e7 req-f315d5be-86e0-42b2-a2d0-a72d72025f38 service nova] Creating event network-vif-deleted:edfad690-e63a-423d-95b3-5b28b2bda4e1 for instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae on nch1-master-1 Oct 15 21:08:50.248738 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d937295f-2005-485f-af8b-51cc4283a0e7 req-f315d5be-86e0-42b2-a2d0-a72d72025f38 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.427728 Oct 15 21:08:50.250075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3113/4227] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:50.258323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:50.259247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:50.272636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.273563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.274296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.289901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.292825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.293581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.559195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:50.570684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7b5e83c1-498b-49ea-8968-13722903558b tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.319202 Oct 15 21:08:50.575119 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3114/4228] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:50 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:50.583853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:50.585427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] [instance: 66b37c78-8e50-4dde-80af-ca3fcb2c67ae] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:50.594447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.595220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.595870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.596779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.597595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.598224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.621392 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] HTTP exception thrown: Instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae could not be found. Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Request method failure captured: Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae HTTP/1.1 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyFJArXSjMNpb9Sbsi6tmqbNIPYQQX5pjMjXGCvkqHF50BRxFP6ScAdDtp8-l0a0pYf12-hRo-WYBS6ebwCp1qvEf38IT0MFkGkUPgPfpvwy58ixiPH8pkDEb3Rb0M6EXeeDfLKEQ461q_KTLaNueedDvYtccODBEm5ztyLcbyA64kI Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 93dac014c11c45029f5b8569b7f509d0 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerPasswordTestJSON-881695632 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/93dac014c11c45029f5b8569b7f509d0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_93dac014c11c45029f5b8569b7f509d0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/93dac014c11c45029f5b8569b7f509d0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/93dac014c11c45029f5b8569b7f509d0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerPasswordTestJSON-881695632 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 93dac014c11c45029f5b8569b7f509d0 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerPasswordTestJSON-881695632 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerPasswordTestJSON-881695632-project Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 06e2c573568846d095c2a42a1df4e918 Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerPasswordTestJSON-881695632-project Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae could not be found. Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '66b37c78-8e50-4dde-80af-ca3fcb2c67ae'} Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae could not be found. Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae could not be found. Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:08:50.624857 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae could not be found. Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] Returning 404 to user: Instance 66b37c78-8e50-4dde-80af-ca3fcb2c67ae could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7687cbe-3e63-4c62-9f83-42b5684de4a0 tempest-ServerPasswordTestJSON-881695632 tempest-ServerPasswordTestJSON-881695632-project] 10.222.0.16 "GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae" status: 404 len: 111 microversion: 2.1 time: 0.050117 Oct 15 21:08:50.629379 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3115/4229] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:50 2022] GET /compute/v2.1/servers/66b37c78-8e50-4dde-80af-ca3fcb2c67ae => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:08:50.636713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:50.639137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:50.651687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.652194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.652622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.654569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.655031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.655463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.901080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:50.915830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-763f635b-ba95-414c-8a87-c7f073f2dfd1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.287122 Oct 15 21:08:50.916874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3116/4230] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:50 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:50.923561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:50.924531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:50.933153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.935132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.936043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:50.937119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:50.937826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:50.938481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:51.184660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:51.202266 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a02e941-eb9b-4245-88c1-6e24cd042321 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.283807 Oct 15 21:08:51.202873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3117/4231] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:50 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:51.599463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:51.601557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:51.615751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:51.617621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:51.618150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:51.620814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:51.622015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:51.623446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:51.874163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:51.892517 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30449e81-0ff3-4848-b6f3-3f0fa332853c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.296453 Oct 15 21:08:51.901383 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3118/4232] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:51 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:51.948051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:51.948702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:51.957980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:51.958757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:51.959448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:51.960658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:51.961306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:51.962029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:52.011075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:52.295106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:52.308386 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb5a70cf-c195-4e2b-8c9a-02db7925318b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.364915 Oct 15 21:08:52.308984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3119/4233] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:51 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:52.323635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:52.327582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:52.344562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:52.345499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:52.346201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:52.348943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:52.351737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:52.352398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:52.683640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:52.698129 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fbc0b453-102d-4fc6-9909-428251b6f5da tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.384120 Oct 15 21:08:52.699353 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3120/4234] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:52 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:52.919780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:52.920604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:52.931762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:52.936083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:52.936554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:52.941062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:52.941574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:52.943905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:53.305563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:53.327702 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6a599d3-8ee9-47f3-8ea8-746aaaa59076 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.410415 Oct 15 21:08:53.328973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3121/4235] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:52 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:08:53.335377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:53.338685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:53.363409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:53.364209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:53.365069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:53.365948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:53.366564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:53.367142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:53.499015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ec45dd6-4540-44da-a720-1d2dbea8f1af tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.167207 Oct 15 21:08:53.501152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3122/4236] 10.222.0.16 () {60 vars in 1343 bytes} [Sat Oct 15 21:08:53 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729/os-volume_attachments => generated 25 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:53.522488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:53.524105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:53.551767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:53.552838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:53.553538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:53.560034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:53.562310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:53.565981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:53.962310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:53.983665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb16dfd3-4c4f-4485-a165-e211042fdc52 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.476196 Oct 15 21:08:53.986468 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3123/4237] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:53 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:54.004161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:54.009430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:54.021872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:54.022763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:54.023630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:54.026934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:54.028223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:54.029931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:54.067034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:08:54.233067 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-618e5f27-b95e-42e5-8147-8258a7f4acd4 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 204 len: 0 microversion: 2.20 time: 0.240159 Oct 15 21:08:54.235367 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3124/4238] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:08:53 2022] DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:08:54.243256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:54.244175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:54.252907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:54.253741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:54.254330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:54.256335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:54.256962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:54.257665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:54.795249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:54.809855 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48a784fb-3ec3-493a-a124-4e9fe3c2ff6e tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.572718 Oct 15 21:08:54.810925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3125/4239] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:54 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:54.825274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:54.826179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:54.837465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:54.841757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:54.842471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:54.843994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:54.844642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:54.845878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:55.365257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:55.376433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:55.382569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:55.383785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b71a1880-e119-43c1-b4df-9e22614b51e8 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.571736 Oct 15 21:08:55.384388 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3126/4240] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:54 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:55.396161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:55.400885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:55.412697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:55.416321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:55.417011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:55.417989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:55.419283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:55.422454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:55.796310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:55.813627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c93a696-5491-4156-b922-6a023639ed23 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.422955 Oct 15 21:08:55.815001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3127/4241] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:55 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:55.827223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:55.828001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:55.848462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:55.854174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:55.857281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:55.866465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:55.867451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:55.869499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:56.835742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:56.850283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a0c6b8f-efa5-414a-808c-53fb54e59d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 1.032829 Oct 15 21:08:56.865653 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3128/4242] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:55 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 1049 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:56.870427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:56.881854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:56.909430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:56.917724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:56.919125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:56.920164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:56.920809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:56.921692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:57.234798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:57.243953 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-73bd56f1-f741-4902-8674-74c8fd12994d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.377193 Oct 15 21:08:57.255132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3129/4243] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:56 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:57.260524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:57.261211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:57.276816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:57.277570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:57.278728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:57.281106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:57.281636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:57.282085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:57.699348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:57.714331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:57.716348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:57.717605 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f660f665-9f5a-40db-9ff9-19373b1b70c1 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1722 microversion: 2.20 time: 0.461209 Oct 15 21:08:57.719026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3130/4244] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:57 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1722 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:57.737544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:57.738451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:57.753362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:57.754062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:57.754823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:57.755950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:57.756581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:57.757190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:58.053542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:58.071286 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2eb45189-01ae-4197-979a-706717fffb8c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.350456 Oct 15 21:08:58.072903 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3131/4245] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:57 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:58.082480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:58.084429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:58.094092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:58.095473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:58.096450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:58.099660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:58.100494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:58.101616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:58.533103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:58.544054 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fc3a650-8baa-4c20-a625-63dec9b52de5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1684 microversion: 2.1 time: 0.468453 Oct 15 21:08:58.553162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3132/4246] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:58 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1684 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:58.553162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:58.553162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:58.558451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:58.559502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:58.560057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:58.561762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:58.562256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:58.562704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:58.892227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:58.914327 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3704193-009a-4b17-8676-2774c4e36ce7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.367836 Oct 15 21:08:58.915084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3133/4247] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:58 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:58.932625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2eecd069-6a9b-4fb2-bddb-a8774cf5b6fb req-2c46609c-ae40-4903-803d-f6feb8c11da6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "536ca0a5-93a3-4c03-b4d5-ab535cbc1729", "name": "network-vif-deleted", "tag": "5e20c6d7-6448-453e-8e67-dab298ab2ffe"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:08:58.963560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2eecd069-6a9b-4fb2-bddb-a8774cf5b6fb req-2c46609c-ae40-4903-803d-f6feb8c11da6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:58.964223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2eecd069-6a9b-4fb2-bddb-a8774cf5b6fb req-2c46609c-ae40-4903-803d-f6feb8c11da6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:58.965206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2eecd069-6a9b-4fb2-bddb-a8774cf5b6fb req-2c46609c-ae40-4903-803d-f6feb8c11da6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:58.998210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2eecd069-6a9b-4fb2-bddb-a8774cf5b6fb req-2c46609c-ae40-4903-803d-f6feb8c11da6 service nova] Creating event network-vif-deleted:5e20c6d7-6448-453e-8e67-dab298ab2ffe for instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 on nch2-master-1 Oct 15 21:08:59.005203 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2eecd069-6a9b-4fb2-bddb-a8774cf5b6fb req-2c46609c-ae40-4903-803d-f6feb8c11da6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087622 Oct 15 21:08:59.007300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3134/4248] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:08:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:08:59.014886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:59.016048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:59.028488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.029319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.029986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:59.030994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.031847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.032558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:59.052394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:08:59.097318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.098016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.098285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:59.098767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lazy-loading 'fault' on Instance uuid 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:08:59.157072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:59.181970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:59.205539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fb8dc371c65f4bbb92d559c7ed78964c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c04b2976577d42799a800d21545d1969', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:59.213276 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a99cc53-a136-4d56-bdaa-a4b1d562f765 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 200 len: 1370 microversion: 2.20 time: 0.204510 Oct 15 21:08:59.217931 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3135/4249] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:08:59 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 1370 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:08:59.253318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:59.255181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:59.289370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.291445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.294723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:59.301575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.302723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.304727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:59.749052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:08:59.757917 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e3df397f-65a2-4a35-8467-42361a620d30 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.535817 Oct 15 21:08:59.768120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3136/4250] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:59 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:08:59.773496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:08:59.774231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:08:59.791065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.791460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.791937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:08:59.792701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:08:59.793834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:08:59.794429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.114182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:00.138076 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5fdebd7-95ff-401f-b870-a73c65007528 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.368651 Oct 15 21:09:00.139261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3137/4251] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:08:59 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:00.144089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:00.144901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:00.160818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.161641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.162252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.163517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.164176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.164816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.383258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:00.396574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e16e71c-81a8-463d-91b5-7f4dfd4fd553 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.255979 Oct 15 21:09:00.397427 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3138/4252] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:00 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:00.402174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:00.403341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:00.409360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.410090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.410727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.411638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.412219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.412869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.428131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] HTTP exception thrown: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Request method failure captured: Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 HTTP/1.1 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9tfj-IfeDlmhCVopaxKqpvSD6hcfJy661w2SylEc8Y4h2qBwVnqJG8EPnKb0eyLrD36V761YW1_ot578qNdTTF4wGF86vEaSHTq2fmlx2SGsXOHDZnkb0j7gd99vnTAGZaImphiObDk2NPNtACw5rhiHELEFOKd6GSfPiHcqDCZvU Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.20 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c04b2976577d42799a800d21545d1969"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c04b2976577d42799a800d21545d1969"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fb8dc371c65f4bbb92d559c7ed78964c Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '536ca0a5-93a3-4c03-b4d5-ab535cbc1729'} Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:00.430221 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Returning 404 to user: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:00.434356 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fc8c733f-26f5-4e3d-8466-eae098440dde tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 404 len: 111 microversion: 2.20 time: 0.033175 Oct 15 21:09:00.436491 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3139/4253] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:09:00 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:09:00.442183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:00.443094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:00.450221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.450934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.451530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.452474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.452937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.453423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.606649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:00.614748 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cbe9eeee-90b4-4000-b4e2-5fb98c3181f9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.176515 Oct 15 21:09:00.615934 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3140/4254] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:00 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:00.620855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f77afe9-8690-452a-be47-d27c4134e932 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '.version_select at 0x7fe74fc17d30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:00.651418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f77afe9-8690-452a-be47-d27c4134e932 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-593384543" status: 204 len: 0 microversion: 2.20 time: 0.034219 Oct 15 21:09:00.654194 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3141/4255] 10.222.0.16 () {60 vars in 1259 bytes} [Sat Oct 15 21:09:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-593384543 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:09:00.796030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:00.796960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:00.803855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.804632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.805385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:00.806342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:00.807079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:00.807731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:01.033622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:01.046226 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2ed0da5-032a-415f-88fa-2c100678e7eb tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.255003 Oct 15 21:09:01.047188 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3142/4256] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:00 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:09:01.429158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:01.432242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:01.446261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:01.447188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:01.447933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:01.449132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:01.451316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:01.452273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:01.750943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:01.759447 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2aa250da-7d3f-41b2-a3ae-c8da6a51bf3d tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.341073 Oct 15 21:09:01.763141 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3143/4257] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:01 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:02.070547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:02.071644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:02.088192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:02.089539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:02.090433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:02.094383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:02.097527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:02.098110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:02.335693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:02.349447 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca7b79c6-b831-4bb5-b8f5-c64a73cce243 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.283438 Oct 15 21:09:02.368835 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3144/4258] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:02 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:09:02.795645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:02.796349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:02.816184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:02.816727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:02.817316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:02.817955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:02.823350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:02.826410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:03.729318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:03.745702 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f8fb2f79-6f3a-441e-83ea-0641fea89986 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.963696 Oct 15 21:09:03.746864 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3145/4259] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:02 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 966 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:03.769561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:03.770262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:03.782967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:03.783824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:03.787478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:03.788510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:03.790435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:03.790945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:03.813928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] HTTP exception thrown: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Request method failure captured: Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 HTTP/1.1 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9tfj-IfeDlmhCVopaxKqpvSD6hcfJy661w2SylEc8Y4h2qBwVnqJG8EPnKb0eyLrD36V761YW1_ot578qNdTTF4wGF86vEaSHTq2fmlx2SGsXOHDZnkb0j7gd99vnTAGZaImphiObDk2NPNtACw5rhiHELEFOKd6GSfPiHcqDCZvU Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.20 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c04b2976577d42799a800d21545d1969"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c04b2976577d42799a800d21545d1969"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fb8dc371c65f4bbb92d559c7ed78964c Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '536ca0a5-93a3-4c03-b4d5-ab535cbc1729'} Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:03.816285 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Returning 404 to user: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:03.822025 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d1bc359-912f-4082-81ef-c7e1856613ab tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 404 len: 111 microversion: 2.20 time: 0.070953 Oct 15 21:09:03.823723 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3146/4260] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:09:03 2022] DELETE /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:09:03.829483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:03.830513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:03.837417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:03.838192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:03.838632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:03.839477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:03.840055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:03.840722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.170453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:04.180231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a87ab691-b293-40b9-9877-41901e9f40ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.354377 Oct 15 21:09:04.180890 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3147/4261] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:03 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:04.186590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:04.187328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:04.193607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.194369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.195040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.196517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.197242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.197908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.555551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:04.566818 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5e2a166-22ca-4db3-b588-0da0c0a4b19c tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.384614 Oct 15 21:09:04.570045 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3148/4262] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:04 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:04.577306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d6d9a994-0119-4ebe-b070-0c4899902747 req-3360d345-e337-4c2a-b7d1-e6065d24cf4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-unplugged", "status": "completed", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:04.595482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d6d9a994-0119-4ebe-b070-0c4899902747 req-3360d345-e337-4c2a-b7d1-e6065d24cf4d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.596281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d6d9a994-0119-4ebe-b070-0c4899902747 req-3360d345-e337-4c2a-b7d1-e6065d24cf4d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.596917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d6d9a994-0119-4ebe-b070-0c4899902747 req-3360d345-e337-4c2a-b7d1-e6065d24cf4d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.620151 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d6d9a994-0119-4ebe-b070-0c4899902747 req-3360d345-e337-4c2a-b7d1-e6065d24cf4d service nova] Creating event network-vif-unplugged:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:09:04.627745 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d6d9a994-0119-4ebe-b070-0c4899902747 req-3360d345-e337-4c2a-b7d1-e6065d24cf4d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.055925 Oct 15 21:09:04.628656 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3149/4263] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:04 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:04.642378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:04.643199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 536ca0a5-93a3-4c03-b4d5-ab535cbc1729] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:04.655272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.656008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.656927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.658057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.658792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.659431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.675591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] HTTP exception thrown: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Request method failure captured: Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 HTTP/1.1 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9tfj-IfeDlmhCVopaxKqpvSD6hcfJy661w2SylEc8Y4h2qBwVnqJG8EPnKb0eyLrD36V761YW1_ot578qNdTTF4wGF86vEaSHTq2fmlx2SGsXOHDZnkb0j7gd99vnTAGZaImphiObDk2NPNtACw5rhiHELEFOKd6GSfPiHcqDCZvU Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.20 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c04b2976577d42799a800d21545d1969"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c04b2976577d42799a800d21545d1969"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fb8dc371c65f4bbb92d559c7ed78964c Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '536ca0a5-93a3-4c03-b4d5-ab535cbc1729'} Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:04.678611 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:04.684187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Returning 404 to user: Instance 536ca0a5-93a3-4c03-b4d5-ab535cbc1729 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:04.686719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba621c36-9795-4a25-bcd1-9e0ec370d527 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729" status: 404 len: 111 microversion: 2.20 time: 0.053218 Oct 15 21:09:04.690496 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3150/4264] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:09:04 2022] GET /compute/v2.1/servers/536ca0a5-93a3-4c03-b4d5-ab535cbc1729 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:09:04.692863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:04.693455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:04.705344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.705840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.706475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:04.707339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:04.707964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:04.708542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:05.080663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:05.090195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0dffbf9-7d94-464d-bee6-c8c428d06f9c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.401176 Oct 15 21:09:05.093404 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3151/4265] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:04 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:05.099098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:05.100190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:05.117750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:05.123576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:05.124036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:05.125404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:05.126249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:05.126964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:05.426493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:05.442420 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1d26ed75-c29b-4acf-8633-4bc6d8088b06 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.347392 Oct 15 21:09:05.443060 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3152/4266] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:05 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:05.467434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:05.469195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:05.477664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:05.479717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:05.480932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:05.482302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:05.482961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:05.483672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:05.831037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:05.842010 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5b61a6b-117c-4ccf-b4df-ad89397e63ec tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.381112 Oct 15 21:09:05.853067 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3153/4267] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:05 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:05.853067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:05.853067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:05.858335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:05.859042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:05.859640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:05.861870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:05.862511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:05.863099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.124408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:06.132494 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3aad58ab-de91-4c06-9926-03c6ce537bc5 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.288397 Oct 15 21:09:06.148162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3154/4268] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:05 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:06.157203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:06.160692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:06.169071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.169924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.170558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.171437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.172056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.172779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.179060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:06.294746 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-64da9976-293f-4a78-b096-070b172df66e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.144888 Oct 15 21:09:06.295830 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3155/4269] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:09:06 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 147 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:09:06.301517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:06.302350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:06.311663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.312418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.314588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.315868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.316860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.317789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.338696 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] HTTP exception thrown: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Request method failure captured: Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 HTTP/1.1 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9tfj-IfeDlmhCVopaxKqpvSD6hcfJy661w2SylEc8Y4h2qBwVnqJG8EPnKb0eyLrD36V761YW1_ot578qNdTTF4wGF86vEaSHTq2fmlx2SGsXOHDZnkb0j7gd99vnTAGZaImphiObDk2NPNtACw5rhiHELEFOKd6GSfPiHcqDCZvU Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.20 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c04b2976577d42799a800d21545d1969"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c04b2976577d42799a800d21545d1969"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fb8dc371c65f4bbb92d559c7ed78964c Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8dc308ad-212c-4df0-baa6-f446dac1d979'} Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:06.341687 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Returning 404 to user: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:06.346793 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-38720ce1-6f65-4deb-a412-63efb1adf4d5 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 404 len: 111 microversion: 2.20 time: 0.046706 Oct 15 21:09:06.349815 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3156/4270] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:09:06 2022] DELETE /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:09:06.354433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:06.356025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:06.363424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.364197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.365221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.366164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.368139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.368959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.744530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:06.758007 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f3a97d5-c191-4a74-bbb3-06c5e23ae36c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1673 microversion: 2.1 time: 0.407807 Oct 15 21:09:06.760896 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3157/4271] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:06 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1673 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:06.777961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:06.778793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] [instance: 8dc308ad-212c-4df0-baa6-f446dac1d979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:06.790940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:06.795445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.797694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.798359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.800002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.800776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.801660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.850407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] HTTP exception thrown: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Request method failure captured: Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 HTTP/1.1 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSx9tfj-IfeDlmhCVopaxKqpvSD6hcfJy661w2SylEc8Y4h2qBwVnqJG8EPnKb0eyLrD36V761YW1_ot578qNdTTF4wGF86vEaSHTq2fmlx2SGsXOHDZnkb0j7gd99vnTAGZaImphiObDk2NPNtACw5rhiHELEFOKd6GSfPiHcqDCZvU Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.20 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c04b2976577d42799a800d21545d1969"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c04b2976577d42799a800d21545d1969"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c04b2976577d42799a800d21545d1969"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: c04b2976577d42799a800d21545d1969 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-208196953 Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: fb8dc371c65f4bbb92d559c7ed78964c Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeShelveTestJSON-208196953-project Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8dc308ad-212c-4df0-baa6-f446dac1d979'} Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.856325 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] Returning 404 to user: Instance 8dc308ad-212c-4df0-baa6-f446dac1d979 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:06.860717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b534f9e-de11-4392-9197-2c312ace0e59 tempest-AttachVolumeShelveTestJSON-208196953 tempest-AttachVolumeShelveTestJSON-208196953-project] 10.222.0.16 "GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979" status: 404 len: 111 microversion: 2.20 time: 0.098196 Oct 15 21:09:06.871729 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3158/4272] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:09:06 2022] GET /compute/v2.1/servers/8dc308ad-212c-4df0-baa6-f446dac1d979 => generated 111 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:09:06.878127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:06.878756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:06.892150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.893123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.893552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:06.894291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:06.894752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:06.897286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:07.344720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:07.355177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecea54eb-9a44-43d5-9d6c-da5bd026ce0f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.482246 Oct 15 21:09:07.359377 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3159/4273] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:06 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:07.366943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:07.368680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:07.388721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:07.393402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:07.394705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:07.395680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:07.398627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:07.399307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:07.890062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:07.899861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0cbff03f-6dbf-42c9-bade-9302ed728df8 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.538935 Oct 15 21:09:07.911630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3160/4274] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:07 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:07.917635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:07.918999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:07.927238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:07.928044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:07.928634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:07.930869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:07.931929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:07.932605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:08.493493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:08.507741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5cc0340-f912-4a2a-b1cd-982a489ca2a8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1673 microversion: 2.1 time: 0.596780 Oct 15 21:09:08.523336 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3161/4275] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1673 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:08.540799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:08.544461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:08.562270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:08.564098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:08.564559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:08.565325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:08.565851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:08.566292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:08.981865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:08.997138 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c88da79-40fd-40f3-8921-603079a59cd6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1516 microversion: 2.1 time: 0.470617 Oct 15 21:09:08.999470 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3162/4276] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:08 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1516 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:09.007987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:09.008956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:09.030532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:09.031716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:09.033490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:09.036290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:09.037100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:09.039055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:09.382493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:09.393932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5fdf0835-6b54-477b-b796-f9152759454d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.390798 Oct 15 21:09:09.405507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3163/4277] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:08 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:09.759591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-5570935", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:09.849866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] validate_networks() for [('855fc75b-c131-45d1-adfc-303e1e3ad3a6', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:09:11.070012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:09:11.070593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:09:11.070920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:09:11.071337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:09:11.071775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:09:11.087402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.087913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.088363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.089133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.091235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.094590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.121789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for project e2b1936403344a4994393f2b098327a0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:09:11.130372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for user c48346e56c47481cab7f248b67369a27 and project e2b1936403344a4994393f2b098327a0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:09:11.150992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:09:11.155980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:09:11.242844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6603b3ba-9728-46df-a7a3-a0129c5bd2a4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.835771 Oct 15 21:09:11.247551 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3164/4278] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:09:09 2022] POST /compute/v2.1/servers => generated 383 bytes in 1841 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:09:11.264046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:11.264824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:11.284340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.284790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.285446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.286297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.288395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.289647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.613756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:11.622440 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ada836f-c2c8-450a-976a-9e665f2d4240 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.373597 Oct 15 21:09:11.623055 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3165/4279] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:11 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:11.632881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:11.639269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:11.656160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.656160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.656484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.657336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.657942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.658245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.955896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:11.969131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4fdcca20-0582-4415-8e9d-0ffacbbc14fc tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1534 microversion: 2.1 time: 0.344823 Oct 15 21:09:11.970023 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3166/4280] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:11 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1534 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:11.976239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:11.978066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:11.990549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.991389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.992020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:11.993128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:11.993885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:11.995231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.267164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:12.287006 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d32b7ea-5cd7-48df-9289-d173685d252a tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.312567 Oct 15 21:09:12.287645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3167/4281] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:11 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:12.300755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:12.301281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:12.325934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:12.326461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:12.328155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.331925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:12.333976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:12.334525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.400010 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8bf12adb-317c-4032-a425-6fb42126c267 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/ips" status: 200 len: 144 microversion: 2.1 time: 0.111254 Oct 15 21:09:12.404976 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3168/4282] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:09:12 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/ips => generated 144 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:12.443047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:12.445316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:12.468343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:12.469557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:12.471971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.474823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:12.477396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:12.479137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.660910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:12.680285 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-06d30f82-f831-41ab-9d12-238c737cad21 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1223 microversion: 2.1 time: 0.273142 Oct 15 21:09:12.687592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3169/4283] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:12 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1223 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:12.694813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 3}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:12.723303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:12.766776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:12.767285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:12.767723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.776457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:12.779816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:12.782720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:12.955772 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c076315-338d-4db8-be44-622302af539f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 200 len: 324 microversion: 2.1 time: 0.266845 Oct 15 21:09:12.969384 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3170/4284] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:09:12 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 324 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:12.969384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:12.969384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:13.010145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:13.010639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:13.011081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:13.012348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:13.020484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:13.022920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:13.450533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:13.474000 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ebc076d2-43a9-4d42-b436-6f7ebe7310b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.510272 Oct 15 21:09:13.478781 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3171/4285] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:12 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:13.500461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:13.502534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:13.518760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:13.519394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:13.519939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:13.521410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:13.522070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:13.522625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:13.773206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:13.791755 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b27d06b1-e59f-4232-baf7-1c62865ab17a tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1522 microversion: 2.1 time: 0.310986 Oct 15 21:09:13.792812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3172/4286] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:13 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1522 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:13.800858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-302725549", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:13.916763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] validate_networks() for [('855fc75b-c131-45d1-adfc-303e1e3ad3a6', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:09:15.415360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:09:15.416035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:09:15.416469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:09:15.417070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:09:15.418341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:09:15.440220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:15.440906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:15.441544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:15.442531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:15.443226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:15.443824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:15.513661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for project e2b1936403344a4994393f2b098327a0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:09:15.528414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for user c48346e56c47481cab7f248b67369a27 and project e2b1936403344a4994393f2b098327a0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:09:15.581141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:09:15.583848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:09:15.763719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-131ed584-b7f5-45b2-9b6e-10d302b8baab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.968626 Oct 15 21:09:15.766371 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3173/4287] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:09:13 2022] POST /compute/v2.1/servers => generated 383 bytes in 1972 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:09:15.786424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:15.787313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:15.806639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:15.807232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:15.807728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:15.809219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:15.810335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:15.813406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:15.825354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:16.295852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:16.313916 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3f80fc6-a1dc-4263-b67c-5f5227fdd9b6 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1483 microversion: 2.1 time: 0.544638 Oct 15 21:09:16.314773 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3174/4288] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:15 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1483 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:16.326022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:16.328799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:16.341909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:16.345192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:16.345897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:16.346862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:16.349191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:16.349891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:16.957069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:16.974202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-948ec029-758a-41ba-8e5c-11b6768260be tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.657202 Oct 15 21:09:16.976135 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3175/4289] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:16 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:16.993378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "23ae0121-69fa-41e9-b9e5-f82bb6991d5d"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:17.005703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:17.013289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:17.014125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:17.014781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:17.017321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:17.018858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:17.019562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:17.551031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2944cf3c-b56b-4a0c-a675-c15aacfcae60 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.571667 Oct 15 21:09:17.552591 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3176/4290] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:09:16 2022] POST /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments => generated 194 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:17.575244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:17.577434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:17.587558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:17.588229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:17.589153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:17.591018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:17.604721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:17.605907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.228332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:18.238063 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-deb65644-b8dd-4d96-8fed-2a8c8adcc88f tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1522 microversion: 2.1 time: 0.683683 Oct 15 21:09:18.239212 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3177/4291] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:17 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1522 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:18.253071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:18.254823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:18.266727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.267664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.268227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.269822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.270301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.271127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.589212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:18.601800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f54e3ea2-7d66-4a01-8fcc-c48cd024743e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1279 microversion: 2.1 time: 0.361009 Oct 15 21:09:18.603685 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3178/4292] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:18 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1279 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:18.612544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-90c281f5-ad8d-43a1-b186-bff846ea9752 req-25476030-1b5a-460f-ad91-1cdbcc71f60f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1f541f-e2cd-4527-b301-15fc8d6b34e9", "name": "network-vif-unplugged", "status": "completed", "tag": "cdfb68d1-a92c-47c1-b12e-066c2ab0987e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:18.643606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-90c281f5-ad8d-43a1-b186-bff846ea9752 req-25476030-1b5a-460f-ad91-1cdbcc71f60f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.647007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-90c281f5-ad8d-43a1-b186-bff846ea9752 req-25476030-1b5a-460f-ad91-1cdbcc71f60f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.649749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-90c281f5-ad8d-43a1-b186-bff846ea9752 req-25476030-1b5a-460f-ad91-1cdbcc71f60f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.680798 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-90c281f5-ad8d-43a1-b186-bff846ea9752 req-25476030-1b5a-460f-ad91-1cdbcc71f60f service nova] Creating event network-vif-unplugged:cdfb68d1-a92c-47c1-b12e-066c2ab0987e for instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 on nch1-master-1 Oct 15 21:09:18.715052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-90c281f5-ad8d-43a1-b186-bff846ea9752 req-25476030-1b5a-460f-ad91-1cdbcc71f60f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.107877 Oct 15 21:09:18.715810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3179/4293] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:18 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:18.737606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:18.738956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:18.771214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.772127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.772794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.775228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.779884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.780623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.916808 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78661422-ab1e-4e7b-812c-9a586edfb854 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "POST /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5/action" status: 202 len: 0 microversion: 2.1 time: 0.193636 Oct 15 21:09:18.919120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3180/4294] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:09:18 2022] POST /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5/action => generated 0 bytes in 203 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:09:18.926514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:18.927589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:18.937654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.938772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.939459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:18.940721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:18.941659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:18.942402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:19.477335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:19.486126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1e2c2c5b-dbd4-43a6-bd3d-c134ebee2587 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.565504 Oct 15 21:09:19.487225 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3181/4295] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:18 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:19.494141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9bdb7263-62b6-4d5f-8f2e-1795721cbe9f req-c73ea1a8-a3e4-4d9d-bd0a-50fae42c2fa3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4ab84bd-5610-44e0-86cf-2877eac80927", "tag": "bdfe27f9-115c-4ed4-88ed-0bf0adaabfac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:19.512624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9bdb7263-62b6-4d5f-8f2e-1795721cbe9f req-c73ea1a8-a3e4-4d9d-bd0a-50fae42c2fa3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:19.513167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9bdb7263-62b6-4d5f-8f2e-1795721cbe9f req-c73ea1a8-a3e4-4d9d-bd0a-50fae42c2fa3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:19.513566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9bdb7263-62b6-4d5f-8f2e-1795721cbe9f req-c73ea1a8-a3e4-4d9d-bd0a-50fae42c2fa3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:19.528351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9bdb7263-62b6-4d5f-8f2e-1795721cbe9f req-c73ea1a8-a3e4-4d9d-bd0a-50fae42c2fa3 service nova] Creating event network-changed:bdfe27f9-115c-4ed4-88ed-0bf0adaabfac for instance b4ab84bd-5610-44e0-86cf-2877eac80927 on nch1-master-1 Oct 15 21:09:19.537742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9bdb7263-62b6-4d5f-8f2e-1795721cbe9f req-c73ea1a8-a3e4-4d9d-bd0a-50fae42c2fa3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046943 Oct 15 21:09:19.538599 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3182/4296] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:19.547656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:19.550340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:19.563899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:19.564820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:19.565451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:19.566350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:19.566966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:19.567684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:19.961898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:19.971250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d616e42c-c4d0-4286-b4f9-c1b625368ac0 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1491 microversion: 2.1 time: 0.430591 Oct 15 21:09:19.972234 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3183/4297] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:19 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1491 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:19.977332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:19.978470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:19.986715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:19.987569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:19.988254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:19.989554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:19.990237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:19.990905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:20.583370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:20.591823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54ed9eaf-659c-48d0-a1d6-415c6b1d8cc5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1522 microversion: 2.1 time: 0.618095 Oct 15 21:09:20.592424 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3184/4298] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:19 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1522 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:20.613763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:20.614659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:20.621656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:20.622357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:20.622998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:20.623859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:20.624479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:20.625089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:20.999677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:21.003912 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54cfb59f-3d90-477a-a534-6de5bfd28d94 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.393835 Oct 15 21:09:21.005822 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3185/4299] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:20 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:21.032165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:21.034766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:21.046636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:21.047432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:21.048869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:21.051153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:21.051807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:21.052485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:21.405893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:21.426626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a4c6dede-9fad-4155-bd2a-9a9141562110 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1321 microversion: 2.1 time: 0.405113 Oct 15 21:09:21.427963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3186/4300] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:21 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1321 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:21.433699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b4ca61c3-d719-4769-8ca9-e39ca3310df7 req-1a2d8c32-cd64-4710-96eb-3af8a5639306 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1f541f-e2cd-4527-b301-15fc8d6b34e9", "name": "network-vif-plugged", "status": "completed", "tag": "cdfb68d1-a92c-47c1-b12e-066c2ab0987e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:21.454881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b4ca61c3-d719-4769-8ca9-e39ca3310df7 req-1a2d8c32-cd64-4710-96eb-3af8a5639306 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:21.455655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b4ca61c3-d719-4769-8ca9-e39ca3310df7 req-1a2d8c32-cd64-4710-96eb-3af8a5639306 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:21.456714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b4ca61c3-d719-4769-8ca9-e39ca3310df7 req-1a2d8c32-cd64-4710-96eb-3af8a5639306 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:21.474801 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b4ca61c3-d719-4769-8ca9-e39ca3310df7 req-1a2d8c32-cd64-4710-96eb-3af8a5639306 service nova] Creating event network-vif-plugged:cdfb68d1-a92c-47c1-b12e-066c2ab0987e for instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 on nch1-master-1 Oct 15 21:09:21.482060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b4ca61c3-d719-4769-8ca9-e39ca3310df7 req-1a2d8c32-cd64-4710-96eb-3af8a5639306 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052629 Oct 15 21:09:21.490538 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3187/4301] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:21.496512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:21.498392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:21.529388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:21.530147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:21.531086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:21.532126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:21.532755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:21.533396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:22.028821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:22.046429 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d7b4c35-9ceb-4558-8d74-d68ddd6bbef9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1491 microversion: 2.1 time: 0.554373 Oct 15 21:09:22.047346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3188/4302] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:21 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1491 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:22.052733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1416370788", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:22.146648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] validate_networks() for [('c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:09:22.996820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:09:22.997410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:09:22.997901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:09:22.998414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:09:22.998840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:09:23.009913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.010686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.012040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.012903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.013553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.014150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.032698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:09:23.038731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for user 1aadd87b97c247cb95321664ba0c32a9 and project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:09:23.049962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:23.053945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:09:23.056229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:09:23.148497 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb44bc7a-1bde-493f-abf2-3b2824b40096 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.097089 Oct 15 21:09:23.153336 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3189/4303] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:09:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 1105 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:09:23.158583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:23.168260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:23.188123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.210375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.211063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.213101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.218428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.219339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.595671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:23.606046 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d9e6813-16bd-4b53-9ace-2d8b52271234 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1522 microversion: 2.1 time: 0.451589 Oct 15 21:09:23.613868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3190/4304] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:23 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1522 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:23.626542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:23.636409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:23.649460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.650479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.651303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.652537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.653300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.654335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.681951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:23.711086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8509d3cb-fdf3-4c89-842a-db7a4f8a7d07 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.094465 Oct 15 21:09:23.714586 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3191/4305] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:09:23 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments => generated 197 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:23.724096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:23.725934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:23.737894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.738654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.739552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.740630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.741250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.741991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.947237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:23.957024 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e3d0c74-3d2f-40f8-b72f-55e3b188bd8f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1321 microversion: 2.1 time: 0.241239 Oct 15 21:09:23.957751 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3192/4306] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:23 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1321 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:23.963483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1759ef78-d9d0-45b9-ba59-229b94e49d33 req-1267fd02-7573-47a2-a41b-ed4b5d86555e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "76691e91-f327-49c4-88ce-d6932390510f", "tag": "b51cdbdc-d3ee-4cf7-85b1-07fec0d15ffc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:23.975572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1759ef78-d9d0-45b9-ba59-229b94e49d33 req-1267fd02-7573-47a2-a41b-ed4b5d86555e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:23.976254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1759ef78-d9d0-45b9-ba59-229b94e49d33 req-1267fd02-7573-47a2-a41b-ed4b5d86555e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:23.976726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1759ef78-d9d0-45b9-ba59-229b94e49d33 req-1267fd02-7573-47a2-a41b-ed4b5d86555e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:23.993524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1759ef78-d9d0-45b9-ba59-229b94e49d33 req-1267fd02-7573-47a2-a41b-ed4b5d86555e service nova] Creating event network-changed:b51cdbdc-d3ee-4cf7-85b1-07fec0d15ffc for instance 76691e91-f327-49c4-88ce-d6932390510f on nch2-master-1 Oct 15 21:09:23.999942 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1759ef78-d9d0-45b9-ba59-229b94e49d33 req-1267fd02-7573-47a2-a41b-ed4b5d86555e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040552 Oct 15 21:09:24.000808 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3193/4307] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:24.010534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:24.012532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:24.021331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.022008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.022805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.023824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.024586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.025352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.262531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:24.275405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72ee76f6-cbbf-4380-9ee4-c8f12ea5129e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1497 microversion: 2.1 time: 0.272566 Oct 15 21:09:24.289708 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3194/4308] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:24 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1497 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:24.300884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:24.306686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:24.315083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.315780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.317073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.318994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.328587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.333028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.438690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:24.449808 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9238d064-ec66-412e-9e95-8147b14939fb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1267 microversion: 2.1 time: 0.157809 Oct 15 21:09:24.457862 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3195/4309] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:24 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:24.469703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:24.471101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:24.477923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.478770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.479420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.481795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.482466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.483090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.523986 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f4e09c5-ca19-4f8a-8d03-26d4040829d9 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/23ae0121-69fa-41e9-b9e5-f82bb6991d5d" status: 200 len: 194 microversion: 2.1 time: 0.063437 Oct 15 21:09:24.525348 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3196/4310] 10.222.0.16 () {58 vars in 1413 bytes} [Sat Oct 15 21:09:24 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/23ae0121-69fa-41e9-b9e5-f82bb6991d5d => generated 194 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:24.531967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:24.532750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:24.541508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.542770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.543438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.544467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.545072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.545621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.577471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:09:24.726863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac0a4222-aca9-4fdd-88ed-5728c0bb9a50 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "DELETE /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 204 len: 0 microversion: 2.1 time: 0.199175 Oct 15 21:09:24.727955 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3197/4311] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:24 2022] DELETE /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:24.735589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:24.736837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:24.747027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.747914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.749065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:24.750120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:24.750725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:24.751525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:25.239623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:25.251208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7519d7ee-aa3d-46fc-bd12-94affd12c5e2 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1507 microversion: 2.1 time: 0.521621 Oct 15 21:09:25.273696 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3198/4312] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:24 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1507 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:25.279115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:25.279582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:25.303230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:25.303887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:25.304478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:25.305310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:25.305786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:25.306227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:25.895833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:25.907195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0be4cdc1-8061-4bd1-aec6-0d3fb0751aad tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.632300 Oct 15 21:09:25.909440 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3199/4313] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:25 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:25.925347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:25.928480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:25.942727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:25.943506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:25.944135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:25.945157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:25.950103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:25.951376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.220299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:26.230381 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26656afa-0ec1-4be9-bad9-0570d9d3ee1c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1321 microversion: 2.1 time: 0.316382 Oct 15 21:09:26.231110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3200/4314] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:25 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1321 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:26.245663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:26.245663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:26.245663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.245663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.245663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.245663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.246217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.246443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.485952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:26.515188 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d83a2d7c-f61e-4452-9bff-c3d31f2f1cbe tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 200 len: 1507 microversion: 2.1 time: 0.282918 Oct 15 21:09:26.518450 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3201/4315] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:26 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 1507 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:26.523451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:26.524065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:26.531634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.532607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.533242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.537518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.538028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.538676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.816553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:26.828185 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dab2ceaf-5f39-4516-8a9d-8da5f8a4c868 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1315 microversion: 2.1 time: 0.308604 Oct 15 21:09:26.830067 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3202/4316] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:26 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1315 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:26.835967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0ea9c6cb-1ed0-41b8-a475-65daf538a336 req-134ae046-593f-4929-baf7-1866f54b5b58 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b4ab84bd-5610-44e0-86cf-2877eac80927", "name": "network-vif-plugged", "status": "completed", "tag": "bdfe27f9-115c-4ed4-88ed-0bf0adaabfac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:26.866101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ea9c6cb-1ed0-41b8-a475-65daf538a336 req-134ae046-593f-4929-baf7-1866f54b5b58 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.866603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ea9c6cb-1ed0-41b8-a475-65daf538a336 req-134ae046-593f-4929-baf7-1866f54b5b58 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.867043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ea9c6cb-1ed0-41b8-a475-65daf538a336 req-134ae046-593f-4929-baf7-1866f54b5b58 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.895774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0ea9c6cb-1ed0-41b8-a475-65daf538a336 req-134ae046-593f-4929-baf7-1866f54b5b58 service nova] Creating event network-vif-plugged:bdfe27f9-115c-4ed4-88ed-0bf0adaabfac for instance b4ab84bd-5610-44e0-86cf-2877eac80927 on nch1-master-1 Oct 15 21:09:26.915080 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0ea9c6cb-1ed0-41b8-a475-65daf538a336 req-134ae046-593f-4929-baf7-1866f54b5b58 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081812 Oct 15 21:09:26.916207 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3203/4317] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:26.923797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "775f9211-fc12-4110-8490-24df7940a524"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:26.939517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:26.951228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.951997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.952610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:26.954174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:26.955239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:26.955913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:27.343850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c532223-0518-4ece-a80a-6e05ff4a6b53 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "POST /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.425198 Oct 15 21:09:27.350884 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3204/4318] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:09:26 2022] POST /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments => generated 194 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:27.358663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:27.359630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:27.372127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:27.372705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:27.373399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:27.374289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:27.375073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:27.375824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:27.441928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:09:27.488301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:27.649103 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4eef6148-1b48-4306-9158-9779d08010fd tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 204 len: 0 microversion: 2.1 time: 0.296292 Oct 15 21:09:27.649714 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3205/4319] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:27 2022] DELETE /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:27.666530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:27.670034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:27.679267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:27.680502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:27.683836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:27.685286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:27.686228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:27.686906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:27.953057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:27.966095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18985e24-2c21-42ed-82a9-32c59dc49e77 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.312336 Oct 15 21:09:27.976204 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3206/4320] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:27 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:27.983482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:27.984327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:27.992159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:27.993238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:27.994025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:27.994913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:27.996222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:27.997087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:28.182779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:28.217092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-507de522-d5b6-4ff9-939a-a2799a3b0732 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1321 microversion: 2.1 time: 0.239520 Oct 15 21:09:28.221636 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3207/4321] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:27 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1321 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:28.233750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8a0c6fa7-d29e-49b7-a567-71d1969d661c req-aef04ace-72af-4b68-adf9-28bdcddfc06c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6420b15-803c-41f2-b430-b770b2f46e64", "tag": "fdaa8af5-5d45-4b21-be18-4bc6b852d171"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:28.250673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8a0c6fa7-d29e-49b7-a567-71d1969d661c req-aef04ace-72af-4b68-adf9-28bdcddfc06c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:28.251538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8a0c6fa7-d29e-49b7-a567-71d1969d661c req-aef04ace-72af-4b68-adf9-28bdcddfc06c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:28.251989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8a0c6fa7-d29e-49b7-a567-71d1969d661c req-aef04ace-72af-4b68-adf9-28bdcddfc06c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:28.268441 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8a0c6fa7-d29e-49b7-a567-71d1969d661c req-aef04ace-72af-4b68-adf9-28bdcddfc06c service nova] Creating event network-changed:fdaa8af5-5d45-4b21-be18-4bc6b852d171 for instance b6420b15-803c-41f2-b430-b770b2f46e64 on nch1-master-1 Oct 15 21:09:28.274118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8a0c6fa7-d29e-49b7-a567-71d1969d661c req-aef04ace-72af-4b68-adf9-28bdcddfc06c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050217 Oct 15 21:09:28.274886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3208/4322] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:28.280351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:28.281305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:28.287698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:28.288190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:28.288702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:28.289431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:28.290221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:28.290669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:28.594102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:28.605428 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4df6f8e9-fd5d-44b8-ab15-2c9009d53118 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1315 microversion: 2.1 time: 0.329093 Oct 15 21:09:28.606127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3209/4323] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:28 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1315 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:28.622187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1385616250", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7cae7909-1e8f-4b21-a25f-4a74daa4a2c1"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:28.708755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] validate_networks() for [('7cae7909-1e8f-4b21-a25f-4a74daa4a2c1', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:09:30.123056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:09:30.123608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:09:30.123962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:09:30.124372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:09:30.124983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:09:30.137346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:30.138142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:30.138748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:30.139655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:30.140512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:30.141259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:30.162629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Getting quotas for project 96091c3dac03410dbb4f59faa1482781. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:09:30.169379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Getting quotas for user dd615ca74887465b8e21b7f4011fb53f and project 96091c3dac03410dbb4f59faa1482781. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:09:30.184697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:09:30.187624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:09:30.266133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-131723dc-a905-4b29-8d2c-9b7966de7e4b tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.653470 Oct 15 21:09:30.271769 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3210/4324] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:09:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 1661 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:09:30.282852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:30.284062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:30.304623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:30.305616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:30.306095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:30.306838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:30.313606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:30.314216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:30.925710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:30.934553 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d351bd27-082c-458c-a615-abd1dfe8738d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1503 microversion: 2.1 time: 0.660639 Oct 15 21:09:30.948756 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3211/4325] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:30 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1503 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:30.960531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:30.961570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:30.970162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:30.970776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:30.971327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:30.972319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:30.973020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:30.973581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:31.786104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:31.802289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b4b4740-a69a-45e5-b1d8-16c37e443ede tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1321 microversion: 2.1 time: 0.852087 Oct 15 21:09:31.810341 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3212/4326] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:30 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1321 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:31.815386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:31.819714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:31.837312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:31.839221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:31.840417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:31.842301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:31.843131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:31.843798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.507290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:32.530402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43362760-e6e5-4ea1-babf-a236677ec798 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1315 microversion: 2.1 time: 0.718552 Oct 15 21:09:32.532273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3213/4327] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:31 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1315 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:32.539880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7b5f6f65-ffeb-4257-93b4-7b9345b8cde7 req-a1c32919-0050-464c-a73e-bb7826ad1a11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76691e91-f327-49c4-88ce-d6932390510f", "name": "network-vif-plugged", "status": "completed", "tag": "b51cdbdc-d3ee-4cf7-85b1-07fec0d15ffc"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:32.564377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7b5f6f65-ffeb-4257-93b4-7b9345b8cde7 req-a1c32919-0050-464c-a73e-bb7826ad1a11 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.564879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7b5f6f65-ffeb-4257-93b4-7b9345b8cde7 req-a1c32919-0050-464c-a73e-bb7826ad1a11 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.565352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7b5f6f65-ffeb-4257-93b4-7b9345b8cde7 req-a1c32919-0050-464c-a73e-bb7826ad1a11 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.584211 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7b5f6f65-ffeb-4257-93b4-7b9345b8cde7 req-a1c32919-0050-464c-a73e-bb7826ad1a11 service nova] Creating event network-vif-plugged:b51cdbdc-d3ee-4cf7-85b1-07fec0d15ffc for instance 76691e91-f327-49c4-88ce-d6932390510f on nch2-master-1 Oct 15 21:09:32.590239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7b5f6f65-ffeb-4257-93b4-7b9345b8cde7 req-a1c32919-0050-464c-a73e-bb7826ad1a11 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056688 Oct 15 21:09:32.591563 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3214/4328] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:32.599493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:32.600796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:32.612130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.612997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.614460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.632741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.633569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.636531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.862220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:32.872459 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7a9647f-0d99-4028-82de-9b582ba217f6 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1276 microversion: 2.1 time: 0.279102 Oct 15 21:09:32.884490 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3215/4329] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:32 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1276 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:32.898803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:32.902815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:32.910044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.910691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.911229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.912132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.912589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.913281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.960430 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4e2eb1ae-82f5-45d8-8d5f-06ca5e604370 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.067143 Oct 15 21:09:32.968747 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3216/4330] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:09:32 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments => generated 371 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:32.976899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:32.977749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:32.986018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.987184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.987801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:32.989012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:32.990024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:32.990608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.042687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.043686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.044333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.045158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lazy-loading 'fault' on Instance uuid 91f53d83-e9a4-425b-ba6c-5685483898e5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:09:33.199387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:33.224778 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51bb9221-9d3c-44f0-be7b-139e2cc5f71d tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 200 len: 1302 microversion: 2.1 time: 0.253360 Oct 15 21:09:33.241817 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3217/4331] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:32 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 1302 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:33.249003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e4182633-37f0-4bcd-9a35-39daa2d78e0b req-6069efbb-5bd7-463c-a49b-d98612fac2fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f1f541f-e2cd-4527-b301-15fc8d6b34e9", "name": "network-vif-deleted", "tag": "cdfb68d1-a92c-47c1-b12e-066c2ab0987e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:33.274963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e4182633-37f0-4bcd-9a35-39daa2d78e0b req-6069efbb-5bd7-463c-a49b-d98612fac2fd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.275455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e4182633-37f0-4bcd-9a35-39daa2d78e0b req-6069efbb-5bd7-463c-a49b-d98612fac2fd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.276322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e4182633-37f0-4bcd-9a35-39daa2d78e0b req-6069efbb-5bd7-463c-a49b-d98612fac2fd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.292125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e4182633-37f0-4bcd-9a35-39daa2d78e0b req-6069efbb-5bd7-463c-a49b-d98612fac2fd service nova] Creating event network-vif-deleted:cdfb68d1-a92c-47c1-b12e-066c2ab0987e for instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 on nch1-master-1 Oct 15 21:09:33.301077 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e4182633-37f0-4bcd-9a35-39daa2d78e0b req-6069efbb-5bd7-463c-a49b-d98612fac2fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056636 Oct 15 21:09:33.301628 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3218/4332] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:33.306840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:33.307754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:33.316191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.318368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.319136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.320075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.321745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.322450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.734584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:33.747791 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b31be996-f3fb-43b5-b946-c5c33b4c560c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1321 microversion: 2.1 time: 0.444799 Oct 15 21:09:33.753673 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3219/4333] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:33 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1321 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:33.757003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:33.757712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:33.772919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.774100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.774670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.775493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.776202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.776825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.826769 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dffdad48-9ee9-4661-9a59-a71ddc69884d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/23ae0121-69fa-41e9-b9e5-f82bb6991d5d" status: 200 len: 194 microversion: 2.1 time: 0.076768 Oct 15 21:09:33.827565 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3220/4334] 10.222.0.16 () {58 vars in 1413 bytes} [Sat Oct 15 21:09:33 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/23ae0121-69fa-41e9-b9e5-f82bb6991d5d => generated 194 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:33.837991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:33.838901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:33.849341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.849970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.850281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:33.851630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:33.852270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:33.852860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:34.097378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:34.108536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83e7db63-1bee-4a4c-95a0-1900250c86ea tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1315 microversion: 2.1 time: 0.275581 Oct 15 21:09:34.113786 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3221/4335] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:33 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1315 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:34.119777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:34.123843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:34.140484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:34.141072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:34.141522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:34.142401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:34.142896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:34.144044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:34.335046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lazy-loading 'services' on Instance uuid b381ebc7-0aed-4382-90c2-832cc391b53e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:09:34.538392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:34.561575 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee2c312c-d542-4ad6-9462-ae9b8850eca6 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/23ae0121-69fa-41e9-b9e5-f82bb6991d5d" status: 202 len: 0 microversion: 2.1 time: 0.443996 Oct 15 21:09:34.563688 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3222/4336] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:09:34 2022] DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/23ae0121-69fa-41e9-b9e5-f82bb6991d5d => generated 0 bytes in 449 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:09:34.579261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:34.582706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:34.597317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=6,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:34.602949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:34.605560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:34.606035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:34.606876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:34.607352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:34.607787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.082107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:35.095313 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9abe33e4-49ed-4c27-936b-7924ca52afbf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.528070 Oct 15 21:09:35.105202 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3223/4337] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:34 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:35.106215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:35.109720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: 91f53d83-e9a4-425b-ba6c-5685483898e5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:35.126606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.127166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.127536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.128238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.130864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.133614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.158135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] HTTP exception thrown: Instance 91f53d83-e9a4-425b-ba6c-5685483898e5 could not be found. Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Request method failure captured: Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 HTTP/1.1 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD_zQWFNCn2uFS1y8QvtpN-gOYc2e9NZdP3hf5g6H06SlbtDtExWbKArUYMS3FInJyyFKpbCsbODvBEvBq5x76bFqOgrSbqXjvuc-HzdxI_3nLGNHRiDdcGpiIcx13pmJLGKjMO_2s7Ct5fPo9x7lCUla8vutdoGSdoQRn1jzVhK3E Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 5572b130042c482384a1a06e6a91c784 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueTestJSON-1456269388 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5572b130042c482384a1a06e6a91c784"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5572b130042c482384a1a06e6a91c784"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5572b130042c482384a1a06e6a91c784"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5572b130042c482384a1a06e6a91c784"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueTestJSON-1456269388 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 5572b130042c482384a1a06e6a91c784 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueTestJSON-1456269388 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueTestJSON-1456269388-project Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 461a5c1e334a47eb9d70908227cca0b2 Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueTestJSON-1456269388-project Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 91f53d83-e9a4-425b-ba6c-5685483898e5 could not be found. Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '91f53d83-e9a4-425b-ba6c-5685483898e5'} Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91f53d83-e9a4-425b-ba6c-5685483898e5 could not be found. Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91f53d83-e9a4-425b-ba6c-5685483898e5 could not be found. Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:35.159809 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91f53d83-e9a4-425b-ba6c-5685483898e5 could not be found. Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Returning 404 to user: Instance 91f53d83-e9a4-425b-ba6c-5685483898e5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e6b7dbd-e884-4d07-aa43-8005d047bf44 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5" status: 404 len: 111 microversion: 2.1 time: 0.063547 Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3224/4338] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:35 2022] GET /compute/v2.1/servers/91f53d83-e9a4-425b-ba6c-5685483898e5 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:35.167668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:35.168533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:35.174734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.175303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.175795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.176506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.176986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.177473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.458859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:35.471414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5bc49bda-845f-4c95-b117-847144ca6efd tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1514 microversion: 2.1 time: 0.307528 Oct 15 21:09:35.471931 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3225/4339] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:35 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1514 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:35.477364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:35.478376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:35.493019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.495353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.497808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.500097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.507464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.509215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.826322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:35.837717 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf0cf1e9-e72c-4f4f-addc-1591e159036e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1315 microversion: 2.1 time: 0.364270 Oct 15 21:09:35.838306 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3226/4340] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:35 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1315 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:35.865647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:35.866157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:35.878349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.879342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.881066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.882993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:35.883622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:35.884213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:35.924912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:09:36.087458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4603099a-bceb-4661-9a9b-c03570cadf93 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 204 len: 0 microversion: 2.1 time: 0.234259 Oct 15 21:09:36.090309 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3227/4341] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:35 2022] DELETE /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:36.098986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fa13a066-4113-42d4-8c5a-3a0afc3f7b6d req-019426fc-81ae-492e-81bf-7ad5fd6f30b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91f53d83-e9a4-425b-ba6c-5685483898e5", "name": "network-vif-deleted", "tag": "0a05acd5-32b4-4c32-a416-77c9c17a3aec"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:36.112225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fa13a066-4113-42d4-8c5a-3a0afc3f7b6d req-019426fc-81ae-492e-81bf-7ad5fd6f30b4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.113725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fa13a066-4113-42d4-8c5a-3a0afc3f7b6d req-019426fc-81ae-492e-81bf-7ad5fd6f30b4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.114663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fa13a066-4113-42d4-8c5a-3a0afc3f7b6d req-019426fc-81ae-492e-81bf-7ad5fd6f30b4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.141467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fa13a066-4113-42d4-8c5a-3a0afc3f7b6d req-019426fc-81ae-492e-81bf-7ad5fd6f30b4 service nova] Creating event network-vif-deleted:0a05acd5-32b4-4c32-a416-77c9c17a3aec for instance 91f53d83-e9a4-425b-ba6c-5685483898e5 on nch2-master-1 Oct 15 21:09:36.148959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fa13a066-4113-42d4-8c5a-3a0afc3f7b6d req-019426fc-81ae-492e-81bf-7ad5fd6f30b4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055907 Oct 15 21:09:36.149765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3228/4342] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:36.160135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:36.162846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:36.170020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.170503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.171117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.171854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.172307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.173020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.199149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lazy-loading 'flavor' on Instance uuid 76691e91-f327-49c4-88ce-d6932390510f {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:09:36.288177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1de8a85c-a24d-46db-86be-dfe2c6718da3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action" status: 200 len: 32 microversion: 2.1 time: 0.131867 Oct 15 21:09:36.289123 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3229/4343] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:09:36 2022] POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action => generated 32 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:09:36.296586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:36.297203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:36.303448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.303936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.304388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.305094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.305555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.305985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.337373 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-247ae492-5e26-4eeb-95f9-11b34aed819d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/775f9211-fc12-4110-8490-24df7940a524" status: 200 len: 194 microversion: 2.1 time: 0.046837 Oct 15 21:09:36.338126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3230/4344] 10.222.0.16 () {58 vars in 1413 bytes} [Sat Oct 15 21:09:36 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/775f9211-fc12-4110-8490-24df7940a524 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:36.343257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b27994f2-910f-46e3-9cd1-8ad2f35fa783 req-b288ac1e-35d8-46aa-a268-92d62f7cfc3c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6420b15-803c-41f2-b430-b770b2f46e64", "name": "network-vif-plugged", "status": "completed", "tag": "fdaa8af5-5d45-4b21-be18-4bc6b852d171"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:36.356604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b27994f2-910f-46e3-9cd1-8ad2f35fa783 req-b288ac1e-35d8-46aa-a268-92d62f7cfc3c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.357224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b27994f2-910f-46e3-9cd1-8ad2f35fa783 req-b288ac1e-35d8-46aa-a268-92d62f7cfc3c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.357665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b27994f2-910f-46e3-9cd1-8ad2f35fa783 req-b288ac1e-35d8-46aa-a268-92d62f7cfc3c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.373471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b27994f2-910f-46e3-9cd1-8ad2f35fa783 req-b288ac1e-35d8-46aa-a268-92d62f7cfc3c service nova] Creating event network-vif-plugged:fdaa8af5-5d45-4b21-be18-4bc6b852d171 for instance b6420b15-803c-41f2-b430-b770b2f46e64 on nch1-master-1 Oct 15 21:09:36.378469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b27994f2-910f-46e3-9cd1-8ad2f35fa783 req-b288ac1e-35d8-46aa-a268-92d62f7cfc3c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038704 Oct 15 21:09:36.380416 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3231/4345] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:36.386640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:36.387429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:36.394346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.395121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.395742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.396626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.397317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.397902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.641355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:36.650012 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9ae45ba-7bc1-45b3-bade-3654f3dfc570 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1491 microversion: 2.1 time: 0.268327 Oct 15 21:09:36.651406 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3232/4346] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:36 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1491 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:36.659988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:36.660954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:36.668459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.668944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.669590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.670300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.670750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.671431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.820415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:36.828765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6bd51e29-5e2a-4d01-9015-3251f6fc1d77 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.176119 Oct 15 21:09:36.829367 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3233/4347] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:36 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:36.844652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:36.845375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:36.851724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.852298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.852810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:36.853554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:36.854100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:36.854652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.011362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:37.020009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fbbf985-df1c-4f7f-87a5-dd38a0007773 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.189420 Oct 15 21:09:37.020581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3234/4348] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:36 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:37.026840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:37.028088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:37.040485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.040980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.041953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.042668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.043124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.043557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.141012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lazy-loading 'services' on Instance uuid b381ebc7-0aed-4382-90c2-832cc391b53e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:09:37.237227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7662bee1-6827-48e4-8bc8-fa4ad7823860 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/775f9211-fc12-4110-8490-24df7940a524" status: 202 len: 0 microversion: 2.1 time: 0.215150 Oct 15 21:09:37.238149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3235/4349] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:09:37 2022] DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments/775f9211-fc12-4110-8490-24df7940a524 => generated 0 bytes in 217 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:09:37.244102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:37.245125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:37.254502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.255423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.256102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.257637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.258422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.259081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.471725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:37.483455 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-874bb0c4-5aae-4309-a173-4f301cf71c13 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1315 microversion: 2.1 time: 0.243894 Oct 15 21:09:37.484234 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3236/4350] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:37 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1315 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:37.676017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:37.677173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:37.699471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.704398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.705110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.706053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.706726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.707544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.712787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:37.902311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:37.912338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abb466c3-9fd0-4c43-bf19-df0bd9648673 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1491 microversion: 2.1 time: 0.240744 Oct 15 21:09:37.917684 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3237/4351] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:37 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1491 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:09:37.933338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:37.934485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:37.947419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.949606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.950447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:37.951420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:37.952321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:37.953092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:38.289502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:38.301207 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe743bdd-fc22-4f4c-ab54-bf5bdb16fdd0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.380876 Oct 15 21:09:38.303620 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3238/4352] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:37 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:38.310732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:38.311364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:38.319042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:38.319893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:38.324634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:38.325955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:38.327351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:38.327960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:38.862487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:38.872250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da73ab9b-343a-41e9-b8f5-d35e05a3d35f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.567109 Oct 15 21:09:38.885681 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3239/4353] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:38 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:38.885681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2b387f13-8e99-421a-be14-88573c5301e0 req-e3084c72-5aeb-40d4-af66-c89ac586b64e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b48d8804-59f9-482f-8140-7fb39718c9f9", "tag": "9e359c39-e0d2-4949-be03-d016177ea9c8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:38.892487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2b387f13-8e99-421a-be14-88573c5301e0 req-e3084c72-5aeb-40d4-af66-c89ac586b64e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:38.893021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2b387f13-8e99-421a-be14-88573c5301e0 req-e3084c72-5aeb-40d4-af66-c89ac586b64e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:38.893586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2b387f13-8e99-421a-be14-88573c5301e0 req-e3084c72-5aeb-40d4-af66-c89ac586b64e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:38.907091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2b387f13-8e99-421a-be14-88573c5301e0 req-e3084c72-5aeb-40d4-af66-c89ac586b64e service nova] Creating event network-changed:9e359c39-e0d2-4949-be03-d016177ea9c8 for instance b48d8804-59f9-482f-8140-7fb39718c9f9 on nch2-master-1 Oct 15 21:09:38.912937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2b387f13-8e99-421a-be14-88573c5301e0 req-e3084c72-5aeb-40d4-af66-c89ac586b64e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038136 Oct 15 21:09:38.913806 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3240/4354] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:38.920173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:38.922744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:38.929313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:38.930018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:38.930611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:38.931474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:38.932624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:38.933270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:38.978492 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5cd224b8-2c94-45e1-8018-a508e9515cd7 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.062723 Oct 15 21:09:38.986202 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3241/4355] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:09:38 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments => generated 25 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:09:38.990384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:38.990384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:39.005417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:39.005955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:39.006427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:39.007038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:39.008189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:39.008927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:39.640541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:39.649243 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ac50aa3-0ea3-45eb-b6e0-daf1501c2ffe tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1503 microversion: 2.1 time: 0.666808 Oct 15 21:09:39.651236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3242/4356] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:38 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1503 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:39.657134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:39.657757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:39.664020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:39.664782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:39.665428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:39.667264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:39.667888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:39.668465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:39.916293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:39.930992 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f94a01a0-5248-4bcb-bd03-7ef2b6e423f9 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1491 microversion: 2.1 time: 0.275167 Oct 15 21:09:39.931911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3243/4357] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:39 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1491 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:39.945569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:39.951469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:39.966808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:39.967362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:39.967789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:39.968532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:39.969001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:39.969752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.036859 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1874f955-d016-47ad-8ce3-7c67b66b8c7a tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.102259 Oct 15 21:09:40.037747 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3244/4358] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:09:39 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e/os-volume_attachments => generated 25 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:09:40.046030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:40.047244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:40.061068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.065264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.065895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.066657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.067138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.067578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.152101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:40.177581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=24 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:40.326155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:40.340857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2bee441-ab77-4c6c-985b-28e6c0e027df tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.301601 Oct 15 21:09:40.342335 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3245/4359] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:40 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:40.349887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a03a540f-10f2-48ff-9e88-28d84c740ce8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:40.360404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03a540f-10f2-48ff-9e88-28d84c740ce8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.360404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03a540f-10f2-48ff-9e88-28d84c740ce8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.360404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a03a540f-10f2-48ff-9e88-28d84c740ce8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.407254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a03a540f-10f2-48ff-9e88-28d84c740ce8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:09:40.464598 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a03a540f-10f2-48ff-9e88-28d84c740ce8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64/action" status: 202 len: 0 microversion: 2.1 time: 0.119645 Oct 15 21:09:40.470331 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3246/4360] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:09:40 2022] POST /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64/action => generated 0 bytes in 126 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:09:40.483530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:40.487960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:40.500588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.501165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.501858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.502546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.503151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.503555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.724075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:40.733911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b7413491-da0f-4483-87a0-c29b7e2090de tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.263015 Oct 15 21:09:40.734504 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3247/4361] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:40 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:40.739475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:40.739988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:40.747456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.748416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.748416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.749119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.749779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.750586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.782922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:09:40.924474 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ede01c1-4d5d-46bb-88b1-806f8dc60429 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 204 len: 0 microversion: 2.1 time: 0.187330 Oct 15 21:09:40.925432 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3248/4362] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:40 2022] DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:40.931537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:40.932579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:40.940114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.941437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.942039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:40.942903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:40.943936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:40.944515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.171929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:41.198368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e06a2b00-49aa-45a1-a985-4b1e2451473e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.270427 Oct 15 21:09:41.214376 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3249/4363] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:40 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:41.222100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:41.224734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:41.238950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.239798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.240412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.241493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.243011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.243604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.540040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:41.551961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0af90035-7687-406f-93df-cb8ed8f2b48d tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1673 microversion: 2.1 time: 0.343095 Oct 15 21:09:41.555647 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3250/4364] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:41 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1673 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:41.559373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:41.560003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:41.568147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.568870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.569404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.570069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.570539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.571547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.753261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '461a5c1e334a47eb9d70908227cca0b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5572b130042c482384a1a06e6a91c784', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:41.761326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:41.770218 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55fde902-8017-4ec8-b437-5ded326ddf00 tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 200 len: 1449 microversion: 2.1 time: 0.214740 Oct 15 21:09:41.771527 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3251/4365] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:41 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 1449 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:41.805697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3c1b5d2a-ca52-4fe7-91bc-a1bd5745d373 req-228b3bcf-ef0f-4d26-b78c-e4328a83c449 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a621f033-1287-458c-aa82-a651da088114", "name": "network-vif-deleted", "tag": "abf8f7e6-9012-4b3c-b0df-1c78ab33ae1b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:41.836252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3c1b5d2a-ca52-4fe7-91bc-a1bd5745d373 req-228b3bcf-ef0f-4d26-b78c-e4328a83c449 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.839520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3c1b5d2a-ca52-4fe7-91bc-a1bd5745d373 req-228b3bcf-ef0f-4d26-b78c-e4328a83c449 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.840401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3c1b5d2a-ca52-4fe7-91bc-a1bd5745d373 req-228b3bcf-ef0f-4d26-b78c-e4328a83c449 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.869086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3c1b5d2a-ca52-4fe7-91bc-a1bd5745d373 req-228b3bcf-ef0f-4d26-b78c-e4328a83c449 service nova] Creating event network-vif-deleted:abf8f7e6-9012-4b3c-b0df-1c78ab33ae1b for instance a621f033-1287-458c-aa82-a651da088114 on nch2-master-1 Oct 15 21:09:41.875870 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3c1b5d2a-ca52-4fe7-91bc-a1bd5745d373 req-228b3bcf-ef0f-4d26-b78c-e4328a83c449 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102177 Oct 15 21:09:41.876563 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3252/4366] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:41.882786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:41.884012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:41.892659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.894246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.894921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:41.896631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:41.897378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:41.898169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:42.137401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:42.148927 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c585764-3f00-4a0d-a517-4d2b2fddc85c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.269136 Oct 15 21:09:42.151926 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3253/4367] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:41 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:42.161844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:42.163195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:42.172371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:42.172881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:42.173362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:42.174108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:42.174569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:42.174999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:42.511158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:42.527270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9c2ca6e-ca21-4785-91c6-f30d922de964 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.372215 Oct 15 21:09:42.528001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3254/4368] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:42 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:42.532494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:42.533160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:42.553355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:42.553862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:42.554308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:42.555290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:42.555502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:42.560045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:43.036242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:43.044684 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47648834-8bca-495d-ac2e-6cf786a42ceb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.515814 Oct 15 21:09:43.071089 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3255/4369] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:42 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:43.076956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:43.077808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:43.091213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:43.097976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:43.098856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:43.100435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:43.101278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:43.102395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:43.662329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:43.678290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-628a6915-1b5d-4cb8-94ca-0cd56ed1d49e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1673 microversion: 2.1 time: 0.605284 Oct 15 21:09:43.680930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3256/4370] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:43 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1673 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:43.686895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:43.688313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] [instance: a621f033-1287-458c-aa82-a651da088114] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:43.710573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:43.711504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:43.712174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:43.713777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:43.714699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:43.716883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:43.739566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] HTTP exception thrown: Instance a621f033-1287-458c-aa82-a651da088114 could not be found. Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Request method failure captured: Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 HTTP/1.1 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD_zQWFNCn2uFS1y8QvtpN-gOYc2e9NZdP3hf5g6H06SlbtDtExWbKArUYMS3FInJyyFKpbCsbODvBEvBq5x76bFqOgrSbqXjvuc-HzdxI_3nLGNHRiDdcGpiIcx13pmJLGKjMO_2s7Ct5fPo9x7lCUla8vutdoGSdoQRn1jzVhK3E Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 5572b130042c482384a1a06e6a91c784 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueTestJSON-1456269388 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5572b130042c482384a1a06e6a91c784"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5572b130042c482384a1a06e6a91c784"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5572b130042c482384a1a06e6a91c784"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5572b130042c482384a1a06e6a91c784"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueTestJSON-1456269388 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 5572b130042c482384a1a06e6a91c784 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueTestJSON-1456269388 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueTestJSON-1456269388-project Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 461a5c1e334a47eb9d70908227cca0b2 Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueTestJSON-1456269388-project Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance a621f033-1287-458c-aa82-a651da088114 could not be found. Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'a621f033-1287-458c-aa82-a651da088114'} Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a621f033-1287-458c-aa82-a651da088114 could not be found. Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a621f033-1287-458c-aa82-a651da088114 could not be found. Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:43.741733 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a621f033-1287-458c-aa82-a651da088114 could not be found. Oct 15 21:09:43.747389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:43.749569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] Returning 404 to user: Instance a621f033-1287-458c-aa82-a651da088114 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:43.750711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b65b5b9-bba3-42c7-9ecc-de4b1e62753e tempest-ServerRescueTestJSON-1456269388 tempest-ServerRescueTestJSON-1456269388-project] 10.222.0.16 "GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114" status: 404 len: 111 microversion: 2.1 time: 0.068555 Oct 15 21:09:43.751516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3257/4371] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:43 2022] GET /compute/v2.1/servers/a621f033-1287-458c-aa82-a651da088114 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:43.756606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:43.757536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:43.769888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:43.771550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:43.772562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:43.774083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:43.776586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:43.777420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:44.086771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:44.100149 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7490212b-0940-4f78-a589-b5ed9b9217a4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.346746 Oct 15 21:09:44.101432 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3258/4372] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:43 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:44.127767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:44.132096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:44.142415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:44.143170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:44.145215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:44.146179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:44.146762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:44.147798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:44.502761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:44.522434 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0face4e0-948d-4476-871c-dfca7ad8bddf tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.419631 Oct 15 21:09:44.523323 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3259/4373] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:44 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:44.534994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:44.542950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:44.569642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:44.570151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:44.570591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:44.576192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:44.576688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:44.577164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:45.236257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:45.256654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b78c22f2-ddee-43a5-9459-11147f23d1c9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.731844 Oct 15 21:09:45.262150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3260/4374] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:44 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:45.277549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:45.278439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:45.290322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:45.291607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:45.295215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:45.297990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:45.298637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:45.301523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:45.488125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f4e4fc2f97f4c1a832c6348a97fd155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfe55a7ec21b4e6ebbf53684a480e535', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:45.507838 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efceae83-1ea6-4b83-8d1e-07662abb4a04 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 200 len: 1323 microversion: 2.1 time: 0.240341 Oct 15 21:09:45.512141 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3261/4375] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:45 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 1323 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:45.528061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:45.531708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:45.550590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:45.552133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:45.553650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:45.554415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:45.554900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:45.555342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:45.906436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:45.916621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8be31668-499b-40ce-9bb8-f1288be74161 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.402903 Oct 15 21:09:45.922616 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3262/4376] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:45 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:45.937000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f6a7b107-b481-42f7-82e2-61c339e0e7c9 req-3c9424d5-a30f-4155-a9ec-420aa32235d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b381ebc7-0aed-4382-90c2-832cc391b53e", "name": "network-vif-deleted", "tag": "3d83816b-d4d9-4bb6-ad20-6c0105588621"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:45.976695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f6a7b107-b481-42f7-82e2-61c339e0e7c9 req-3c9424d5-a30f-4155-a9ec-420aa32235d3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:45.983548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f6a7b107-b481-42f7-82e2-61c339e0e7c9 req-3c9424d5-a30f-4155-a9ec-420aa32235d3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:45.988414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f6a7b107-b481-42f7-82e2-61c339e0e7c9 req-3c9424d5-a30f-4155-a9ec-420aa32235d3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.045698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f6a7b107-b481-42f7-82e2-61c339e0e7c9 req-3c9424d5-a30f-4155-a9ec-420aa32235d3 service nova] Creating event network-vif-deleted:3d83816b-d4d9-4bb6-ad20-6c0105588621 for instance b381ebc7-0aed-4382-90c2-832cc391b53e on nch1-master-1 Oct 15 21:09:46.064617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f6a7b107-b481-42f7-82e2-61c339e0e7c9 req-3c9424d5-a30f-4155-a9ec-420aa32235d3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.136008 Oct 15 21:09:46.067491 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3263/4377] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:46.088103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:46.088870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:46.107202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.107867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.108796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.109660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.110238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.110797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.373077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:46.394114 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-11030535-98eb-4f87-8d89-e9ebeba90fe4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.325233 Oct 15 21:09:46.395607 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3264/4378] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:46 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:46.401005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-516e2523-e84f-48bb-93cc-ed4a029fde2d req-f2afe0a8-1724-49c8-afff-97849980a561 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b48d8804-59f9-482f-8140-7fb39718c9f9", "name": "network-vif-plugged", "status": "completed", "tag": "9e359c39-e0d2-4949-be03-d016177ea9c8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:46.422733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-516e2523-e84f-48bb-93cc-ed4a029fde2d req-f2afe0a8-1724-49c8-afff-97849980a561 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.423226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-516e2523-e84f-48bb-93cc-ed4a029fde2d req-f2afe0a8-1724-49c8-afff-97849980a561 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.424396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-516e2523-e84f-48bb-93cc-ed4a029fde2d req-f2afe0a8-1724-49c8-afff-97849980a561 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.447049 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-516e2523-e84f-48bb-93cc-ed4a029fde2d req-f2afe0a8-1724-49c8-afff-97849980a561 service nova] Creating event network-vif-plugged:9e359c39-e0d2-4949-be03-d016177ea9c8 for instance b48d8804-59f9-482f-8140-7fb39718c9f9 on nch2-master-1 Oct 15 21:09:46.463318 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-516e2523-e84f-48bb-93cc-ed4a029fde2d req-f2afe0a8-1724-49c8-afff-97849980a561 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066005 Oct 15 21:09:46.463943 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3265/4379] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:46.471385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:46.472385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:46.480311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.481114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.481808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.482775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.483473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.484211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.839158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:46.847162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4e737ac-564c-4035-931f-c3484255f084 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.379736 Oct 15 21:09:46.850634 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3266/4380] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:46 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:46.858587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:46.863484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:46.875302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:46.878855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.882384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.883183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.884111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:46.885353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:46.888345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:46.927728 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] HTTP exception thrown: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Request method failure captured: Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e HTTP/1.1 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyB4RquiCnYmgp3soXHTxPzpZGgu0GC4FJjNmBkyJLbR7XR-EuLkOyVrYSEJIWELDgGGJ4liNiIb-CZIzbEllaNfkI-nG3QKBEFn61aO-uS1NViXHjKT6dW1_WcVD-lpOc_ivykZy27VcldLmm5gwwTaP1sA39wITsqmIqUEZPXWwfE Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2f4e4fc2f97f4c1a832c6348a97fd155 Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b381ebc7-0aed-4382-90c2-832cc391b53e'} Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:46.929476 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Returning 404 to user: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:46.934705 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d409401b-41e0-480f-a4e4-8efba923afbf tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 404 len: 111 microversion: 2.1 time: 0.076133 Oct 15 21:09:46.936466 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3267/4381] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:46 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:46.948534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bd8c4f16-8f72-4260-89e8-924000e7ae0e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '.version_select at 0x7fe74e16a4c0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:46.970535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bd8c4f16-8f72-4260-89e8-924000e7ae0e tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1950741633" status: 202 len: 0 microversion: 2.1 time: 0.028016 Oct 15 21:09:46.971591 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3268/4382] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:09:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1950741633 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:09:46.992713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:46.995103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:47.011376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:47.012298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:47.013958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:47.017179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:47.017926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:47.020039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:47.700109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:47.707982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f69a595-115a-4f95-ab72-5296fb70a613 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1318 microversion: 2.1 time: 0.735173 Oct 15 21:09:47.708885 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3269/4383] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:46 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1318 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:47.714277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:47.715164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:47.721673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:47.722482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:47.723220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:47.724318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:47.725070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:47.725663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:48.051540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:48.060742 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d867be0-7fdf-4bd6-9d3f-1fa6291a3728 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.350534 Oct 15 21:09:48.061365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3270/4384] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:47 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:48.069020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:48.069856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:48.078304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:48.079019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:48.079727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:48.080935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:48.081330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:48.081866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:48.424804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:48.437870 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a0e5e25-61ad-4890-8331-527a42488a35 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.375196 Oct 15 21:09:48.439261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3271/4385] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:48 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:48.732535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:48.733196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:48.748690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:48.749207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:48.750025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:48.750983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:48.751816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:48.752632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:49.113432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:49.118834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:49.126074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d255466-960d-4cb9-a2c5-58c9fb2800eb tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1507 microversion: 2.1 time: 0.398160 Oct 15 21:09:49.127644 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3272/4386] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:48 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1507 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:09:49.166812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:49.170887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:49.210240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:49.213582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:49.214355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:49.215427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:49.216149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:49.217516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:49.495403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:49.534167 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-92424675-e060-4583-a9e8-3c716fadc2e7 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.404302 Oct 15 21:09:49.535129 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3273/4387] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:49 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:49.564953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:49.568816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:49.582059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:49.582987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:49.583842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:49.585175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:49.585402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:49.585997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:49.995896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:50.030763 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66ff4172-2d11-4198-b9f1-c27f7a6c0038 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1507 microversion: 2.1 time: 0.472869 Oct 15 21:09:50.032040 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3274/4388] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:49 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1507 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:50.044535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:50.046301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:50.085995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.086559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.090045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.091524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.093488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.093965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.503262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:50.521102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2635d9c3-0e9a-41c8-b331-539fdc65e269 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.484538 Oct 15 21:09:50.540483 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3275/4389] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:50 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:50.553292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:50.559509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:50.579143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.579754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.580483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.581595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.582229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.582837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.715141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61a526f9-4dd5-416d-bf30-4942a766d7f0 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "PUT /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1079 microversion: 2.1 time: 0.171707 Oct 15 21:09:50.716213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3276/4390] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:09:50 2022] PUT /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1079 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:50.723037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:50.724104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:50.732105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.732818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.733543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.734668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.735468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.736232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.902915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:50.913578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa62ccf5-8c2e-4441-b961-c1b149612c14 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.195041 Oct 15 21:09:50.914463 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3277/4391] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:50 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:50.931116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:50.934574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:50.951852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.952370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.952820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:50.953610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:50.954076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:50.957209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.374234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:51.389564 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c5c587a-8afc-40e5-8526-9a3b224ab8ab tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1505 microversion: 2.1 time: 0.472860 Oct 15 21:09:51.391217 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3278/4392] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:50 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1505 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:51.417999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:51.418600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:51.426916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.427679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.428328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.429442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.430386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.431280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.452493 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] HTTP exception thrown: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Request method failure captured: Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e HTTP/1.1 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyB4RquiCnYmgp3soXHTxPzpZGgu0GC4FJjNmBkyJLbR7XR-EuLkOyVrYSEJIWELDgGGJ4liNiIb-CZIzbEllaNfkI-nG3QKBEFn61aO-uS1NViXHjKT6dW1_WcVD-lpOc_ivykZy27VcldLmm5gwwTaP1sA39wITsqmIqUEZPXWwfE Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2f4e4fc2f97f4c1a832c6348a97fd155 Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b381ebc7-0aed-4382-90c2-832cc391b53e'} Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:51.458496 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.463070 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.465356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Returning 404 to user: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:51.465356 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6159af5-7700-4667-be45-457c50e40899 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 404 len: 111 microversion: 2.1 time: 0.054936 Oct 15 21:09:51.465892 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3279/4393] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:51 2022] DELETE /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:51.471068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:51.471909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:51.478284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.479669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.481321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.482439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.483154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.483791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.692666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:51.707611 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72d44308-7cfe-4f94-9d2f-b0e97d8ce58c tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1505 microversion: 2.1 time: 0.240595 Oct 15 21:09:51.722237 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3280/4394] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:51 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1505 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:51.733447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:51.733995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: b381ebc7-0aed-4382-90c2-832cc391b53e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:51.740734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.741518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.742121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.743178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.743794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.744972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.761205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] HTTP exception thrown: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Request method failure captured: Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e HTTP/1.1 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyB4RquiCnYmgp3soXHTxPzpZGgu0GC4FJjNmBkyJLbR7XR-EuLkOyVrYSEJIWELDgGGJ4liNiIb-CZIzbEllaNfkI-nG3QKBEFn61aO-uS1NViXHjKT6dW1_WcVD-lpOc_ivykZy27VcldLmm5gwwTaP1sA39wITsqmIqUEZPXWwfE Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2f4e4fc2f97f4c1a832c6348a97fd155 Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b381ebc7-0aed-4382-90c2-832cc391b53e'} Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:51.763164 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. Oct 15 21:09:51.766980 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:51.769431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Returning 404 to user: Instance b381ebc7-0aed-4382-90c2-832cc391b53e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:51.770476 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4b2c058c-10b7-412f-aa77-940f5ebdd3c0 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e" status: 404 len: 111 microversion: 2.1 time: 0.047037 Oct 15 21:09:51.774417 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3281/4395] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:51 2022] GET /compute/v2.1/servers/b381ebc7-0aed-4382-90c2-832cc391b53e => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:51.788209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:51.789160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:51.804065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.804964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.809486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:51.810532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:51.811270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:51.812418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:52.180617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:52.191771 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4865da8-c286-4c60-bac4-c11d7a6ec1ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.418930 Oct 15 21:09:52.193949 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3282/4396] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:51 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:52.209622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "MANUAL"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:52.216474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:52.241198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:52.241752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:52.242449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:52.243506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:52.244278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:52.245783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:52.470846 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5bcd2441-6f73-4b5e-9f2f-bcc445d4cc17 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "PUT /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1081 microversion: 2.1 time: 0.273355 Oct 15 21:09:52.472051 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3283/4397] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:09:52 2022] PUT /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1081 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:52.477583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:52.478670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:52.486111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:52.487768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:52.488457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:52.490547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:52.492807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:52.493653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:52.733922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:52.748441 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9800523-e32c-4f1c-8aab-3a3e905e2cfe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.275268 Oct 15 21:09:52.749084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3284/4398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:52 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:52.761878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:52.762775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:52.772289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:52.773842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:52.774286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:52.775218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:52.778002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:52.778513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.016807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:53.026285 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-29cfef12-a44d-4fda-af65-d17d677730b4 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1507 microversion: 2.1 time: 0.275739 Oct 15 21:09:53.027194 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3285/4399] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:52 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1507 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:53.057390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:53.060595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:53.068742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.069448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.070002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.071565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.072294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.073235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.287918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:53.297537 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5142ed4-d421-4c41-a4c4-34e9bf3698bf tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1507 microversion: 2.1 time: 0.250726 Oct 15 21:09:53.298864 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3286/4400] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:53 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1507 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:09:53.303624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:53.304370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:53.313181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.314011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.314676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.315639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.316323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.316976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.334233 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] HTTP exception thrown: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Request method failure captured: Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 HTTP/1.1 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyB4RquiCnYmgp3soXHTxPzpZGgu0GC4FJjNmBkyJLbR7XR-EuLkOyVrYSEJIWELDgGGJ4liNiIb-CZIzbEllaNfkI-nG3QKBEFn61aO-uS1NViXHjKT6dW1_WcVD-lpOc_ivykZy27VcldLmm5gwwTaP1sA39wITsqmIqUEZPXWwfE Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2f4e4fc2f97f4c1a832c6348a97fd155 Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd4ea7154-0623-44f0-8bad-076461f12dd5'} Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:53.336025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.340348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Returning 404 to user: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:53.342053 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d971e26-c4fb-4a15-b45a-44d4ea2c7837 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 404 len: 111 microversion: 2.1 time: 0.041525 Oct 15 21:09:53.343103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3287/4401] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:53 2022] DELETE /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:53.348736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:53.349781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:53.362423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.363039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.363555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.364538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.365001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.365461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.541953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:53.554895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c7de273-e3c3-4886-aaf9-374f27327d2c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.210671 Oct 15 21:09:53.571787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3288/4402] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:53 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:53.579510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:53.580362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:53.587500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.592785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.593968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.595134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.595833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.596392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.627832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:09:53.645497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:09:53.782062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a31d612-7b11-43cd-8801-5b23814e3767 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 204 len: 0 microversion: 2.1 time: 0.206718 Oct 15 21:09:53.783280 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3289/4403] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:53 2022] DELETE /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:53.793185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:53.794182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] [instance: d4ea7154-0623-44f0-8bad-076461f12dd5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:53.802522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.803311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.803984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.805007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.805740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.806503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.824793 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] HTTP exception thrown: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Request method failure captured: Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 HTTP/1.1 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyB4RquiCnYmgp3soXHTxPzpZGgu0GC4FJjNmBkyJLbR7XR-EuLkOyVrYSEJIWELDgGGJ4liNiIb-CZIzbEllaNfkI-nG3QKBEFn61aO-uS1NViXHjKT6dW1_WcVD-lpOc_ivykZy27VcldLmm5gwwTaP1sA39wITsqmIqUEZPXWwfE Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfe55a7ec21b4e6ebbf53684a480e535"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cfe55a7ec21b4e6ebbf53684a480e535 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1281435182 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 2f4e4fc2f97f4c1a832c6348a97fd155 Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeTestJSON-1281435182-project Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd4ea7154-0623-44f0-8bad-076461f12dd5'} Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:53.832153 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] Returning 404 to user: Instance d4ea7154-0623-44f0-8bad-076461f12dd5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:53.836551 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01e2de5e-3201-4270-ad81-0bcda2203f71 tempest-AttachVolumeTestJSON-1281435182 tempest-AttachVolumeTestJSON-1281435182-project] 10.222.0.16 "GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5" status: 404 len: 111 microversion: 2.1 time: 0.049523 Oct 15 21:09:53.837209 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3290/4404] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:53 2022] GET /compute/v2.1/servers/d4ea7154-0623-44f0-8bad-076461f12dd5 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:53.842345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:53.843038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:53.849437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.849907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.850445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:53.851236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:53.851891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:53.852403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.059545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:54.070249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4047617-0c94-470a-b176-7f4d28f89876 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.231853 Oct 15 21:09:54.070858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3291/4405] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:53 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:54.075831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:54.076566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:54.085372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:54.086248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:54.086855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.088492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:54.089140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:54.089750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.119398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:09:54.170928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.compute.api [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Instance is already in deleting state, ignoring this request Oct 15 21:09:54.171849 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25a9fa92-e14b-40bd-8c73-f189321749aa tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 204 len: 0 microversion: 2.1 time: 0.100060 Oct 15 21:09:54.172495 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3292/4406] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:54 2022] DELETE /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:09:54.184293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:54.185203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:54.192344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:54.193270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:54.194855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.195783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:54.196442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:54.197168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.454206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:54.469210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad3a7f48-621b-48a9-91f7-ca8df110e338 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1513 microversion: 2.1 time: 0.287102 Oct 15 21:09:54.472712 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3293/4407] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:54 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1513 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:54.584773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:54.586530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:54.602012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:54.602973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:54.605332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.609763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:54.610726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:54.611524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:54.924899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:54.936932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-faddf812-151e-408a-9f9a-dcc130014a89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.356515 Oct 15 21:09:54.942812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3294/4408] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:54 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:55.102647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:55.104777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:55.123881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:55.124985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:55.126588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:55.128343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:55.129139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:55.130983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:55.560366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:55.590020 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0810a01c-f4ba-45c3-b54d-48bcea977b5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.493765 Oct 15 21:09:55.595430 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3295/4409] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:55 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:55.601573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:55.607924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:55.624305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:55.625215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:55.625902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:55.627140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:55.627962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:55.628985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:56.145403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd615ca74887465b8e21b7f4011fb53f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96091c3dac03410dbb4f59faa1482781', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:56.154069 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb1ed254-f002-4b66-98ca-284e5c24ba39 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 200 len: 1513 microversion: 2.1 time: 0.557205 Oct 15 21:09:56.154645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3296/4410] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:55 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 1513 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:56.171673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:56.172541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:56.182404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:56.182899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:56.183336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:56.184046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:56.184507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:56.184930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:56.641309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:56.654261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a921ff5-105f-43b7-9f41-6344640cf220 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.487641 Oct 15 21:09:56.655169 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3297/4411] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:56 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:56.662393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-db3f8a7f-8dac-41a0-a64c-73dac9aa0a92 req-404b01b3-e363-460a-abf7-e1f4510d028e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b48d8804-59f9-482f-8140-7fb39718c9f9", "name": "network-vif-deleted", "tag": "9e359c39-e0d2-4949-be03-d016177ea9c8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:56.691258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-db3f8a7f-8dac-41a0-a64c-73dac9aa0a92 req-404b01b3-e363-460a-abf7-e1f4510d028e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:56.691794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-db3f8a7f-8dac-41a0-a64c-73dac9aa0a92 req-404b01b3-e363-460a-abf7-e1f4510d028e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:56.694520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-db3f8a7f-8dac-41a0-a64c-73dac9aa0a92 req-404b01b3-e363-460a-abf7-e1f4510d028e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:56.727095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-db3f8a7f-8dac-41a0-a64c-73dac9aa0a92 req-404b01b3-e363-460a-abf7-e1f4510d028e service nova] Creating event network-vif-deleted:9e359c39-e0d2-4949-be03-d016177ea9c8 for instance b48d8804-59f9-482f-8140-7fb39718c9f9 on nch2-master-1 Oct 15 21:09:56.735578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-db3f8a7f-8dac-41a0-a64c-73dac9aa0a92 req-404b01b3-e363-460a-abf7-e1f4510d028e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078637 Oct 15 21:09:56.743222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3298/4412] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:09:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:56.784093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:56.784959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:56.839230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:56.841553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:56.843639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:56.846287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:56.848553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:56.850174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:57.749009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:57.757819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0719d461-cb4a-460d-a989-55c689e3c44d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 1.012784 Oct 15 21:09:57.764538 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3299/4413] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:56 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 1020 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:57.790549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:57.791352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: b48d8804-59f9-482f-8140-7fb39718c9f9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:57.804650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:57.805158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:57.805597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:57.806847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:57.807308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:57.807732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:57.840520 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] HTTP exception thrown: Instance b48d8804-59f9-482f-8140-7fb39718c9f9 could not be found. Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Request method failure captured: Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 HTTP/1.1 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD5WRIH7AIxdZYDucTULbEPC5dyYsGuMnEKDjK1rS-3Xg_f0WQ-LyLU3hFXpFOljzToVK-F0s0lLgWI6QXoJw5ljVU8fC-g_qWljXlHMXL6vpyfFJmyONYRUxxSDyD9n_smzmV14LWfcUfFpkN4IwF2tSRrXf5kZanrwGzxZ_Zn-qc Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96091c3dac03410dbb4f59faa1482781"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: dd615ca74887465b8e21b7f4011fb53f Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b48d8804-59f9-482f-8140-7fb39718c9f9 could not be found. Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b48d8804-59f9-482f-8140-7fb39718c9f9'} Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b48d8804-59f9-482f-8140-7fb39718c9f9 could not be found. Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b48d8804-59f9-482f-8140-7fb39718c9f9 could not be found. Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:57.858645 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b48d8804-59f9-482f-8140-7fb39718c9f9 could not be found. Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Returning 404 to user: Instance b48d8804-59f9-482f-8140-7fb39718c9f9 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-96de55a8-2dcd-44ba-8312-a8be46382534 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9" status: 404 len: 111 microversion: 2.1 time: 0.088383 Oct 15 21:09:57.864224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3300/4414] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:57 2022] GET /compute/v2.1/servers/b48d8804-59f9-482f-8140-7fb39718c9f9 => generated 111 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:58.400665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40d377b5-74c3-4656-92ac-12e2327debf8 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Action: 'create', calling method: .version_select at 0x7fe74e13edc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-727934137", "metadata": {"Type": "work"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:09:59.043303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40d377b5-74c3-4656-92ac-12e2327debf8 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 1.177645 Oct 15 21:09:59.044087 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3301/4415] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:09:57 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 1180 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 15 21:09:59.050461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:59.053938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:59.089988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.094802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.098647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.105431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.115337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.118385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.576837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:59.587983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50ab0d2a-6d21-499a-a6e6-49a2f84e40ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.541980 Oct 15 21:09:59.588764 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3302/4416] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:59 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:59.594701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:59.595522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:59.602305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.603110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.604127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.605162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.606362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.607001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.646272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] HTTP exception thrown: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Request method failure captured: Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 HTTP/1.1 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD5WRIH7AIxdZYDucTULbEPC5dyYsGuMnEKDjK1rS-3Xg_f0WQ-LyLU3hFXpFOljzToVK-F0s0lLgWI6QXoJw5ljVU8fC-g_qWljXlHMXL6vpyfFJmyONYRUxxSDyD9n_smzmV14LWfcUfFpkN4IwF2tSRrXf5kZanrwGzxZ_Zn-qc Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96091c3dac03410dbb4f59faa1482781"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: dd615ca74887465b8e21b7f4011fb53f Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '9f1f541f-e2cd-4527-b301-15fc8d6b34e9'} Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:59.653766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Returning 404 to user: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d95bca66-5d73-4a89-b5c9-5a73a3cdc7f7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 404 len: 111 microversion: 2.1 time: 0.066457 Oct 15 21:09:59.657575 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3303/4417] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:09:59 2022] DELETE /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:59.664877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:59.666202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:59.675712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.676245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.676670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.677674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.678328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.678830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.841216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:09:59.851306 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3bc298f-14d9-4d4f-9214-99b7a60e1366 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1520 microversion: 2.1 time: 0.192681 Oct 15 21:09:59.852740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3304/4418] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:59 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1520 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:09:59.858142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6eb95da8-98b3-4f62-aef3-08d6dc151f8b tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Calling method '.version_select at 0x7fe74e134c10>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:59.918974 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6eb95da8-98b3-4f62-aef3-08d6dc151f8b tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b" status: 200 len: 346 microversion: 2.1 time: 0.064319 Oct 15 21:09:59.919569 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3305/4419] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:09:59 2022] GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b => generated 346 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:09:59.926193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:09:59.927049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 9f1f541f-e2cd-4527-b301-15fc8d6b34e9] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:09:59.933928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.934427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.934894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.935824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:09:59.936299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:09:59.936740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:09:59.950851 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] HTTP exception thrown: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Request method failure captured: Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 HTTP/1.1 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD5WRIH7AIxdZYDucTULbEPC5dyYsGuMnEKDjK1rS-3Xg_f0WQ-LyLU3hFXpFOljzToVK-F0s0lLgWI6QXoJw5ljVU8fC-g_qWljXlHMXL6vpyfFJmyONYRUxxSDyD9n_smzmV14LWfcUfFpkN4IwF2tSRrXf5kZanrwGzxZ_Zn-qc Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96091c3dac03410dbb4f59faa1482781"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: dd615ca74887465b8e21b7f4011fb53f Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '9f1f541f-e2cd-4527-b301-15fc8d6b34e9'} Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:09:59.953084 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Returning 404 to user: Instance 9f1f541f-e2cd-4527-b301-15fc8d6b34e9 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9a4dddb-4ea2-43a6-9667-e797afcbec01 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9" status: 404 len: 111 microversion: 2.1 time: 0.032992 Oct 15 21:09:59.956828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3306/4420] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:09:59 2022] GET /compute/v2.1/servers/9f1f541f-e2cd-4527-b301-15fc8d6b34e9 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:09:59.963426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c595a109-1d7a-4b1d-84e4-3e7684243944 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Calling method '.version_select at 0x7fe758462160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:00.033912 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c595a109-1d7a-4b1d-84e4-3e7684243944 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b" status: 200 len: 346 microversion: 2.1 time: 0.077960 Oct 15 21:10:00.035462 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3307/4421] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:09:59 2022] GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b => generated 346 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:00.041109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:00.042878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:00.057160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:00.057983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:00.060369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:00.061830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:00.064104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:00.064813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:00.095037 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] HTTP exception thrown: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Request method failure captured: Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a HTTP/1.1 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD5WRIH7AIxdZYDucTULbEPC5dyYsGuMnEKDjK1rS-3Xg_f0WQ-LyLU3hFXpFOljzToVK-F0s0lLgWI6QXoJw5ljVU8fC-g_qWljXlHMXL6vpyfFJmyONYRUxxSDyD9n_smzmV14LWfcUfFpkN4IwF2tSRrXf5kZanrwGzxZ_Zn-qc Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96091c3dac03410dbb4f59faa1482781"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: dd615ca74887465b8e21b7f4011fb53f Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '18be360f-e339-42ec-89ff-72acf428932a'} Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:10:00.097272 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.101113 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.108541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Returning 404 to user: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:00.113638 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4c91ae1-7316-47e2-b155-116fc95f0fa5 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "DELETE /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 404 len: 111 microversion: 2.1 time: 0.074117 Oct 15 21:10:00.119984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3308/4422] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:10:00 2022] DELETE /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:10:00.128796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c548047c-7478-4881-9b4e-7575440b9c96 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Calling method '.version_select at 0x7fe74e1218b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:00.257869 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c548047c-7478-4881-9b4e-7575440b9c96 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b" status: 202 len: 0 microversion: 2.1 time: 0.140036 Oct 15 21:10:00.258537 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3309/4423] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:10:00 2022] DELETE /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b => generated 0 bytes in 141 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:00.270640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:00.271717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] [instance: 18be360f-e339-42ec-89ff-72acf428932a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:00.315799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:00.316305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:00.316754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:00.317618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:00.318091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:00.318530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:00.361871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] HTTP exception thrown: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Request method failure captured: Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a HTTP/1.1 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyD5WRIH7AIxdZYDucTULbEPC5dyYsGuMnEKDjK1rS-3Xg_f0WQ-LyLU3hFXpFOljzToVK-F0s0lLgWI6QXoJw5ljVU8fC-g_qWljXlHMXL6vpyfFJmyONYRUxxSDyD9n_smzmV14LWfcUfFpkN4IwF2tSRrXf5kZanrwGzxZ_Zn-qc Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96091c3dac03410dbb4f59faa1482781"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96091c3dac03410dbb4f59faa1482781"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 96091c3dac03410dbb4f59faa1482781 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-382496067 Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: dd615ca74887465b8e21b7f4011fb53f Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiskConfigTestJSON-382496067-project Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '18be360f-e339-42ec-89ff-72acf428932a'} Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:00.364624 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. Oct 15 21:10:00.370273 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:00.373832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] Returning 404 to user: Instance 18be360f-e339-42ec-89ff-72acf428932a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:00.376414 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95b6a543-b617-4d69-bfe8-2b42c3b32ed7 tempest-ServerDiskConfigTestJSON-382496067 tempest-ServerDiskConfigTestJSON-382496067-project] 10.222.0.16 "GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a" status: 404 len: 111 microversion: 2.1 time: 0.114990 Oct 15 21:10:00.378615 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3310/4424] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:00 2022] GET /compute/v2.1/servers/18be360f-e339-42ec-89ff-72acf428932a => generated 111 bytes in 119 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:10:00.392042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83a8658c-d4b7-438e-afeb-7a1e476f18b3 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Calling method '.version_select at 0x7fe74e1210d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:00.521652 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83a8658c-d4b7-438e-afeb-7a1e476f18b3 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b" status: 200 len: 345 microversion: 2.1 time: 0.139736 Oct 15 21:10:00.522566 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3311/4425] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:10:00 2022] GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b => generated 345 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:00.613317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:00.614680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:00.631884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:00.633460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:00.635145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:00.638940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:00.640361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:00.641086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:01.187923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:01.250012 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0548282c-9f68-43e5-8d74-67973c4c313c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.599978 Oct 15 21:10:01.250012 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3312/4426] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:00 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:01.250012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:01.250012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:01.251237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:01.259566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:01.260401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:01.262059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:01.262726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:01.264509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:01.325593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:01.759009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:01.771167 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f90f18e-4eae-44c5-80ab-4d79605550f2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1500 microversion: 2.1 time: 0.541664 Oct 15 21:10:01.771762 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3313/4427] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:01 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1500 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:01.794921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d703728c-b8b9-40c9-aa16-ce1615f56384 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Calling method '.version_select at 0x7fe74e121550>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:01.832029 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d703728c-b8b9-40c9-aa16-ce1615f56384 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] HTTP exception thrown: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d703728c-b8b9-40c9-aa16-ce1615f56384 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Request method failure captured: Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b HTTP/1.1 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyGl3NWMH5UBDOj2G-J26qV6pKSqX045xyNBSP71GNCiLE1xfz5UxLCrCYoy3Gq5vFPtxEpKFsFPkVOwZHeyONgVQ2kOQk8jkSzFBC3X7d9MIrsloXLBN-_xjrNCPZXteT8YKIph70uldiZEs9TEQLx8UDAOamwfFG_v6L_zoYhURP8 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 4e6b708b260348cf904e392e1329e1d2 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesGetTestJSON-28415155 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4e6b708b260348cf904e392e1329e1d2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4e6b708b260348cf904e392e1329e1d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4e6b708b260348cf904e392e1329e1d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4e6b708b260348cf904e392e1329e1d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesGetTestJSON-28415155 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 4e6b708b260348cf904e392e1329e1d2 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesGetTestJSON-28415155 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesGetTestJSON-28415155-project Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 44af21476ed8446d98640881d322eea3 Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesGetTestJSON-28415155-project Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: exception: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '63af8a7a-9943-4704-880b-fce7ba8de25b'} Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. (HTTP 404) (Request-ID: req-c4e7daf7-a419-407f-af9c-fb959c8d1568) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:10:01.834856 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. Oct 15 21:10:01.841506 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:01.845024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d703728c-b8b9-40c9-aa16-ce1615f56384 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] Returning 404 to user: Volume 63af8a7a-9943-4704-880b-fce7ba8de25b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:01.846364 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d703728c-b8b9-40c9-aa16-ce1615f56384 tempest-VolumesGetTestJSON-28415155 tempest-VolumesGetTestJSON-28415155-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b" status: 404 len: 109 microversion: 2.1 time: 0.063606 Oct 15 21:10:01.847134 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3314/4428] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:10:01 2022] GET /compute/v2.1/os-volumes/63af8a7a-9943-4704-880b-fce7ba8de25b => generated 109 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:10:01.858704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:01.859323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:01.870137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:01.870998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:01.871777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:01.872771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:01.873806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:01.874409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:02.373805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:02.387158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fa03bd89-ed2c-45cc-88c5-508f6d397b1e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1512 microversion: 2.1 time: 0.530945 Oct 15 21:10:02.388099 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3315/4429] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:01 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1512 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:02.417446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:02.419786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:02.442691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:02.443206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:02.443662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:02.444397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:02.444865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:02.445352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:02.877400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:02.894100 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e614b39-ea61-458b-bee5-dc826397e251 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.501926 Oct 15 21:10:02.896193 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3316/4430] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:02 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:02.919499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa06c0ef-5376-4b88-b959-9b40d7732ca6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"rescue": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:02.929662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa06c0ef-5376-4b88-b959-9b40d7732ca6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: bc9cd226-fcaa-40a2-9a7e-b93f523335a3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:02.947110 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-aa06c0ef-5376-4b88-b959-9b40d7732ca6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Instance bc9cd226-fcaa-40a2-9a7e-b93f523335a3 could not be found. Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa06c0ef-5376-4b88-b959-9b40d7732ca6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/bc9cd226-fcaa-40a2-9a7e-b93f523335a3/action HTTP/1.1 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 14 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: {"rescue": {}} Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance bc9cd226-fcaa-40a2-9a7e-b93f523335a3 could not be found. Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'bc9cd226-fcaa-40a2-9a7e-b93f523335a3', 'body': {'rescue': {}}} Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bc9cd226-fcaa-40a2-9a7e-b93f523335a3 could not be found. Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 53, in _rescue Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bc9cd226-fcaa-40a2-9a7e-b93f523335a3 could not be found. Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:02.961240 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bc9cd226-fcaa-40a2-9a7e-b93f523335a3 could not be found. Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa06c0ef-5376-4b88-b959-9b40d7732ca6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 404 to user: Instance bc9cd226-fcaa-40a2-9a7e-b93f523335a3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa06c0ef-5376-4b88-b959-9b40d7732ca6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/bc9cd226-fcaa-40a2-9a7e-b93f523335a3/action" status: 404 len: 111 microversion: 2.1 time: 0.067414 Oct 15 21:10:02.967613 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3317/4431] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:02 2022] POST /compute/v2.1/servers/bc9cd226-fcaa-40a2-9a7e-b93f523335a3/action => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:10:02.977212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:02.978095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:03.001493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:03.002270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:03.003538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:03.005413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:03.007105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:03.007962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:03.161903 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6b05179-03c0-4d71-8b26-a76ffcc67bdb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action" status: 202 len: 0 microversion: 2.1 time: 0.189632 Oct 15 21:10:03.165672 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3318/4432] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:02 2022] POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:03.171439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:03.176155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:03.205629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:03.208556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:03.209289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:03.209910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:03.210946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:03.211564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:03.212355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:03.862455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:03.879714 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-978b69f4-974e-42f2-b2c9-39d17b60ee62 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1517 microversion: 2.1 time: 0.712269 Oct 15 21:10:03.897984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3319/4433] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:03 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1517 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:03.925089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:03.925954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:03.955987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:03.956507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:03.959148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:03.960074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:03.960558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:03.961007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:04.906972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:04.919009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5234ab3a-93f8-4462-9900-ba9fdc21a704 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1513 microversion: 2.1 time: 0.998670 Oct 15 21:10:04.923411 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3320/4434] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:03 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1513 bytes in 1004 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:04.929548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:04.930890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:04.954350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:04.955201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:04.956760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:04.957902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:04.960276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:04.961918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:05.798899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:05.809268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35900e9e-5c53-427a-9961-a9aa46bd23aa tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1497 microversion: 2.1 time: 0.884035 Oct 15 21:10:05.816146 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3321/4435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:04 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1497 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:05.843052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"rescue": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:05.846287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:05.862496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:05.863457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:05.864781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:05.867505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:05.868271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:05.869289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:05.927651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lazy-loading 'flavor' on Instance uuid b4ab84bd-5610-44e0-86cf-2877eac80927 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:10:05.952934 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Cannot 'rescue' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state paused Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action HTTP/1.1 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 14 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: {"rescue": {}} Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'rescue' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state paused Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b4ab84bd-5610-44e0-86cf-2877eac80927', 'body': {'rescue': {}}} Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 62, in _rescue Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.rescue(context, instance, Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 in vm_state paused. Cannot rescue while the instance is in this state. Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 72, in _rescue Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state( Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'rescue' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state paused Oct 15 21:10:05.960258 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'rescue' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state paused Oct 15 21:10:05.966347 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:05.967890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 409 to user: Cannot 'rescue' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state paused {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:05.969085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb1b6a95-3451-4c85-a3ee-1ac18ba9bb9d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action" status: 409 len: 144 microversion: 2.1 time: 0.130120 Oct 15 21:10:05.970159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3322/4436] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:05 2022] POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action => generated 144 bytes in 132 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:10:05.980565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:05.981438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:05.998087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:05.998775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:05.999437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.000727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.001231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.001663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.396134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:06.404474 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36d9918c-9403-4004-b163-b1c1544cc938 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1490 microversion: 2.1 time: 0.431908 Oct 15 21:10:06.417955 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3323/4437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:05 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1490 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:06.430805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:06.431442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:06.439064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.440063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.440777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.441604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.442138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.442568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.543011 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c977317a-a56a-4c8a-b2ba-825e983cb3f3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action" status: 202 len: 0 microversion: 2.1 time: 0.122992 Oct 15 21:10:06.545087 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3324/4438] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:06 2022] POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:06.554441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 3}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:06.557108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:06.575315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.576760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.577522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.578644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.581292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.582159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.682046 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a0aa9da-111a-45ba-b9e0-bb366bbd0aba tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 200 len: 142 microversion: 2.1 time: 0.131534 Oct 15 21:10:06.682706 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3325/4439] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:06 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 142 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:06.689786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:06.691174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:06.700184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.701086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.701753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:06.707704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:06.708410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:06.709014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:07.288734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:07.305408 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d471ab56-e04f-4be2-af0c-5e3ceed1461c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1504 microversion: 2.1 time: 0.620900 Oct 15 21:10:07.306246 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3326/4440] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:06 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1504 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:07.315512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:07.316181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:07.344849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:07.345394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:07.345832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:07.346542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:07.347156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:07.347953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:07.685143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:07.693462 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-029f5b2d-b7be-4a0c-9f83-f26514c37b05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.386073 Oct 15 21:10:07.711117 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3327/4441] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:07.729653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:07.730228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:07.737026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:07.737797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:07.738394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:07.739505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:07.740147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:07.740732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:08.191391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:08.205265 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-caca3082-049d-4266-bad2-e825e2ce4dd4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.479999 Oct 15 21:10:08.206853 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3328/4442] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:08.230365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:08.231366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:08.240190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:08.240860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:08.241488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:08.242542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:08.243178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:08.244045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:08.587862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:08.599850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2612dbcb-5577-4d54-b4cb-8b5e81ef3320 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.374371 Oct 15 21:10:08.601275 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3329/4443] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:08 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:08.609729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:08.610474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:08.630987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:08.634528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:08.640956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:08.643986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:08.646620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:08.647298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:08.678866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:09.049721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:09.081094 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b00bded-bf18-48a8-a313-5012535149ab tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1512 microversion: 2.1 time: 0.473435 Oct 15 21:10:09.082962 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3330/4444] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:08 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1512 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:09.097919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:09.098846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:09.116508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:09.117354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:09.118839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:09.124632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:09.125425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:09.126314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:09.525694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:09.539720 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa3ee302-6515-46f2-a18e-7395efe632b6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.452843 Oct 15 21:10:09.543685 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3331/4445] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:09 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:09.994388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-38090748", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:10.099923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] validate_networks() for [('c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:10:11.635126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:10:11.635736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:10:11.636082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:10:11.636476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:10:11.636893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:10:11.655219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:11.655734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:11.656186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:11.658074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:11.660282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:11.661101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:11.704830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:10:11.716205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for user 1aadd87b97c247cb95321664ba0c32a9 and project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:10:11.746253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:10:11.750390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:10:11.847343 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13f6e202-2410-4f36-9555-4b125fa34804 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.857294 Oct 15 21:10:11.848883 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3332/4446] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 1860 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:10:11.856842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:11.866026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:11.877735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:11.878966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:11.879779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:11.880767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:11.882229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:11.883106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:11.917948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lazy-loading 'flavor' on Instance uuid b4ab84bd-5610-44e0-86cf-2877eac80927 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:10:12.023998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecaa037c-856e-40ea-a07c-2d7449270438 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action" status: 200 len: 32 microversion: 2.1 time: 0.173520 Oct 15 21:10:12.027943 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3333/4447] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:11 2022] POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action => generated 32 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:12.043974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b8ba334-ed23-4607-835b-a0edf83a9e33 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:12.047093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b8ba334-ed23-4607-835b-a0edf83a9e33 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:12.255566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b8ba334-ed23-4607-835b-a0edf83a9e33 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:12.265425 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b8ba334-ed23-4607-835b-a0edf83a9e33 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1213 microversion: 2.1 time: 0.235563 Oct 15 21:10:12.266018 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3334/4448] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:12 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1213 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:12.292069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:12.292750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:12.318708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:12.319223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:12.319661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:12.320388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:12.320855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:12.333804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:12.740860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:12.754953 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f6a2f30-d2b0-4807-9d47-05fb1fe22af5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.481057 Oct 15 21:10:12.758412 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3335/4449] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:12 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:13.291120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:13.291966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:13.308285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:13.309743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:13.312098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:13.314804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:13.317978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:13.320595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:13.776014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:13.820136 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1551ec67-88f3-448d-a3d9-fe820097bb5c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1265 microversion: 2.1 time: 0.532353 Oct 15 21:10:13.823309 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3336/4450] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:13 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1265 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:13.846641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:13.852826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:13.878627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:13.881220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:13.882511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:13.884914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:13.886067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:13.886748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:14.227605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:14.240902 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97de38e4-b6fb-4408-bf42-4fc9fd3fb7e6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.416025 Oct 15 21:10:14.257159 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3337/4451] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:13 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:14.848240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:14.848938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:14.862436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:14.863284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:14.873213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:14.874209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:14.874843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:14.875294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:15.080269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:15.090989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22689756-16aa-49e0-a6bd-5aab978f4e89 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1271 microversion: 2.1 time: 0.246415 Oct 15 21:10:15.101727 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3338/4452] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:14 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1271 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:15.268128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:15.268924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:15.299064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:15.300009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:15.301952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:15.304892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:15.309762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:15.310227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:15.861207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:15.877022 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18d9f559-70d0-4759-8811-cd912e3c5462 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.613181 Oct 15 21:10:15.889150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3339/4453] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:15 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:16.126129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:16.127021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:16.137687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:16.138604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:16.142820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:16.143805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:16.144448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:16.145260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:16.526152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:16.553640 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-053a241e-44d4-4737-b4d0-a9585611dc5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 0.421839 Oct 15 21:10:16.556997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3340/4454] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:16 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:16.907399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:16.908735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:16.936037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:16.938127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:16.938764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:16.943433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:16.945120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:16.949080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:18.041896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:18.053896 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c15fbb99-1e70-45f6-8e79-6797092b29f5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 1.155238 Oct 15 21:10:18.059181 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3341/4455] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:16 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 1161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:18.068431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:18.073118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:18.084663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:18.085552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:18.086403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:18.088102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:18.088954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:18.089661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:18.904716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:18.919460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b0708e4-39eb-4f77-a265-df54afa3a6f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 0.859022 Oct 15 21:10:18.923641 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3342/4456] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:18 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:19.084264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:19.086827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:19.099888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:19.113338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:19.117688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:19.121379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:19.122586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:19.124714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:19.727780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:19.743834 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ca1d3de-8c26-4675-b756-7acbbee74027 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.663773 Oct 15 21:10:19.744674 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3343/4457] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:19 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:19.756898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6d2adde0-f062-4015-8ef7-1cbc094ad52d req-5ef6a7e1-0e2f-484c-8378-079d5dd39399 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "854dff46-3f20-4740-aef1-35486a5f9738", "tag": "ed98b007-0df9-43dd-9145-95a58eb49e8e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:19.783496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d2adde0-f062-4015-8ef7-1cbc094ad52d req-5ef6a7e1-0e2f-484c-8378-079d5dd39399 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:19.784322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d2adde0-f062-4015-8ef7-1cbc094ad52d req-5ef6a7e1-0e2f-484c-8378-079d5dd39399 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:19.784986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6d2adde0-f062-4015-8ef7-1cbc094ad52d req-5ef6a7e1-0e2f-484c-8378-079d5dd39399 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:19.809562 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6d2adde0-f062-4015-8ef7-1cbc094ad52d req-5ef6a7e1-0e2f-484c-8378-079d5dd39399 service nova] Creating event network-changed:ed98b007-0df9-43dd-9145-95a58eb49e8e for instance 854dff46-3f20-4740-aef1-35486a5f9738 on nch1-master-1 Oct 15 21:10:19.825578 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6d2adde0-f062-4015-8ef7-1cbc094ad52d req-5ef6a7e1-0e2f-484c-8378-079d5dd39399 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077883 Oct 15 21:10:19.828529 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3344/4458] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:10:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:19.957219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:19.960032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:19.967599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:19.968436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:19.969177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:19.970595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:19.973064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:19.973753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:20.810162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:20.818881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d9fbe16-e199-4273-9c63-e9feb433338a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 0.870676 Oct 15 21:10:20.819477 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3345/4459] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:19 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:20.849154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:20.851279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:20.872284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:20.872806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:20.873261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:20.874336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:20.875403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:20.876009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:21.587676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:21.596950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-170d365c-472e-40b1-8d6d-838ce34acc6a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.762557 Oct 15 21:10:21.607194 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3346/4460] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:20 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:21.843940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:21.846073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:21.850826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:21.866902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:21.869281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:21.870869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:21.871829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:21.873977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:21.875902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:21.933431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:22.261548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:22.342508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:22.360173 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1adf99b-de43-4393-b117-7e6da8b03f26 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 0.521053 Oct 15 21:10:22.362680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3347/4461] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:21 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:22.623413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:22.624052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:22.635111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:22.635623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:22.636075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:22.636832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:22.637326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:22.641426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:24.095991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:24.114653 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be65004a-0b94-4a4e-a923-041ee3de88ac tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 1.494914 Oct 15 21:10:24.131856 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3348/4462] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:22 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 1514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:24.150712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:24.153413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:24.182221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:24.182993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:24.183796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:24.185631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:24.186935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:24.187627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:25.176953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:25.210731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b8ee7d0-3478-489f-8140-9b0d5dec0c2d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 1.072996 Oct 15 21:10:25.211764 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3349/4463] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:24 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 1077 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:25.223686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:25.228049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:25.249548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:25.250087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:25.250589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:25.251320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:25.251799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:25.252245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:25.859859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:25.872266 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-493fe05d-db89-4731-8198-b76ed86e0bda tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.659363 Oct 15 21:10:25.872864 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3350/4464] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:25 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:26.387874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55e09aa0-a028-417a-8545-cf07ef582d61 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Action: 'create', calling method: .version_select at 0x7fe74e0f0430>, body: {"server_group": {"name": "server-group", "policy": "affinity"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:26.416245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-55e09aa0-a028-417a-8545-cf07ef582d61 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Getting quotas for project cd6d218d2e75459eae3548b0a9ba3000. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:10:26.423239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-55e09aa0-a028-417a-8545-cf07ef582d61 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Getting quotas for user 3d2974383cae4b5485ce72b2e24a8dc6 and project cd6d218d2e75459eae3548b0a9ba3000. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:10:26.481282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-55e09aa0-a028-417a-8545-cf07ef582d61 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Getting quotas for project cd6d218d2e75459eae3548b0a9ba3000. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:10:26.484234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:26.489315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-55e09aa0-a028-417a-8545-cf07ef582d61 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Getting quotas for user 3d2974383cae4b5485ce72b2e24a8dc6 and project cd6d218d2e75459eae3548b0a9ba3000. Resources: {'server_groups'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:10:26.513598 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55e09aa0-a028-417a-8545-cf07ef582d61 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 235 microversion: 2.64 time: 0.638701 Oct 15 21:10:26.514707 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3351/4465] 10.222.0.16 () {62 vars in 1214 bytes} [Sat Oct 15 21:10:25 2022] POST /compute/v2.1/os-server-groups => generated 235 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:10:26.522755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:26.524325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:26.531381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:26.532198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:26.532965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:26.533906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:26.534567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:26.535149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:26.758236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:26.771859 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-11d1c171-e500-4008-ae48-d41df2361faf tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 0.255693 Oct 15 21:10:26.772476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3352/4466] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:26 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:26.777502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-64736489-b111-4991-ac4d-1aa1b8bcbe07 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:26.799314 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-64736489-b111-4991-ac4d-1aa1b8bcbe07 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 238 microversion: 2.64 time: 0.025635 Oct 15 21:10:26.799904 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3353/4467] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Oct 15 21:10:26 2022] GET /compute/v2.1/os-server-groups => generated 238 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:10:26.814452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-291bedc5-efba-4db5-a3a5-20c59ceb4658 req-d43b49a5-60f3-471c-b49c-3c476fc3b2e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "854dff46-3f20-4740-aef1-35486a5f9738", "name": "network-vif-plugged", "status": "completed", "tag": "ed98b007-0df9-43dd-9145-95a58eb49e8e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:26.827713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-291bedc5-efba-4db5-a3a5-20c59ceb4658 req-d43b49a5-60f3-471c-b49c-3c476fc3b2e7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:26.828383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-291bedc5-efba-4db5-a3a5-20c59ceb4658 req-d43b49a5-60f3-471c-b49c-3c476fc3b2e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:26.828828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-291bedc5-efba-4db5-a3a5-20c59ceb4658 req-d43b49a5-60f3-471c-b49c-3c476fc3b2e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:26.843658 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-291bedc5-efba-4db5-a3a5-20c59ceb4658 req-d43b49a5-60f3-471c-b49c-3c476fc3b2e7 service nova] Creating event network-vif-plugged:ed98b007-0df9-43dd-9145-95a58eb49e8e for instance 854dff46-3f20-4740-aef1-35486a5f9738 on nch1-master-1 Oct 15 21:10:26.849342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-291bedc5-efba-4db5-a3a5-20c59ceb4658 req-d43b49a5-60f3-471c-b49c-3c476fc3b2e7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045441 Oct 15 21:10:26.850578 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3354/4468] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:10:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:26.894351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff8f19c7-f868-4f5a-9e60-f95511d7dc5b tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:26.916457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ff8f19c7-f868-4f5a-9e60-f95511d7dc5b tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07" status: 200 len: 235 microversion: 2.64 time: 0.062885 Oct 15 21:10:26.917022 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3355/4469] 10.222.0.16 () {60 vars in 1304 bytes} [Sat Oct 15 21:10:26 2022] GET /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 => generated 235 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:10:26.924793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:26.925587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:26.937283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:26.938131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:26.938844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:26.939821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:26.940525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:26.941220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:27.168020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:27.178225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5b0fadf-12b3-428b-b16b-5756791e0d7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.258516 Oct 15 21:10:27.180850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3356/4470] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:26 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:27.185594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14ede87a-b673-4ed3-8fe0-62b7689da001 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:27.250301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14ede87a-b673-4ed3-8fe0-62b7689da001 tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07" status: 204 len: 0 microversion: 2.64 time: 0.068286 Oct 15 21:10:27.251547 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3357/4471] 10.222.0.16 () {60 vars in 1307 bytes} [Sat Oct 15 21:10:27 2022] DELETE /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:10:27.262508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-195109b0-be93-4045-97a3-8ed7e9c2333e tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:27.274579 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-195109b0-be93-4045-97a3-8ed7e9c2333e tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] HTTP exception thrown: Instance group 6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 could not be found. Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-195109b0-be93-4045-97a3-8ed7e9c2333e tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Request method failure captured: Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 HTTP/1.1 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyHBPbFDMKBoumvh7j24pgMRt4tOS6wSLTxjDEGN8F75sA_rWlPRVZHngwTOS0kPpI9TW___Uqw9DdB9HPbh1a7CNLm30g42wK_-3DBc6VQ-F3MpiGVa1BMdgdHviVuSg-12O8YH3TqjYD1V26mVJ7cE6mPM1N1xlSFOZPdv33WUaME Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Openstack-Nova-Api-Version: 2.64 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cd6d218d2e75459eae3548b0a9ba3000 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerGroup264TestJSON-441683593 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd6d218d2e75459eae3548b0a9ba3000"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cd6d218d2e75459eae3548b0a9ba3000"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd6d218d2e75459eae3548b0a9ba3000"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cd6d218d2e75459eae3548b0a9ba3000"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerGroup264TestJSON-441683593 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cd6d218d2e75459eae3548b0a9ba3000 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerGroup264TestJSON-441683593 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerGroup264TestJSON-441683593-project Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 3d2974383cae4b5485ce72b2e24a8dc6 Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerGroup264TestJSON-441683593-project Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance group 6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 could not be found. Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6a3ff92b-c9ce-4a10-84d5-eb08ba189f07'} Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 could not be found. Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 could not be found. Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:27.276073 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:27.280728 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:27.280728 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:27.280728 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:27.280728 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:27.280728 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 could not be found. Oct 15 21:10:27.280728 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:27.282163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-195109b0-be93-4045-97a3-8ed7e9c2333e tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] Returning 404 to user: Instance group 6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:27.283263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-195109b0-be93-4045-97a3-8ed7e9c2333e tempest-ServerGroup264TestJSON-441683593 tempest-ServerGroup264TestJSON-441683593-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07" status: 404 len: 117 microversion: 2.64 time: 0.025223 Oct 15 21:10:27.283886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3358/4472] 10.222.0.16 () {60 vars in 1307 bytes} [Sat Oct 15 21:10:27 2022] DELETE /compute/v2.1/os-server-groups/6a3ff92b-c9ce-4a10-84d5-eb08ba189f07 => generated 117 bytes in 26 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 15 21:10:27.794697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:27.795304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:27.804371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:27.807050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:27.807798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:27.808691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:27.809797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:27.810509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:28.133779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:28.145316 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b9cfaff-0d5f-4e33-a486-68f4b7f0e782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1313 microversion: 2.1 time: 0.354820 Oct 15 21:10:28.157064 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3359/4473] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:27 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1313 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:28.206447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:28.206447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:28.220437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:28.220933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:28.227566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:28.228965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:28.232725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:28.233936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:28.705101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:28.720999 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ea81546-9cbf-40a8-a7af-f413718638ee tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1498 microversion: 2.1 time: 0.520718 Oct 15 21:10:28.721944 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3360/4474] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:28 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1498 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:28.748906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e9363965-eca8-4164-8d38-e82d50fa1ef4"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:28.751625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:28.761193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:28.761676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:28.762156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:28.762892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:28.763355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:28.763784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:28.837773 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Cannot 'attach_volume' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state rescued Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/os-volume_attachments HTTP/1.1 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 74 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: {"volumeAttachment": {"volumeId": "e9363965-eca8-4164-8d38-e82d50fa1ef4"}} Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: exception: Cannot 'attach_volume' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state rescued Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': 'b4ab84bd-5610-44e0-86cf-2877eac80927', 'body': {'volumeAttachment': {'volumeId': 'e9363965-eca8-4164-8d38-e82d50fa1ef4'}}} Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 15 21:10:28.841417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state rescued Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state rescued Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:28.845424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 409 to user: Cannot 'attach_volume' instance b4ab84bd-5610-44e0-86cf-2877eac80927 while it is in vm_state rescued {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:28.849141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-873c59d5-6bcb-4872-b329-6cc46cc6aa85 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.104357 Oct 15 21:10:28.850629 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3361/4475] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:10:28 2022] POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/os-volume_attachments => generated 152 bytes in 107 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 15 21:10:28.866280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:28.867041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:28.881146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:28.888588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:28.889706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:28.890682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:28.891333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:28.892038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:29.016639 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52bc9491-dffd-45d4-87b6-e15e6c755798 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action" status: 202 len: 0 microversion: 2.1 time: 0.159544 Oct 15 21:10:29.018032 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3362/4476] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:28 2022] POST /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927/action => generated 0 bytes in 162 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:29.050725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:29.053950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:29.063307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:29.064258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:29.064960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:29.066330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:29.067127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:29.067761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:29.575785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:29.599969 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b130d4bd-adc2-444c-8865-e6e9975399ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1506 microversion: 2.1 time: 0.563690 Oct 15 21:10:29.604086 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3363/4477] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:29 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1506 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:29.616836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:29.620293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:29.639284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:29.640558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:29.641218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:29.643230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:29.644781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:29.645419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:30.000236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:30.016959 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16a089db-ea65-48e0-b58e-a2ed3ef52c36 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1500 microversion: 2.1 time: 0.410796 Oct 15 21:10:30.017944 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3364/4478] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:29 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1500 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:30.043449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:30.053203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:30.060740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:30.061305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:30.061737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:30.062463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:30.063013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:30.063452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:30.172050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95f8bd1e-5234-48cf-bb06-deb3690af3c4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.132714 Oct 15 21:10:30.172932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3365/4479] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:30 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 21:10:30.634665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:30.636065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:30.659847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:30.670214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:30.672182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:30.673815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:30.674637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:30.676765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:31.153579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:31.165158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36378b88-4243-4bb0-b171-b4bfcc7185dc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1506 microversion: 2.1 time: 0.540409 Oct 15 21:10:31.177906 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3366/4480] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:30 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1506 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:31.185950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:31.187504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:31.201064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:31.202778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:31.213809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:31.213809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:31.213809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:31.213809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:31.676410 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aad6ce89-64ef-4241-b680-45a1cf684334 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.496384 Oct 15 21:10:31.677100 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3367/4481] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:31 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 21:10:31.789617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG neutronclient.v2_0.client [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Error message: {"error": {"code": 401, "title": "Unauthorized", "message": "The request you have made requires authentication."}} {{(pid=138689) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 15 21:10:31.790524 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 500 len: 0 microversion: 2.25 time: 902.564556 Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Caught error: Not authorized.: nova.exception.Unauthorized: Not authorized. Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack Traceback (most recent call last): Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack exception_handler_v20(status_code, error_body) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack raise client_exc(message=error_message, Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack neutronclient.common.exceptions.Unauthorized: 401-{'error': {'code': 401, 'title': 'Unauthorized', 'message': 'The request you have made requires authentication.'}} Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack Neutron server returns request_ids: ['req-ce2129c4-aca3-4f00-b832-7c7a417d5165'] Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack During handling of the above exception, another exception occurred: Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack Traceback (most recent call last): Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/__init__.py", line 95, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return req.get_response(self.application) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 104, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack self._log_req(req, res, start) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack self.force_reraise() Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack raise self.value Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 99, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack res = req.get_response(self.application) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return resp(environ, start_response) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/osprofiler/web.py", line 111, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return request.get_response(self.application) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack response = req.get_response(self._app) Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 15 21:10:31.848200 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return resp(environ, start_response) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return resp(environ, start_response) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/routes/middleware.py", line 153, in __call__ Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack response = self.app(environ, start_response) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return resp(environ, start_response) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 486, in __call__ Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self._process_stack(request, action, action_args, Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return method(req=request, **action_args) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return f(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack [Previous line repeated 10 more times] Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack instances, resv_id = self.compute_api.create( Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self._create_instance( Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ) = self._validate_and_build_base_options( Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1080, in _validate_and_build_base_options Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack max_network_count = self._check_requested_networks( Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 542, in _check_requested_networks Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.network_api.validate_networks(context, requested_networks, Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 2626, in validate_networks Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ports_needed_per_instance = self._ports_needed_per_instance( Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 2548, in _ports_needed_per_instance Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack nets = self._get_available_networks( Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 454, in _get_available_networks Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack nets = neutron.list_networks(**search_opts).get('networks', []) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 860, in list_networks Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.list('networks', self.networks_path, retrieve_all, Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 372, in list Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack for r in self._pagination(collection, path, **params): Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 387, in _pagination Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack res = self.get(path, params=params) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.retry_request("GET", action, body=body, Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:10:31.852221 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack return self.do_request(method, action, body=body, Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack self._handle_fault_response(status_code, replybody, resp) Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 204, in wrapper Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack raise exception.Unauthorized() Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack nova.exception.Unauthorized: Not authorized. Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] http://10.222.0.16/compute/v2.1/servers returned with HTTP 401 Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Returning 401 to user: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:31.856525 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:10:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:10:31.858755 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:10:31 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.16) Oct 15 21:10:31.858755 ncdv-master-1 devstack@n-api.service[138689]: CRITICAL nova [None req-464e8776-3e66-440f-b279-5b085cedc2ba tempest-LiveAutoBlockMigrationV225Test-778882528 tempest-LiveAutoBlockMigrationV225Test-778882528-project] Unhandled error: OSError: write error Oct 15 21:10:31.858755 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova OSError: write error Oct 15 21:10:31.858755 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova Oct 15 21:10:31.858755 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1115/4482] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 20:55:29 2022] POST /compute/v2.1/servers => generated 0 bytes in 902633 msecs (HTTP/1.1 401) 9 headers in 0 bytes (1 switches on core 0) Oct 15 21:10:32.194151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:32.204860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:32.232942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:32.233484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:32.233922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:32.245808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:32.246325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:32.246834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:32.501457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:32.540326 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-549a72b3-797b-4532-a6a0-c292207dd65c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1512 microversion: 2.1 time: 0.340846 Oct 15 21:10:32.561015 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3368/4483] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:32 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1512 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:32.690105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:32.692726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:32.707126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:32.707555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:32.708111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:32.708708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:32.709194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:32.711497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:32.781399 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b2630fe-1cad-4a34-9ec8-2242a9557326 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.096676 Oct 15 21:10:32.787296 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1116/4484] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:32 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:33.794621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:33.797314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:33.807529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:33.808177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:33.808623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:33.809351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:33.810106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:33.810691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:33.852393 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6bc199ac-8232-4151-9e26-6fef77110d7c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.061668 Oct 15 21:10:33.853758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3369/4485] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:33 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:34.132081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce303450-0ec7-4a1a-bf60-13eaa1a312c2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'create', calling method: .version_select at 0x7fe74f063ee0>, body: {"keypair": {"name": "tempest-keypair-1395606911", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAapbRcxstehZ/fFQJbwe4CewKpvkaiKp/p8MZIZACT5vAnw6oUJ3DfQEaZQEFNHGB7Kfx5HHnHpug1APP8Unt6oxklVclaBYqU9DQ5qMrKdY5K32GLSQYqX9vZ4uuBraw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:34.173637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-ce303450-0ec7-4a1a-bf60-13eaa1a312c2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for project e2b1936403344a4994393f2b098327a0. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:10:34.182086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-ce303450-0ec7-4a1a-bf60-13eaa1a312c2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for user c48346e56c47481cab7f248b67369a27 and project e2b1936403344a4994393f2b098327a0. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:10:34.234227 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce303450-0ec7-4a1a-bf60-13eaa1a312c2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.107048 Oct 15 21:10:34.235592 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1117/4486] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:10:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:34.867046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:34.870252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:34.878993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:34.879755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:34.880439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:34.881701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:34.882235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:34.883480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:34.888601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:34.943451 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a66a3616-48ae-44a8-89f3-e5cc2f5864dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.082180 Oct 15 21:10:34.947794 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3370/4487] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:34 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:35.950007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-713216600", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "855fc75b-c131-45d1-adfc-303e1e3ad3a6"}], "security_groups": [{"name": "tempest-securitygroup--235182984"}], "key_name": "tempest-keypair-1395606911", "user_data": "}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:35.956503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:35.959437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:35.971007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:35.972308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:35.972949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:35.973786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:35.975166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:35.976371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:36.025630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad08bc6e-c29b-4599-b31b-99e2acf8399e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.073807 Oct 15 21:10:36.029371 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3371/4488] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:35 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:36.094202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] validate_networks() for [('855fc75b-c131-45d1-adfc-303e1e3ad3a6', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:10:36.933754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:10:36.934486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:10:36.934964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:10:36.935374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:10:36.935783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:10:36.946384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:36.946980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:36.947423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:36.948156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:36.952872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:36.953502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:36.977607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for project e2b1936403344a4994393f2b098327a0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:10:36.984023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Getting quotas for user c48346e56c47481cab7f248b67369a27 and project e2b1936403344a4994393f2b098327a0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:10:36.997555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:10:37.000026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:10:37.038084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:37.041180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:37.047575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:37.050076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:37.050900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:37.051636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:37.052422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:37.053111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:37.053762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:37.060105 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d91754ed-98f6-4d0b-aa8e-487bd2ce8da2 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 1.113102 Oct 15 21:10:37.060105 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1118/4489] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:35 2022] POST /compute/v2.1/servers => generated 408 bytes in 1115 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:10:37.071090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-21ad9f95-0209-42f7-9bcc-de01f356607a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:37.072375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-21ad9f95-0209-42f7-9bcc-de01f356607a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:37.136245 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f54b5154-babf-450e-84a5-abc3cfcaca81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.102422 Oct 15 21:10:37.137070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3372/4490] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:37 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:37.141707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-21ad9f95-0209-42f7-9bcc-de01f356607a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:37.151888 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-21ad9f95-0209-42f7-9bcc-de01f356607a tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1245 microversion: 2.1 time: 0.086389 Oct 15 21:10:37.165816 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1119/4491] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:37 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1245 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:38.150656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:38.154268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:38.164649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:38.165655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:38.166556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:38.167552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:38.168572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:38.169271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:38.177433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:38.179011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:38.197424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:38.198364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:38.199047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:38.200395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:38.201300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:38.202268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:38.332843 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b44b074a-0eec-404c-bfef-41904097048d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.187211 Oct 15 21:10:38.335061 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3373/4492] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:38 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 21:10:38.539336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:38.565637 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c8721b1a-3d64-44d5-b9a4-6db685137847 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1297 microversion: 2.1 time: 0.377008 Oct 15 21:10:38.568413 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1120/4493] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:38 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1297 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:39.344445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:39.347198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:39.373591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:39.375628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:39.378880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:39.382101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:39.383242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:39.387681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:39.473744 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c330a89-7fb6-4fb1-8b36-fbb389911dda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 14 microversion: 2.1 time: 0.133076 Oct 15 21:10:39.478867 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3374/4494] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:39 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 14 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:10:39.602411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:39.603592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:39.618583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:39.620825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:39.621582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:39.623415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:39.625602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:39.626506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:39.975774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:39.995573 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5e2ddb17-4356-4778-b49a-6799f3771dd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1303 microversion: 2.1 time: 0.401050 Oct 15 21:10:40.006776 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1121/4495] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:39 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1303 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:41.028506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:41.029180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:41.038981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:41.042695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:41.044230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:41.045280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:41.050121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:41.051048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:41.098879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:41.105540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:41.129005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:41.130559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:41.131559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:41.134546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:41.136116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:41.138473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:41.270100 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f265640-ac37-40f2-bbb9-c841b253c99c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action" status: 200 len: 8203 microversion: 2.1 time: 0.787567 Oct 15 21:10:41.272172 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3375/4496] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:40 2022] POST /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738/action => generated 8203 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:41.289855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:41.292344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:41.320530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:41.321112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:41.321910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:41.323243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:41.324770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:41.325470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:41.722470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:41.730903 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e4138881-f459-42b0-b961-9d0aa5989285 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 0.706038 Oct 15 21:10:41.760990 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1122/4497] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:41 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:42.530501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:42.550722 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3168fad8-4e21-4072-9230-66a9a32c723a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.271546 Oct 15 21:10:42.555893 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3376/4498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:41 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:42.602935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:42.604129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:42.618669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:42.619992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:42.624089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:42.625061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:42.625877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:42.626471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:42.758987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:42.759646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:42.771011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:42.771516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:42.771961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:42.772766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:42.785210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:42.785820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:43.076895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:43.085775 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-23241814-6dfb-4d7b-b4fa-85d9e0fb8e5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 0.331372 Oct 15 21:10:43.090136 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3377/4499] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:42 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:43.136817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:43.162834 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-299611c2-7917-4ff1-83ef-7d0c75e1a0ee tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.558382 Oct 15 21:10:43.169282 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1123/4500] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:42 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:43.187078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:43.187961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:43.202516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:43.204503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:43.206294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:43.207534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:43.209536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:43.210849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:43.221104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b22d09ff-ccdf-4003-a631-c549fbac6dd6 req-b9fbc194-300a-4f34-8ff7-c6e0a3e78513 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8cbc03dc-b1bc-422b-8a26-c1375e5a3c72", "tag": "b013b704-2ff3-4eb1-a130-54222bd18fef"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:43.242419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b22d09ff-ccdf-4003-a631-c549fbac6dd6 req-b9fbc194-300a-4f34-8ff7-c6e0a3e78513 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:43.244197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b22d09ff-ccdf-4003-a631-c549fbac6dd6 req-b9fbc194-300a-4f34-8ff7-c6e0a3e78513 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:43.246015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b22d09ff-ccdf-4003-a631-c549fbac6dd6 req-b9fbc194-300a-4f34-8ff7-c6e0a3e78513 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:43.266674 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b22d09ff-ccdf-4003-a631-c549fbac6dd6 req-b9fbc194-300a-4f34-8ff7-c6e0a3e78513 service nova] Creating event network-changed:b013b704-2ff3-4eb1-a130-54222bd18fef for instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 on nch2-master-1 Oct 15 21:10:43.287823 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b22d09ff-ccdf-4003-a631-c549fbac6dd6 req-b9fbc194-300a-4f34-8ff7-c6e0a3e78513 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072414 Oct 15 21:10:43.289437 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1124/4501] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:10:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:43.902610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:43.926442 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8981d7bf-c463-4c97-b915-5b4b619216a4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.740014 Oct 15 21:10:43.928820 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3378/4502] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:43 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:43.950465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:43.951947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:43.959740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:43.961302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:43.962076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:43.962959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:43.963578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:43.964161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:44.121690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:44.123489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:44.139462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:44.142139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:44.143546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:44.145680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:44.146183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:44.148583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:44.945220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:44.957110 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bdff19ef-9777-4618-9436-4d12ce2aa01a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.008738 Oct 15 21:10:44.958834 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1125/4503] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:43 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1014 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:44.988265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"lock": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:44.989614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:44.996943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:44.998093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:44.998752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:44.999998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:45.000710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:45.001381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:45.063755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Locking {{(pid=138689) lock /opt/stack/nova/nova/compute/api.py:4845}} Oct 15 21:10:45.162413 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c2288f97-0a08-4b7a-8898-8d4fcc243ac2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.177149 Oct 15 21:10:45.164015 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1126/4504] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:44 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:45.185870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:45.186887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:45.196182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:45.197063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:45.197747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:45.199780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:45.200513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:45.201393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:45.255844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:45.267858 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9640dfbf-de1a-46cc-bf3e-207a036d73b6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 1.153047 Oct 15 21:10:45.270632 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3379/4505] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:44 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 1157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:45.655722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:45.665674 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d8b32be-1869-4398-9713-a9f3f5e64158 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.497286 Oct 15 21:10:45.668685 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1127/4506] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:45 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:45.702965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:45.727706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:45.730855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:45.731726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:45.849183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] HTTP exception thrown: Instance 9585dc89-1627-4fc5-9817-abee53538924 is locked Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Request method failure captured: Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action HTTP/1.1 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 15 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCkMhMzgxIb6i7VoC650pGF8tykO4OjlOTVaW1cx9EsLvnp80UEJNdjmkJ9gxWxEKdx-BZyz41tnpRESXgt8mdrfybGIjbfLbEf7abh0rD68HzJxm6T7JWNMVWGCBvXV4emyeRKn3nLIRCE_kdy1202kOCKfFOyewmw-b1yDpMy11U Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-995038554 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1aadd87b97c247cb95321664ba0c32a9 Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: {"os-stop": {}} Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 9585dc89-1627-4fc5-9817-abee53538924 is locked Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '9585dc89-1627-4fc5-9817-abee53538924', 'body': {'os-stop': {}}} Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1449, in _stop_server Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.stop(context, instance) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 240, in inner Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceIsLocked(instance_uuid=instance.uuid) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceIsLocked: Instance 9585dc89-1627-4fc5-9817-abee53538924 is locked Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1451, in _stop_server Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=e.format_message()) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Instance 9585dc89-1627-4fc5-9817-abee53538924 is locked Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9585dc89-1627-4fc5-9817-abee53538924 is locked Oct 15 21:10:45.851366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:10:45.859233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Returning 409 to user: Instance 9585dc89-1627-4fc5-9817-abee53538924 is locked {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:10:45.860410 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-125c166f-6988-46fc-902f-245da83ccd6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 409 len: 107 microversion: 2.1 time: 0.169233 Oct 15 21:10:45.861658 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3380/4507] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:45 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 107 bytes in 171 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:10:45.871195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"unlock": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:45.872646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:45.882869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:45.891032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:45.893147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:45.895014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:45.901486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:45.902048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:46.011607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Unlocking {{(pid=138689) unlock /opt/stack/nova/nova/compute/api.py:4874}} Oct 15 21:10:46.118060 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-14414bcb-af5f-4f8a-97d9-52b0f67d181e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.251613 Oct 15 21:10:46.124148 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1128/4508] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:45 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 259 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:46.133250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51c6e689-3752-468c-91c1-e7279efcbbfc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:46.146253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c6e689-3752-468c-91c1-e7279efcbbfc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:46.147535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c6e689-3752-468c-91c1-e7279efcbbfc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:46.148601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51c6e689-3752-468c-91c1-e7279efcbbfc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:46.219360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51c6e689-3752-468c-91c1-e7279efcbbfc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:10:46.302713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:46.305176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:46.321355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:46.322937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:46.327137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:46.331235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:46.332439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:46.333156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:46.333734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51c6e689-3752-468c-91c1-e7279efcbbfc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.206722 Oct 15 21:10:46.341345 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3381/4509] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:46 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 215 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:10:46.349724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:46.361248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:46.368869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:46.370678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:46.372316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:46.373987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:46.379104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:46.379856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:46.784331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:46.794756 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b16b2dc6-d3d4-4872-b3d9-e2becc422ad0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 0.500971 Oct 15 21:10:46.807306 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1129/4510] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:46 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:46.826742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:46.836642 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c463b8f-6585-4f76-abbf-8572a4547e4f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.492587 Oct 15 21:10:46.837265 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3382/4511] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:46 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:47.821188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:47.822642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:47.841473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:47.849325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:47.864671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:47.866005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:47.875612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:47.878565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:47.880193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:47.882990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:47.884811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:47.885469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:47.886533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:47.888553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:47.892652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:47.897074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:47.900957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:47.901762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:48.968152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:48.981709 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-379be48f-9d36-4cbe-bb1b-07754ee8c100 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 1.121765 Oct 15 21:10:48.982313 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3383/4512] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:47 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 1124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:49.981264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:50.006284 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4fdd9eb-d4c8-4782-86b3-c0d9f031b18c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 2.189896 Oct 15 21:10:50.012119 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1130/4513] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:47 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 2197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:50.019508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:50.020510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:50.048442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:50.048442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:50.048442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:50.048442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:50.049335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:50.060314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:50.686069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSONUnderV235-server-2066305419", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:50.816411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:50.821130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] validate_networks() for [('9f248ce0-36ef-45a7-b1ea-31b162ddb04e', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:10:50.827806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee0ce313-dd5a-4bfc-85a1-9d71f249fa6e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.822612 Oct 15 21:10:50.841837 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3384/4514] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:50 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:51.031176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:51.036275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:51.042130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:51.042635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:51.043087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:51.043808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:51.049548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:51.050259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:51.522338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:51.544561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab197b19-697c-42d5-838b-f23f1b39436f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 0.513958 Oct 15 21:10:51.545781 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3385/4515] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:51 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:51.856897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:51.857604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:51.870946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:51.876245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:51.876973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:51.877923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:51.879956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:51.880599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:52.075026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:10:52.075580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:10:52.075946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:10:52.076402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:10:52.077222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:10:52.095665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:52.096428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:52.097328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:52.098271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:52.098904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:52.099504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:52.122914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=88,nova_cell1:INSERT=4,nova_cell1:UPDATE=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:52.155552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Getting quotas for project 16eb29186a7b4964a49882162a35cf06. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:10:52.162378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Getting quotas for user a7eae9dc0a1a47d9910f49bb33316d91 and project 16eb29186a7b4964a49882162a35cf06. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:10:52.215840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:10:52.223506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:10:52.331154 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-091f2002-2063-4f3a-a639-2f5175461209 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.137341 Oct 15 21:10:52.334287 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1131/4516] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:10:50 2022] POST /compute/v2.1/servers => generated 383 bytes in 2141 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:10:52.344723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1eb2b848-45a8-40a6-9349-9e3bce97b570 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:52.347164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1eb2b848-45a8-40a6-9349-9e3bce97b570 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:52.559599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1eb2b848-45a8-40a6-9349-9e3bce97b570 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:52.568506 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1eb2b848-45a8-40a6-9349-9e3bce97b570 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1223 microversion: 2.1 time: 0.228367 Oct 15 21:10:52.571341 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1132/4517] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:52 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1223 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:52.604830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:52.605576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:52.634896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:52.635452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:52.638613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:52.639395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:52.639871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:52.640305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:52.767423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:52.777575 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-deb451c5-38b8-46ec-b3b5-dd8efbff8369 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.924969 Oct 15 21:10:52.783562 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3386/4518] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:51 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:53.096966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:53.107373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ad32cabe-648a-47ad-8414-96081c93b20c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 0.534735 Oct 15 21:10:53.109681 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1133/4519] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:52 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:53.594403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:53.595063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:53.608761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:53.618636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:53.620294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:53.629214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:53.629897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:53.630696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:53.772847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:53.786925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cfbcfe3f-e459-4bd1-9ed9-18cb673442ec tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1227 microversion: 2.1 time: 0.196297 Oct 15 21:10:53.789509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3387/4520] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:53 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1227 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:53.814016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:53.814687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:53.833804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:53.834350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:53.834825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:53.840973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:53.841543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:53.841986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:54.137243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:54.138070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:54.145667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:54.148647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:54.149427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:54.157466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:54.159607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:54.161674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:54.173267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:54.603343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:54.634009 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-af8a2225-56ea-464e-9165-331fbc50ad88 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.823202 Oct 15 21:10:54.651936 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1134/4521] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:53 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:54.812761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:54.814906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:54.840259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:54.847488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:54.848223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:54.849351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:54.850861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:54.851976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:55.278345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:55.294806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9faf8c4a-3339-4f57-a1fd-e99446231b69 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1275 microversion: 2.1 time: 0.485669 Oct 15 21:10:55.298901 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1135/4522] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:54 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1275 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:55.345464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:55.367358 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36401afa-0565-4a08-adc6-ee3eba2ae0e4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 1.231628 Oct 15 21:10:55.367358 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3388/4523] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:54 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 1235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:55.666832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:55.667467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:55.686947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:55.690606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:55.691141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:55.699885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:55.700507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:55.700961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:56.327445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:56.328233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:56.346800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:56.347531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:56.355982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:56.357213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:56.361530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:56.364437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:56.745802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:56.766478 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-54c6dcf3-8fd7-428c-8615-c868c01f035d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 1.105677 Oct 15 21:10:56.782110 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1136/4524] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:55 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 1111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:56.793369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:56.806790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:56.846686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:56.850101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:56.851309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:56.851995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:56.855720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:56.856372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:56.856996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:56.879958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78494259-4e5e-46d0-8958-9fbddab66b26 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1281 microversion: 2.1 time: 0.553927 Oct 15 21:10:56.898455 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3389/4525] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:56 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1281 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:56.916608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-01969c28-ae2c-48e1-b1e2-83c5c70676cb req-714734a9-2e48-47b7-9191-b9d911ed6591 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8cbc03dc-b1bc-422b-8a26-c1375e5a3c72", "name": "network-vif-plugged", "status": "completed", "tag": "b013b704-2ff3-4eb1-a130-54222bd18fef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:56.936724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=85,nova_api:DELETE=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:10:56.946402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-01969c28-ae2c-48e1-b1e2-83c5c70676cb req-714734a9-2e48-47b7-9191-b9d911ed6591 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:56.950877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-01969c28-ae2c-48e1-b1e2-83c5c70676cb req-714734a9-2e48-47b7-9191-b9d911ed6591 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:56.953413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-01969c28-ae2c-48e1-b1e2-83c5c70676cb req-714734a9-2e48-47b7-9191-b9d911ed6591 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:56.977119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-01969c28-ae2c-48e1-b1e2-83c5c70676cb req-714734a9-2e48-47b7-9191-b9d911ed6591 service nova] Creating event network-vif-plugged:b013b704-2ff3-4eb1-a130-54222bd18fef for instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 on nch2-master-1 Oct 15 21:10:56.991605 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-01969c28-ae2c-48e1-b1e2-83c5c70676cb req-714734a9-2e48-47b7-9191-b9d911ed6591 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089278 Oct 15 21:10:56.996747 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3390/4526] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:10:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:10:57.745932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:57.761572 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7891494d-b599-49b1-970c-0401b1edf429 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 0.988053 Oct 15 21:10:57.765449 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1137/4527] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:56 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:57.798409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:57.799243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:57.839602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:57.851751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:57.852551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:57.853734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:57.856664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:57.857540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:57.917645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:57.919414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:57.935852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:57.936940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:57.937325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:57.942267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:57.943779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:57.946685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:58.254107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:58.274766 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-55f2cb54-6621-4675-be24-5af6bfbebc2f tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1281 microversion: 2.1 time: 0.366582 Oct 15 21:10:58.280498 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1138/4528] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:57 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1281 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:58.383908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:58.399560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0112633a-5564-4c8a-ac15-ea0720ac7ee6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1658 microversion: 2.1 time: 0.603466 Oct 15 21:10:58.400233 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3391/4529] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:57 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1658 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:10:58.430590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-117f3944-d80a-4fa6-bde3-787afc55359f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:10:58.443332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-117f3944-d80a-4fa6-bde3-787afc55359f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:58.445506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-117f3944-d80a-4fa6-bde3-787afc55359f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:58.447136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-117f3944-d80a-4fa6-bde3-787afc55359f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:58.621920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-117f3944-d80a-4fa6-bde3-787afc55359f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Going to try to start instance {{(pid=138689) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 21:10:58.696267 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-117f3944-d80a-4fa6-bde3-787afc55359f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.271266 Oct 15 21:10:58.697428 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1139/4530] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:10:58 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 275 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:10:58.710712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:58.711231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:58.728715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:58.730306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:58.731331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:58.734394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:58.735681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:58.737217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:58.792160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:58.793677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:58.821755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:58.823786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:58.826631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:58.831622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:58.833857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:58.835213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:59.627677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:59.640976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5678d9f7-4322-4eb3-b5da-497b6424fe5d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1667 microversion: 2.1 time: 0.938543 Oct 15 21:10:59.646801 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3392/4531] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:58 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1667 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:10:59.660389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:10:59.661029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:10:59.669472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:59.669960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:59.670571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:59.672265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:10:59.672947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:10:59.673673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:10:59.904990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:10:59.915925 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f4e1fad4-2e52-4c19-a91d-14a8eaa351e5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1370 microversion: 2.1 time: 1.128392 Oct 15 21:10:59.935099 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1140/4532] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:58 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1370 bytes in 1148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:00.488865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:00.504987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-17f28d3c-cec1-41b3-a613-dd748287e25a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.856538 Oct 15 21:11:00.505781 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3393/4533] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:10:59 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 858 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:00.672059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:00.672993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:00.676280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=7 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:00.686605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:00.691938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:00.692830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:00.695728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:00.696471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:00.700018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:00.942506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:00.943759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:00.959761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:00.960634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:00.961299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:00.967591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:00.971678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:00.972625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:01.798409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:01.807890 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5805d562-b4b5-43be-804e-57dbd62aa357 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1667 microversion: 2.1 time: 1.139882 Oct 15 21:11:01.821925 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1141/4534] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:00 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1667 bytes in 1145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:01.828150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:01.831349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:01.844820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:01.845361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:01.845809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:01.846537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:01.847092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:01.847419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:01.857481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:01.884798 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aec4dfe0-c17f-4edc-b5c9-f40ea4ed2d41 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1564 microversion: 2.1 time: 0.946580 Oct 15 21:11:01.890115 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3394/4535] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:00 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1564 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:01.914869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:01.915394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:01.946727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:01.947424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:01.947740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:01.948414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:01.948889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:01.949360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:02.123373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:02.563985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:02.770627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e9d61b3-5db6-4f3a-ada7-90b1ab044288 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.861073 Oct 15 21:11:02.771976 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3395/4536] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:11:01 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-interface => generated 285 bytes in 863 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:11:02.838127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:02.838756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:02.847626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:02.848144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:02.848601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:02.849358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:02.849829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:02.850843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:03.171528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:03.216278 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d967f2e5-796c-47cf-95a1-613a7c5df1b7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 1.402694 Oct 15 21:11:03.216880 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1142/4537] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:01 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 1404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:03.714121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:03.723899 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad1d9c1b-13a2-42d9-b7cb-4ec90f462bda tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1667 microversion: 2.1 time: 0.890128 Oct 15 21:11:03.729968 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3396/4538] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:02 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1667 bytes in 897 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:04.246290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:04.247121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:04.262704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:04.263210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:04.263660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:04.264394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:04.264860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:04.265320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:04.418475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c48691d4-f224-4f46-a7f6-c6c67a6cc862 req-b7abe64a-8b3b-469d-babe-0002a59fdbc0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:04.459659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c48691d4-f224-4f46-a7f6-c6c67a6cc862 req-b7abe64a-8b3b-469d-babe-0002a59fdbc0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:04.467799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c48691d4-f224-4f46-a7f6-c6c67a6cc862 req-b7abe64a-8b3b-469d-babe-0002a59fdbc0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:04.468645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c48691d4-f224-4f46-a7f6-c6c67a6cc862 req-b7abe64a-8b3b-469d-babe-0002a59fdbc0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:04.481892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c48691d4-f224-4f46-a7f6-c6c67a6cc862 req-b7abe64a-8b3b-469d-babe-0002a59fdbc0 service nova] Creating event network-changed:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:11:04.488132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c48691d4-f224-4f46-a7f6-c6c67a6cc862 req-b7abe64a-8b3b-469d-babe-0002a59fdbc0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074154 Oct 15 21:11:04.489660 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3397/4539] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:11:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:11:04.760228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:04.761144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:04.775196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:04.778364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:04.778851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:04.779606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:04.781636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:04.782124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:04.855412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:04.867060 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c56bf40f-f297-48c2-8f57-ae441f491abc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.627298 Oct 15 21:11:04.874640 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1143/4540] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:04 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:05.477793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:05.488311 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21e09a96-e2de-46f2-a20e-5fa3e631bb6a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.735072 Oct 15 21:11:05.493343 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3398/4541] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:04 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:11:05.517464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:05.518083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:05.529802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:05.531506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:05.532169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:05.535278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:05.535930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:05.536635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:06.156871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fffa51c8-a554-4bf9-aff2-279af6ff2525 req-75b964ac-d4ea-48f1-8d3a-d2ae9f071b47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8cbc03dc-b1bc-422b-8a26-c1375e5a3c72", "tag": "b013b704-2ff3-4eb1-a130-54222bd18fef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:06.178945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fffa51c8-a554-4bf9-aff2-279af6ff2525 req-75b964ac-d4ea-48f1-8d3a-d2ae9f071b47 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:06.179617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fffa51c8-a554-4bf9-aff2-279af6ff2525 req-75b964ac-d4ea-48f1-8d3a-d2ae9f071b47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:06.181329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fffa51c8-a554-4bf9-aff2-279af6ff2525 req-75b964ac-d4ea-48f1-8d3a-d2ae9f071b47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:06.207990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fffa51c8-a554-4bf9-aff2-279af6ff2525 req-75b964ac-d4ea-48f1-8d3a-d2ae9f071b47 service nova] Creating event network-changed:b013b704-2ff3-4eb1-a130-54222bd18fef for instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 on nch2-master-1 Oct 15 21:11:06.221575 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fffa51c8-a554-4bf9-aff2-279af6ff2525 req-75b964ac-d4ea-48f1-8d3a-d2ae9f071b47 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.511622 Oct 15 21:11:06.223654 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3399/4542] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:11:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:11:06.254320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:06.255400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:06.274072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:06.275262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:06.276778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:06.278751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:06.281537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:06.284261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:06.908083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:06.927115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4126e492-3f07-4fd3-af22-74c92ae880c7 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.684492 Oct 15 21:11:06.933856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:06.947324 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3400/4543] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:06 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:06.953373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3b349d59-93a1-4ad3-830b-f27152d8e9f3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.439448 Oct 15 21:11:06.954456 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1144/4544] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:05 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:06.981694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"unlock": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:06.982360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:06.990729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:06.991651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:06.992284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:06.993228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:06.993873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:06.994473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:07.074128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Unlocking {{(pid=138688) unlock /opt/stack/nova/nova/compute/api.py:4874}} Oct 15 21:11:07.159636 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d5b4582-5e6b-4e5c-b834-d435172500a2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.181896 Oct 15 21:11:07.164901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3401/4545] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:06 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 185 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:11:07.172460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:07.173160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:07.182952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:07.183471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:07.183909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:07.188079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:07.189718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:07.204974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:07.820669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:07.835894 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ca2dd551-262d-42e2-8b50-fa86cbaebda2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.665487 Oct 15 21:11:07.839665 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1145/4546] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:07.867517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:07.868204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:07.878824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:07.881530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:07.887480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:07.888470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:07.892781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:07.893479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:07.970745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:07.971863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:07.984781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:07.985318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:07.985768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:07.986502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:07.997611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:08.000016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:08.500061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:08.515802 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f0a9a181-988c-4105-9dd8-5857b26ef657 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.654683 Oct 15 21:11:08.516715 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3402/4547] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:08.550770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:08.569948 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4712ce6a-c9e8-481a-a0e2-03157cae26f4 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.603770 Oct 15 21:11:08.571161 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1146/4548] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:07 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:08.589056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:08.589954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:08.596395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:08.596892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:08.604191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:08.605332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:08.605973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:08.607130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:08.910681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Calling method '.version_select at 0x7fe74e0d1a60>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:08.912501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:08.913002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:08.913639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:08.946423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:08.947388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:08.953327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:08.972528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:08.973613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:08.975398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:08.985922 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0eaf97ad-6fbc-4b27-985c-888e710e0f98 tempest-PortsAdminExtendedAttrsTestJSON-543184684 tempest-PortsAdminExtendedAttrsTestJSON-543184684-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.467521 Oct 15 21:11:08.996460 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3403/4549] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:11:08 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:11:09.015456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:09.034072 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-885e455a-7eb9-4e4f-83b0-49491beb0711 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.458191 Oct 15 21:11:09.035080 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1147/4550] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:08 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:09.063386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:09.063940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:09.088376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:09.088722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:09.090132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:09.090859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:09.091944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:09.092690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:09.173026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:09.212641 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77b6b499-30a1-4020-8b40-e4004f3249b2 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.156769 Oct 15 21:11:09.216330 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3404/4551] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:09 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 161 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:11:09.227263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:09.229107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:09.242319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:09.262790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:09.265731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:09.268103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:09.269145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:09.272730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:09.312293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:09.567553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:09.580728 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b201137c-933b-439f-9a03-a284bbc7e0fc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1676 microversion: 2.1 time: 0.357572 Oct 15 21:11:09.582540 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1148/4552] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:09 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1676 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:09.603286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:09.604179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:09.614889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:09.615916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:09.616774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:09.618772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:09.619491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:09.620182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:09.796132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:09.806995 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c21f6ee-99e5-47ed-be2c-8a230381f682 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.215399 Oct 15 21:11:09.836444 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3405/4553] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:09 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:10.606248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:10.607604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:10.624908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:10.628934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:10.629435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:10.632270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:10.635152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:10.635736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:10.835842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:10.836614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:10.860843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:10.862066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:10.862753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:10.864688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:10.866241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:10.870198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:11.569319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:11.582004 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e6b1987-5371-45c9-8ad1-c2619e5552a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1656 microversion: 2.1 time: 0.977820 Oct 15 21:11:11.584719 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1149/4554] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:10 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1656 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:11.607918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:11.608638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:11.616282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:11.616774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:11.617243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:11.617952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:11.618407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:11.618832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:11.702342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:11.718203 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81eaea9d-8ef8-4d7d-b47a-eebd5ea9113c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.887102 Oct 15 21:11:11.720799 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3406/4555] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:10 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:11:11.739846 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-221e0023-f991-447b-bf24-9be90bf5eef7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.131842 Oct 15 21:11:11.749548 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1150/4556] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:11 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:11:11.750944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:11.752444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:11.761565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:11.762382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:11.763056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:11.764408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:11.765438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:11.768069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:12.748163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:12.748922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:12.772918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:12.774961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:12.775446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:12.776161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:12.783453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:12.784323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:13.067258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:13.088668 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1eda8dab-8c85-4e26-9a60-40d352894f03 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1663 microversion: 2.1 time: 1.340189 Oct 15 21:11:13.104430 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3407/4557] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:11 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1663 bytes in 1352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:13.170210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6580e61d-8394-4798-a2a1-d3ba614c22da req-0f187c09-e2e5-4b8d-9058-36d1c2d87334 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "name": "network-vif-plugged", "status": "completed", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:13.192782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6580e61d-8394-4798-a2a1-d3ba614c22da req-0f187c09-e2e5-4b8d-9058-36d1c2d87334 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:13.193944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6580e61d-8394-4798-a2a1-d3ba614c22da req-0f187c09-e2e5-4b8d-9058-36d1c2d87334 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:13.196273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6580e61d-8394-4798-a2a1-d3ba614c22da req-0f187c09-e2e5-4b8d-9058-36d1c2d87334 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:13.228064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6580e61d-8394-4798-a2a1-d3ba614c22da req-0f187c09-e2e5-4b8d-9058-36d1c2d87334 service nova] Creating event network-vif-plugged:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:11:13.237592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6580e61d-8394-4798-a2a1-d3ba614c22da req-0f187c09-e2e5-4b8d-9058-36d1c2d87334 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072532 Oct 15 21:11:13.240832 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3408/4558] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:11:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:11:13.722503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:13.734458 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e4d42536-b3c4-47ae-afbd-fa9a2850396a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.987791 Oct 15 21:11:13.737139 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1151/4559] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:12 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:14.125478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:14.126903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:14.149340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:14.149935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:14.152100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:14.154996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:14.155666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:14.157393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:14.629324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:14.642318 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8019188a-8b95-42b8-bca8-2ad2bc8576b3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.527007 Oct 15 21:11:14.644596 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3409/4560] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:14 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:14.671890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:14.672512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:14.681285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:14.681776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:14.682223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:14.682940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:14.683401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:14.683830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:14.763120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:14.763748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:14.776818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:14.777690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:14.778302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:14.779207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:14.779827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:14.780423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:15.294138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:15.300721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:15.309862 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-18fd7d11-7f73-4439-ae99-5695ce7b8782 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.641669 Oct 15 21:11:15.314042 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-162d05d1-b1b2-489f-ae43-8deeb3686e94 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1323 microversion: 2.1 time: 0.554531 Oct 15 21:11:15.315814 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3410/4562] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:14 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1323 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:15.329915 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1152/4562] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:14 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:15.343592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:15.344530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:15.358938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:15.360062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:15.361560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:15.362958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:15.365340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:15.368172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:15.834402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:15.842277 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b71bded9-8027-43ae-9245-11e5eee39e0f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.504701 Oct 15 21:11:15.842873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3411/4563] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:15 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:15.867206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:15.868189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:15.875594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:15.876276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:15.877071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:15.878389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:15.879028 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:15.879481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.339614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:16.340256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:16.342475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:16.351853 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d27aba4d-4b6c-48b3-85c5-ae68dbae146e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.489439 Oct 15 21:11:16.370705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.370705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.370705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.370705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.370705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.370705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.372571 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1153/4564] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:15 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:16.379091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:16.380290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:16.400248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.400761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.401236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.402017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.402543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.403220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.808254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:16.818057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6401b8bc-7cbb-4487-8264-2cdcd5a13f7b tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1519 microversion: 2.1 time: 0.482166 Oct 15 21:11:16.840549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:16.844737 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d0f6d78e-016b-4e2e-875f-9899f4cbc924 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.469995 Oct 15 21:11:16.845838 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3412/4565] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:16 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1519 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:16.853987 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1154/4566] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:16 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:16.859816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:16.864147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:16.871336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.872113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.872926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.873901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.885616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:16.890822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:16.898787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.900619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.916374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.918938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.923397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:16.925417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:16.932607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:16.934211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:17.009791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lazy-loading 'flavor' on Instance uuid c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:11:17.160983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-60b65819-48c9-4370-a993-8c12bd7de7ce tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action" status: 200 len: 32 microversion: 2.1 time: 0.309611 Oct 15 21:11:17.166849 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3413/4567] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:16 2022] POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action => generated 32 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:11:17.183606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:17.184442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:17.207355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:17.211720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:17.212219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:17.212987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:17.213503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:17.216622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:17.448116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:17.459671 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1bc065aa-46b4-47da-a6e3-1419e73ca493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.583668 Oct 15 21:11:17.460400 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1155/4568] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:16 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:11:18.316789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:18.333863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c2dda50-f7a4-47c5-a19e-13a0c68c706a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 1.163918 Oct 15 21:11:18.334448 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3414/4569] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:17 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 1167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:19.360285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:19.362937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:19.404760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:19.408855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:19.419731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:19.424585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:19.427162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:19.429389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:20.031517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:20.048520 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a750a9fb-b22d-482a-b0aa-ae4a7aae461d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.695000 Oct 15 21:11:20.062370 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1156/4570] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:19 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 710 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:11:21.074208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:21.075557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:21.088282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:21.089489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:21.091341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:21.095605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:21.096407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:21.098519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:21.836558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:21.850391 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a561aa6f-edc8-4a3f-bb52-db057d8f01f0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.775918 Oct 15 21:11:21.851052 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3415/4571] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:21 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:22.877938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:22.878574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:22.901888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:22.902393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:22.903422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:22.914405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:22.938466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:22.939077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.025s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:23.534067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:23.549349 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5e4c67f9-c848-44b2-b62b-2d5c99a79ede tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.673264 Oct 15 21:11:23.550523 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1157/4572] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:22 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:24.577359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:24.578248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:24.597323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:24.598138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:24.600153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:24.601391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:24.602148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:24.610447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:25.031331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:25.050107 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-164b8e16-06e6-4335-b46c-eb1b32d080a2 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.477769 Oct 15 21:11:25.051916 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3416/4573] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:24 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:26.075534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:26.076146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:26.089679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:26.091613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:26.092083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:26.093124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:26.093404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:26.096110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:26.559479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:26.581883 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff0bda12-96e8-4d8c-b3c2-5fc9df52ac3e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.509884 Oct 15 21:11:26.602402 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1158/4574] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:26 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:27.617831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:27.618600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:27.632874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:27.635628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:27.636297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:27.637080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:27.637550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:27.637996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:28.067094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:28.099448 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc3b31ad-02ab-4bfa-a47f-d3c1d061979d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.480545 Oct 15 21:11:28.116954 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3417/4575] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:27 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:29.140366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:29.140989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:29.155525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:29.158870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:29.159600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:29.160707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:29.161874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:29.164144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:29.480488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:29.490173 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-63c15933-3d6f-41bb-87ae-9a9c307bee9c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.354236 Oct 15 21:11:29.490753 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1159/4576] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:29 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:30.517489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:30.518922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:30.523383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=58 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:30.536738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:30.537984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:30.538434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:30.539189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:30.539706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:30.540098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:30.788919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:30.802600 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc71bbbe-7f30-4f7d-a275-c9dbd709e75a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.288564 Oct 15 21:11:30.806549 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3418/4577] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:30 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:31.832905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:31.834214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:31.853930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=38 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:31.855656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:31.856440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:31.860502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:31.861682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:31.862738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:31.863936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:32.616646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:32.626281 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c20a8fce-d21b-43c6-ba9f-57c2aba1479e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.797151 Oct 15 21:11:32.632235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1160/4578] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:31 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:33.654692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:33.655469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:33.676038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:33.676609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:33.679873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:33.681838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:33.685117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:33.685798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:33.731374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:34.365670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:34.377872 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-900803c1-4443-4fe1-9603-e6671fd258bc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.727588 Oct 15 21:11:34.385185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3419/4579] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:33 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:35.404471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:35.405128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:35.419077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:35.421101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:35.421820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:35.427699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:35.428474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:35.429431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:35.943361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:35.955431 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-27217f57-b193-4581-808c-80ca21c5c49c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.554768 Oct 15 21:11:35.956013 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1161/4580] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:35 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:36.980714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:36.981160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:36.989990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:36.990602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:36.991060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:36.998985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:36.999699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:37.000068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:37.467341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:37.491862 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99a148f0-9fe0-4cfb-8555-cacbaf33498c tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.506790 Oct 15 21:11:37.493937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3420/4581] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:36 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:38.516135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:38.516767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:38.537381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:38.538128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:38.538740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:38.539627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:38.540244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:38.540850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:38.956679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:38.965354 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-60bc0cbe-682b-433b-b69f-7fbb71f6f5c6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.453400 Oct 15 21:11:38.986247 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1162/4582] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:38 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:39.994853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:39.995634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:40.009394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:40.010055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:40.010594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:40.012364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:40.014813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:40.015715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:40.385854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:40.400115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8888be0e-3e83-4ec8-a935-a4a3f0f60cc3 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.406866 Oct 15 21:11:40.401152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3421/4583] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:39 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:41.434084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:41.435329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:41.450309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:41.450817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:41.453009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:41.455246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:41.457851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:41.460328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:41.470783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:11:41.780759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:41.814277 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5a2ba764-8040-4a4c-9ab1-0677fc815659 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.389776 Oct 15 21:11:41.816840 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1163/4584] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:41 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:42.842096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:42.842735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:42.862268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:42.862990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:42.864825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:42.865833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:42.866430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:42.866996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:43.462693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:43.482077 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-453799a0-3117-4fd4-a21d-16d3856e5493 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.644247 Oct 15 21:11:43.498022 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3422/4585] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:42 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:44.511443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:44.512712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:44.524780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:44.525477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:44.526151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:44.527103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:44.527729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:44.528350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:44.867632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:44.879499 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4607b731-81ed-4b01-90dc-d58e8e254156 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.372380 Oct 15 21:11:44.880169 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1164/4586] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:44 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:45.905029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:45.905589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:45.916934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:45.917465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:45.917919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:45.918652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:45.919125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:45.919560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:46.534551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:46.543460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1306bb4-77b4-405f-a265-ea6c70898ebc tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1525 microversion: 2.1 time: 0.642348 Oct 15 21:11:46.544294 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3423/4587] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:45 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1525 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:47.573366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:11:47.574776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:47.615133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:47.617561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:47.618232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:47.619392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:47.620210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:47.620797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:48.086957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:11:48.099440 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0eb9a9e7-d155-4311-b6c8-638043c1c197 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1505 microversion: 2.1 time: 0.530701 Oct 15 21:11:48.101146 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1165/4588] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:11:47 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1505 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:11:48.124242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13621c98-b6c0-4389-a514-7e4aa6276439 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'create', calling method: .version_select at 0x7fe74e065280>, body: {"security_group": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-108206139", "description": "tempest-description-1469379912"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:49.001559 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13621c98-b6c0-4389-a514-7e4aa6276439 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 250 microversion: 2.1 time: 0.878261 Oct 15 21:11:49.004131 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3424/4589] 10.222.0.16 () {60 vars in 1180 bytes} [Sat Oct 15 21:11:48 2022] POST /compute/v2.1/os-security-groups => generated 250 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:11:49.016952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-108206139"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:49.018187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:49.025933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:49.026604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:49.027213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:49.028123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:49.028740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:49.029356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:49.434259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.network.security_group_api [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Adding security group a876880d-3c62-4a53-a343-ed926a010747 to port a43a4a31-e3c5-4257-8fec-86f26ec937a6 Oct 15 21:11:50.379851 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52c79262-2c78-4b2b-82d5-63995ddb455a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action" status: 202 len: 0 microversion: 2.1 time: 1.367037 Oct 15 21:11:50.380711 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1166/4590] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:49 2022] POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action => generated 0 bytes in 1370 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:11:50.390176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-108206139"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:50.390876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:50.411251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-40887b61-6c70-4154-8232-b6ac7151a1d1 req-ec136342-8a0f-4d87-a0e2-159364bd6827 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:50.424996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:50.426944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:50.428598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:50.431278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:50.432646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:50.434112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:50.439732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-40887b61-6c70-4154-8232-b6ac7151a1d1 req-ec136342-8a0f-4d87-a0e2-159364bd6827 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:50.442195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-40887b61-6c70-4154-8232-b6ac7151a1d1 req-ec136342-8a0f-4d87-a0e2-159364bd6827 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:50.446779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-40887b61-6c70-4154-8232-b6ac7151a1d1 req-ec136342-8a0f-4d87-a0e2-159364bd6827 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:50.482981 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-40887b61-6c70-4154-8232-b6ac7151a1d1 req-ec136342-8a0f-4d87-a0e2-159364bd6827 service nova] Creating event network-changed:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:11:50.492513 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-40887b61-6c70-4154-8232-b6ac7151a1d1 req-ec136342-8a0f-4d87-a0e2-159364bd6827 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086878 Oct 15 21:11:50.500823 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1167/4591] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:11:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:11:51.045503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.network.security_group_api [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Removing security group a876880d-3c62-4a53-a343-ed926a010747 from port a43a4a31-e3c5-4257-8fec-86f26ec937a6 Oct 15 21:11:52.453326 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f410d527-8a02-40bb-8439-637fd5e8e841 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action" status: 202 len: 0 microversion: 2.1 time: 2.068051 Oct 15 21:11:52.455548 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3425/4592] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:50 2022] POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action => generated 0 bytes in 2071 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:11:52.468829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5d251c43-a761-43e6-a654-5043927874a0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'create', calling method: .version_select at 0x7fe74eee44c0>, body: {"pool": null} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:52.596120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-476ce104-0b48-44d5-96a5-5ccc983c6b17 req-46568f9e-7411-4bd8-8434-245d1656fc0b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:52.607876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-476ce104-0b48-44d5-96a5-5ccc983c6b17 req-46568f9e-7411-4bd8-8434-245d1656fc0b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:52.608656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-476ce104-0b48-44d5-96a5-5ccc983c6b17 req-46568f9e-7411-4bd8-8434-245d1656fc0b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:52.609310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-476ce104-0b48-44d5-96a5-5ccc983c6b17 req-46568f9e-7411-4bd8-8434-245d1656fc0b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:52.623679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-476ce104-0b48-44d5-96a5-5ccc983c6b17 req-46568f9e-7411-4bd8-8434-245d1656fc0b service nova] Creating event network-changed:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:11:52.631554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-476ce104-0b48-44d5-96a5-5ccc983c6b17 req-46568f9e-7411-4bd8-8434-245d1656fc0b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039002 Oct 15 21:11:52.632821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3426/4593] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:11:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:11:57.281312 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5d251c43-a761-43e6-a654-5043927874a0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 4.818266 Oct 15 21:11:57.289952 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1168/4594] 10.222.0.16 () {60 vars in 1170 bytes} [Sat Oct 15 21:11:52 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 4821 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:11:57.296567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.4.38"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:11:57.299051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:11:57.306153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:57.306619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:57.307059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:57.307761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:11:57.308216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:11:57.308647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:11:57.336110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "refresh_cache-c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:11:57.336645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquired lock "refresh_cache-c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:12:00.974610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:12:01.037196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=15 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:01.049242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=23 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:01.054722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-26cd3979-68c3-4d7c-b0ec-62044a3bf0f9 req-7fff633f-74dc-41de-9991-37b98bc8610b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:01.080558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-26cd3979-68c3-4d7c-b0ec-62044a3bf0f9 req-7fff633f-74dc-41de-9991-37b98bc8610b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:01.081528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-26cd3979-68c3-4d7c-b0ec-62044a3bf0f9 req-7fff633f-74dc-41de-9991-37b98bc8610b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:01.083422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-26cd3979-68c3-4d7c-b0ec-62044a3bf0f9 req-7fff633f-74dc-41de-9991-37b98bc8610b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:01.097493 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-26cd3979-68c3-4d7c-b0ec-62044a3bf0f9 req-7fff633f-74dc-41de-9991-37b98bc8610b service nova] Creating event network-changed:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:12:01.105737 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-26cd3979-68c3-4d7c-b0ec-62044a3bf0f9 req-7fff633f-74dc-41de-9991-37b98bc8610b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073110 Oct 15 21:12:01.107454 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1169/4595] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:12:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:12:02.946382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:02.948720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:02.962756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:02.963468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:02.964091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:02.964984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:02.965696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:02.966286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:03.119326 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3a5e4bd5-aeea-4ed8-9df1-b0c24340cc8d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.178012 Oct 15 21:12:03.138017 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1170/4596] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:12:02 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 198 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:12:03.159042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:03.162439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:03.172855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:03.173571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:03.174191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:03.175089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:03.175813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:03.176414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:04.138524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:04.149724 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-daa93101-e393-4f12-87ad-77917a1c1f55 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1673 microversion: 2.1 time: 1.006137 Oct 15 21:12:04.150992 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1171/4597] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:03 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1673 bytes in 1011 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:05.177996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:05.179961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:05.188484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:05.217385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Updating instance_info_cache with network_info: [{"id": "a43a4a31-e3c5-4257-8fec-86f26ec937a6", "address": "fa:16:3e:79:d5:9e", "network": {"id": "9f248ce0-36ef-45a7-b1ea-31b162ddb04e", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-2055464400-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.4.38", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "16eb29186a7b4964a49882162a35cf06", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapa43a4a31-e3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:12:05.236981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:05.240446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=42 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:05.241006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.048s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:05.243010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:05.246472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:05.248876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Releasing lock "refresh_cache-c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:12:05.249525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:05.251308 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2696107b-83e6-42b3-a54c-165d4f99becf tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action" status: 202 len: 0 microversion: 2.1 time: 7.959967 Oct 15 21:12:05.260911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3427/4598] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:11:57 2022] POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action => generated 0 bytes in 7970 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:12:05.277176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.4.38"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:06.382316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:06.385822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=18 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:06.393157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:06.394475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:06.395356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:06.396295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:06.397832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:06.398717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:06.427879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:06.445681 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-37d96d87-61f1-44b8-80c3-991ae7d54e25 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1673 microversion: 2.1 time: 1.270722 Oct 15 21:12:06.447213 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1172/4599] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:05 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1673 bytes in 1275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:12:07.359698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "refresh_cache-c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 15 21:12:07.360759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquired lock "refresh_cache-c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 15 21:12:07.471807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:07.473598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:07.502101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:07.502986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:07.503986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:07.505263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:07.506006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:07.507054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:08.752477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:08.763678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e948ba0b-0e57-4494-a91a-cf676880a64a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1678 microversion: 2.1 time: 1.297180 Oct 15 21:12:08.764544 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1173/4600] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1678 bytes in 1299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:08.939927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "83707e8c-d087-4efb-9ab5-b1c112500f85"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:08.943115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:08.950068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:08.950792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:08.951601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:08.952561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:08.953231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:08.953854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:09.423017 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8cf2a28c-b7b9-4ff2-b8d5-2959701f1823 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.486995 Oct 15 21:12:09.423992 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1174/4601] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:12:08 2022] POST /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments => generated 194 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:12:09.793526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:09.794224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:09.812170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:09.813673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:09.814133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:09.819476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:09.828313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:09.829080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:09.885343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Building network info cache for instance {{(pid=138688) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Oct 15 21:12:10.708775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:10.719977 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d70870ed-776b-4e7a-af90-0b492da54da3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.931662 Oct 15 21:12:10.722749 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1175/4602] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:09 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 935 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:12:10.738943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-332413e5-1c2c-4f58-b908-fa78a0b9af80 req-8c601524-33f3-499d-8a3c-62225ca8f18e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:10.764362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-332413e5-1c2c-4f58-b908-fa78a0b9af80 req-8c601524-33f3-499d-8a3c-62225ca8f18e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:10.765278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-332413e5-1c2c-4f58-b908-fa78a0b9af80 req-8c601524-33f3-499d-8a3c-62225ca8f18e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:10.765933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-332413e5-1c2c-4f58-b908-fa78a0b9af80 req-8c601524-33f3-499d-8a3c-62225ca8f18e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:10.790196 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-332413e5-1c2c-4f58-b908-fa78a0b9af80 req-8c601524-33f3-499d-8a3c-62225ca8f18e service nova] Creating event network-changed:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:12:10.807887 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-332413e5-1c2c-4f58-b908-fa78a0b9af80 req-8c601524-33f3-499d-8a3c-62225ca8f18e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078911 Oct 15 21:12:10.809334 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1176/4603] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:12:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:12:12.762474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Updating instance_info_cache with network_info: [{"id": "a43a4a31-e3c5-4257-8fec-86f26ec937a6", "address": "fa:16:3e:79:d5:9e", "network": {"id": "9f248ce0-36ef-45a7-b1ea-31b162ddb04e", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-2055464400-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "16eb29186a7b4964a49882162a35cf06", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapa43a4a31-e3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138688) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:12:12.788358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Releasing lock "refresh_cache-c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" {{(pid=138688) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 15 21:12:12.789441 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b8cf594-20bb-4ed0-b430-e7b131c3eba6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action" status: 202 len: 0 microversion: 2.1 time: 7.526640 Oct 15 21:12:12.791390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3428/4604] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:12:05 2022] POST /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b/action => generated 0 bytes in 7530 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Oct 15 21:12:12.803064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7341faf-727f-47d1-82ee-b66e195be384 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '.version_select at 0x7fe74ee84b80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:13.027226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:13.032770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:13.041676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:13.043209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:13.043936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:13.044987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:13.045710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:13.046408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:13.089399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lazy-loading 'flavor' on Instance uuid 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:12:13.334087 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-601a7415-3e49-42b9-be4f-dac519078d36 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/action" status: 200 len: 32 microversion: 2.1 time: 0.310514 Oct 15 21:12:13.347508 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3429/4605] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:12:13 2022] POST /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/action => generated 32 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 15 21:12:13.355265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:13.356804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:13.364342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:13.365409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:13.366060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:13.366965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:13.367602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:13.368211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:13.865213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:13.876993 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5fc4cf36-6398-4bdb-9bcd-28ecd693ecd4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.526855 Oct 15 21:12:13.879346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3430/4606] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:13 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:14.902400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:14.903187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:14.918427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:14.924375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:14.926468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:14.927595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:14.930111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:14.930577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:15.891007 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d7341faf-727f-47d1-82ee-b66e195be384 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/18d435ad-128e-44fb-9b4a-fb85c05ceffe" status: 202 len: 0 microversion: 2.1 time: 3.095713 Oct 15 21:12:15.891691 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1177/4607] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:12:12 2022] DELETE /compute/v2.1/os-floating-ips/18d435ad-128e-44fb-9b4a-fb85c05ceffe => generated 0 bytes in 3097 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:12:15.896022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:15.901242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09879d04-1851-4e3a-87cb-586a0b12245d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '.version_select at 0x7fe74eeb9670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:15.911535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da2a5e0c-ece8-4c4a-a0be-b82ae73ac570 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 1.013212 Oct 15 21:12:15.912118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3431/4608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:14 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 1015 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:12:16.778865 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-09879d04-1851-4e3a-87cb-586a0b12245d tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/a876880d-3c62-4a53-a343-ed926a010747" status: 202 len: 0 microversion: 2.1 time: 0.880961 Oct 15 21:12:16.779614 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1178/4609] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:12:15 2022] DELETE /compute/v2.1/os-security-groups/a876880d-3c62-4a53-a343-ed926a010747 => generated 0 bytes in 883 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:12:16.791067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:16.791651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:16.801515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:16.802251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:16.803762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:16.804472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:16.805534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:16.806114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:16.863988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:12:16.941173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:16.942247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:16.961746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:16.962261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:16.968977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:16.969860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:16.970478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:16.971058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:17.027085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ca3bf4c6-0422-4b74-8b97-9893d61f78e6 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 204 len: 0 microversion: 2.1 time: 0.242547 Oct 15 21:12:17.027690 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3432/4610] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:12:16 2022] DELETE /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:12:17.038292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:17.043923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:17.073450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:17.074647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:17.075151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:17.075921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:17.079075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:17.079563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:17.483403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:17.526346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7e202a05-f020-4f29-ab43-42feae8a3d79 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.589226 Oct 15 21:12:17.528279 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1179/4611] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:16 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:12:17.582063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:17.593263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8245cfbf-b499-4040-9722-d7a662b547c0 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1511 microversion: 2.1 time: 0.559519 Oct 15 21:12:17.594245 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3433/4612] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:17 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1511 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:18.556100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:18.556730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:18.567224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:18.568652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:18.569305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:18.570356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:18.571238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:18.573317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:18.618582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:18.619204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:18.634016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:18.634826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:18.635470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:18.636439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:18.637101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:18.645921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:19.736725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:19.741384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:19.751244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec7b8020-b236-4b66-97a8-04b34fa17c6e tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1511 microversion: 2.1 time: 1.134933 Oct 15 21:12:19.751871 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3434/4613] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:18 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1511 bytes in 1138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:12:19.765863 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-560d8323-c74c-4d3e-bd69-d10fb2d653fb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 1.213975 Oct 15 21:12:19.776226 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1180/4614] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:18 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 1225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:20.786569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:20.788016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:20.796480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:20.797791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:20.813236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:20.819145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:20.819827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:20.820570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:20.824500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:20.824908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:20.825549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:20.826324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:20.827236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:20.827844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:20.833191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:20.834125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:21.625863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:21.638562 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffc02bf1-0608-4eb8-a538-fb523fbc8725 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1511 microversion: 2.1 time: 0.858027 Oct 15 21:12:21.666749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:21.679905 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1a8d0b1e-f442-411a-8de0-9b5c8cff0fbe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.889401 Oct 15 21:12:21.680767 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1181/4615] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:20 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:21.712408 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3435/4616] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:20 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1511 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:22.389669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b76e3185-913e-43f5-a0a5-c4ac50a791e3 req-a613e652-9c9d-487c-b516-cbb85168b07a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b", "name": "network-vif-deleted", "tag": "a43a4a31-e3c5-4257-8fec-86f26ec937a6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:22.432053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b76e3185-913e-43f5-a0a5-c4ac50a791e3 req-a613e652-9c9d-487c-b516-cbb85168b07a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:22.433863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b76e3185-913e-43f5-a0a5-c4ac50a791e3 req-a613e652-9c9d-487c-b516-cbb85168b07a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:22.438004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b76e3185-913e-43f5-a0a5-c4ac50a791e3 req-a613e652-9c9d-487c-b516-cbb85168b07a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:22.497543 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b76e3185-913e-43f5-a0a5-c4ac50a791e3 req-a613e652-9c9d-487c-b516-cbb85168b07a service nova] Creating event network-vif-deleted:a43a4a31-e3c5-4257-8fec-86f26ec937a6 for instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b on nch1-master-1 Oct 15 21:12:22.513918 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b76e3185-913e-43f5-a0a5-c4ac50a791e3 req-a613e652-9c9d-487c-b516-cbb85168b07a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.129225 Oct 15 21:12:22.516412 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1182/4617] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:12:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:12:22.682114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:22.682606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:22.698448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:22.698968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:22.700027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:22.710495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:22.711614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:22.715361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:22.718452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:22.719426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:22.726654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:22.727664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:22.730363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:22.731483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:22.733613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:22.734248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:22.825147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:22.825912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:22.828547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:22.830437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lazy-loading 'fault' on Instance uuid c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:12:23.082149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7eae9dc0a1a47d9910f49bb33316d91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '16eb29186a7b4964a49882162a35cf06', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:23.123261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-74612f96-750b-4269-b389-34361c2c1c6a tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 200 len: 1313 microversion: 2.1 time: 0.432432 Oct 15 21:12:23.124814 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3436/4618] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:22 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 1313 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:12:23.404964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:23.414306 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bd0aca0e-3362-4bc3-a353-d9d3312e3c7c tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.709527 Oct 15 21:12:23.426060 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1183/4619] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:22 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:24.153434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:24.170937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] [instance: c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:24.181172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:24.183429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:24.184732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:24.186035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:24.186731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:24.187547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:24.211197 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] HTTP exception thrown: Instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b could not be found. Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Request method failure captured: Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b HTTP/1.1 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyHZgQnF8o0EP0HJdCFcfjkkCAEn6gR-je_glnsHP_1M4sk2rZC1N8Pf42G_mug03SXZzTRNbO5QCX8CURP_X3MG79OqvWlHYKBLXs3ZtFfPi_sNpMkH_lLVd9-JcgMbwYiPV34-4v-2S2IoTdwkB7KrcKNuhV5euwHv0GPmxA7NyeY Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 16eb29186a7b4964a49882162a35cf06 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueTestJSONUnderV235-1896818738 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/16eb29186a7b4964a49882162a35cf06"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_16eb29186a7b4964a49882162a35cf06"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/16eb29186a7b4964a49882162a35cf06"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/16eb29186a7b4964a49882162a35cf06"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueTestJSONUnderV235-1896818738 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 16eb29186a7b4964a49882162a35cf06 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueTestJSONUnderV235-1896818738 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueTestJSONUnderV235-1896818738-project Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a7eae9dc0a1a47d9910f49bb33316d91 Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueTestJSONUnderV235-1896818738-project Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b could not be found. Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b'} Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b could not be found. Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b could not be found. Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:12:24.215473 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b could not be found. Oct 15 21:12:24.222494 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:12:24.234126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] Returning 404 to user: Instance c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:12:24.241625 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9542a45-a0ae-4a3b-bd12-6c22ec2a1a68 tempest-ServerRescueTestJSONUnderV235-1896818738 tempest-ServerRescueTestJSONUnderV235-1896818738-project] 10.222.0.16 "GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b" status: 404 len: 111 microversion: 2.1 time: 0.086551 Oct 15 21:12:24.246623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3437/4620] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:24 2022] GET /compute/v2.1/servers/c90bb78e-e1cd-4129-ad31-1d58e4f4ec7b => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:12:24.444670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:24.445508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:24.452983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:24.453599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:24.454256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:24.455459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:24.456772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:24.457292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:25.248856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:25.262889 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-12fff516-99ae-4f5f-ac1d-1aed86b957f4 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.823112 Oct 15 21:12:25.264414 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1184/4621] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:24 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 825 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:26.288493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:26.289803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:26.300493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:26.301009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:26.301473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:26.302191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:26.305711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:26.306357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:26.684233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:26.698078 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-870433d7-fd00-4f94-8755-378dae7fd423 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.413313 Oct 15 21:12:26.705558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3438/4622] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:26 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:27.731657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:27.731657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:27.747637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:27.748082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:27.748563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:27.749253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:27.750268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:27.750867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:28.312396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:28.358208 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d81d1aed-4582-484c-b2ea-3cdfd33a091b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1735 microversion: 2.1 time: 0.612902 Oct 15 21:12:28.363340 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1185/4623] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:27 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1735 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:29.390667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:29.391490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:29.425248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:29.426594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:29.427461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:29.428566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:29.429335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:29.431760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:30.964745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:30.969282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=89,nova_cell1:SAVEPOINT=2,nova_cell1:UPDATE=5,nova_cell1:RELEASE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:30.984374 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e6f650ce-14b9-4b43-af6b-473c70ee8713 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1715 microversion: 2.1 time: 1.598004 Oct 15 21:12:30.990644 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3439/4624] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:29 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1715 bytes in 1605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:31.196743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:31.197313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:31.208829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=31 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:31.210364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:31.210847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:31.211279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:31.211996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:31.223060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:31.223515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:31.235495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:31.380687 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Cannot 'detach_volume' instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 while it is in vm_state rescued Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments/83707e8c-d087-4efb-9ab5-b1c112500f85 HTTP/1.1 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: exception: Cannot 'detach_volume' instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 while it is in vm_state rescued Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'server_id': '8cbc03dc-b1bc-422b-8a26-c1375e5a3c72', 'id': '83707e8c-d087-4efb-9ab5-b1c112500f85'} Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 while it is in vm_state rescued Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:12:31.382394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:12:31.387345 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:12:31.387345 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:12:31.387345 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:12:31.387345 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:12:31.387345 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 while it is in vm_state rescued Oct 15 21:12:31.387345 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:12:31.389348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 409 to user: Cannot 'detach_volume' instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 while it is in vm_state rescued {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:12:31.390409 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dcacd885-5b12-4e3f-8c82-b18c70699b5e tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments/83707e8c-d087-4efb-9ab5-b1c112500f85" status: 409 len: 152 microversion: 2.1 time: 0.197774 Oct 15 21:12:31.391421 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1186/4625] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:12:31 2022] DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments/83707e8c-d087-4efb-9ab5-b1c112500f85 => generated 152 bytes in 199 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:12:31.402632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:31.403692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:31.411627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:31.412453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:31.413139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:31.414279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:31.415025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:31.415744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:31.537744 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2fd11d0f-189d-4923-ac92-8c1d158fa3e3 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/action" status: 202 len: 0 microversion: 2.1 time: 0.140662 Oct 15 21:12:31.539318 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3440/4626] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:12:31 2022] POST /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/action => generated 0 bytes in 143 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:12:31.548010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:31.549526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:31.561682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:31.562939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:31.564065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:31.569412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:31.570800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:31.571798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:32.354622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:32.391200 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bad50594-1d69-4742-8f8c-f38345f95041 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1723 microversion: 2.1 time: 0.847508 Oct 15 21:12:32.391797 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1187/4627] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:31 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1723 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:33.418982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:33.420290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:33.433464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:33.434212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:33.434882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:33.435844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:33.437017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:33.437782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:34.684693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:34.694932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-639c81e5-e279-4b78-af83-06b407d88eb6 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1723 microversion: 2.1 time: 1.282711 Oct 15 21:12:34.696320 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3441/4628] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:33 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1723 bytes in 1285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:35.720032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:35.720468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:35.740077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:35.741104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:35.741777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:35.742915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:35.743595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:35.744240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:36.188498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:36.218676 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9d15daaa-9046-4fce-b1ed-cb6c71ea6036 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1723 microversion: 2.1 time: 0.502552 Oct 15 21:12:36.220009 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1188/4629] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:35 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1723 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:37.246812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:37.248726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:37.254174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:37.261793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:37.262567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:37.263111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:37.263941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:37.265284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:37.265806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:38.161132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:38.175586 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e621210b-80b2-4ff2-bb4b-d48ee71bbe20 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1729 microversion: 2.1 time: 0.933101 Oct 15 21:12:38.182851 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3442/4630] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:12:37 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1729 bytes in 941 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:12:38.211555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:38.212089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:38.223826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:38.224333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:38.224775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:38.226598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:38.227449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:38.229567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:38.590595 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea0b61e8-8e72-4b22-a4f0-213a465bfda0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.386871 Oct 15 21:12:38.591599 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1189/4631] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:12:38 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-interface => generated 285 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:12:42.236354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c0e061bc-cee6-4ffa-a006-0f1a03a6072d req-8fe2540e-b253-47ab-99eb-e730ce6db816 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8cbc03dc-b1bc-422b-8a26-c1375e5a3c72", "tag": "b013b704-2ff3-4eb1-a130-54222bd18fef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:42.255316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c0e061bc-cee6-4ffa-a006-0f1a03a6072d req-8fe2540e-b253-47ab-99eb-e730ce6db816 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:42.256474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c0e061bc-cee6-4ffa-a006-0f1a03a6072d req-8fe2540e-b253-47ab-99eb-e730ce6db816 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:42.258862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c0e061bc-cee6-4ffa-a006-0f1a03a6072d req-8fe2540e-b253-47ab-99eb-e730ce6db816 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:42.287461 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c0e061bc-cee6-4ffa-a006-0f1a03a6072d req-8fe2540e-b253-47ab-99eb-e730ce6db816 service nova] Creating event network-changed:b013b704-2ff3-4eb1-a130-54222bd18fef for instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 on nch2-master-1 Oct 15 21:12:42.298143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c0e061bc-cee6-4ffa-a006-0f1a03a6072d req-8fe2540e-b253-47ab-99eb-e730ce6db816 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071297 Oct 15 21:12:42.303795 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3443/4632] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:12:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:12:44.337505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e3117b77-83a7-4b65-8741-2e567c861e21 req-d781b09a-d906-4856-b1b6-1172db2e8172 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8cbc03dc-b1bc-422b-8a26-c1375e5a3c72", "tag": "b013b704-2ff3-4eb1-a130-54222bd18fef"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:44.357100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e3117b77-83a7-4b65-8741-2e567c861e21 req-d781b09a-d906-4856-b1b6-1172db2e8172 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:44.361689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e3117b77-83a7-4b65-8741-2e567c861e21 req-d781b09a-d906-4856-b1b6-1172db2e8172 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:44.362744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e3117b77-83a7-4b65-8741-2e567c861e21 req-d781b09a-d906-4856-b1b6-1172db2e8172 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:44.391616 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e3117b77-83a7-4b65-8741-2e567c861e21 req-d781b09a-d906-4856-b1b6-1172db2e8172 service nova] Creating event network-changed:b013b704-2ff3-4eb1-a130-54222bd18fef for instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 on nch2-master-1 Oct 15 21:12:44.399740 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e3117b77-83a7-4b65-8741-2e567c861e21 req-d781b09a-d906-4856-b1b6-1172db2e8172 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065957 Oct 15 21:12:44.402680 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1190/4633] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:12:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:12:54.686157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:54.686923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=25,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:54.914437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerTagsTestJSON-server-1764062205", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "e7b54e3c-f8cc-4776-8590-4099717a8a96"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:12:55.024570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] validate_networks() for [('e7b54e3c-f8cc-4776-8590-4099717a8a96', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:12:56.778849 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:12:56.779524 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:12:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 15 21:12:56.970322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:12:56.971107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:12:56.972043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:12:56.972671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:12:56.973666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:12:56.992834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:56.996533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:56.998329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:56.999468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:57.000148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:57.000796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:57.062213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Getting quotas for project 0be4ac2274794d4392c4f08bc0d2395e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:12:57.076944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Getting quotas for user ca0dba28facd46128c0015cbdf7edd67 and project 0be4ac2274794d4392c4f08bc0d2395e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:12:57.121152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:12:57.123965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:12:57.271708 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70005b2f-c388-4924-9f08-c2806f50e8c2 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.26 time: 2.592969 Oct 15 21:12:57.273935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3444/4634] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Oct 15 21:12:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 2597 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 15 21:12:57.297929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d56824dd-5bd1-4133-82a9-029582611560 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:57.306450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d56824dd-5bd1-4133-82a9-029582611560 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:57.312785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=9 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:57.327047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=23 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:12:57.328263 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:12:57.329837 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:12:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e (ip 10.222.0.16) !!! Oct 15 21:12:57.336300 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:12:57.336493 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:12:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e (ip 10.222.0.16) !!! Oct 15 21:12:57.736805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d56824dd-5bd1-4133-82a9-029582611560 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:57.745583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d56824dd-5bd1-4133-82a9-029582611560 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:57.747157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d56824dd-5bd1-4133-82a9-029582611560 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:57.756935 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d56824dd-5bd1-4133-82a9-029582611560 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1262 microversion: 2.26 time: 0.476106 Oct 15 21:12:57.758256 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1191/4635] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:12:57 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1262 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:12:58.791106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:12:58.792397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:12:58.805858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:58.806881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:58.807666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:58.808630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:12:58.809493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:12:58.810185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:12:58.992013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:59.008724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:59.016363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:12:59.019574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c2bf1e2-56b2-4381-8295-c15afec54bea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1314 microversion: 2.26 time: 0.234068 Oct 15 21:12:59.020770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3445/4636] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:12:58 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:13:00.046610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:00.048319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:00.055587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:00.056518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:00.057414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:00.058519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:00.059167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:00.059768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:00.546805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:00.609285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:00.618259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:00.621902 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a2b8670d-ceba-485c-adb7-8600a5c0aaa7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1320 microversion: 2.26 time: 0.579422 Oct 15 21:13:00.627394 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1192/4637] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:00 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1320 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:01.664192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:01.664750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:01.680657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:01.681493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:01.682172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:01.683162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:01.683890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:01.684568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:02.491591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:02.501680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:02.508143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:02.518086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e89d059f-b028-4459-ad50-22660542c667 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1320 microversion: 2.26 time: 0.858260 Oct 15 21:13:02.520558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3446/4638] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:01 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1320 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:03.552965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:03.555050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:03.562463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:03.564254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:03.564969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:03.567349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:03.568208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:03.568921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:03.768419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:03.781111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:03.786494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:03.788033 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8d96f8f3-b176-4f01-9391-4f111f7583ea tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1320 microversion: 2.26 time: 0.240707 Oct 15 21:13:03.788880 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1193/4639] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:03 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1320 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:04.817799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:04.820208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:04.841193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:04.841701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:04.844778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:04.845895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:04.846371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:04.847554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:05.887325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:05.903409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:05.905840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:05.910022 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9137b103-6c2f-46e0-8252-a9f5a33d3f69 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 1.097670 Oct 15 21:13:05.911841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3447/4640] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:04 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 1101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:06.941464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:06.942082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:06.955743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:06.956514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:06.957179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:06.958083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:06.960357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:06.961644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:07.741686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:07.942624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:07.966616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:07.971142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:07.973853 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7b5e2855-e80d-4828-9ed4-79d0542d4d5c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 1.037012 Oct 15 21:13:07.975048 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1194/4641] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:06 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 1039 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:09.003115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:09.003759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:09.017262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:09.018112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:09.024992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:09.025588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:09.026124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:09.026839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:09.027309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:09.034687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:10.101683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d27b4df2-4887-4ed6-af4a-5046eccb68cf req-807c8233-6a9f-4fa2-a534-78475f3416f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e", "tag": "6384c608-f4ac-48d8-89c8-9495330fed94"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:10.119327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d27b4df2-4887-4ed6-af4a-5046eccb68cf req-807c8233-6a9f-4fa2-a534-78475f3416f0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:10.120048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d27b4df2-4887-4ed6-af4a-5046eccb68cf req-807c8233-6a9f-4fa2-a534-78475f3416f0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:10.120668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d27b4df2-4887-4ed6-af4a-5046eccb68cf req-807c8233-6a9f-4fa2-a534-78475f3416f0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:10.143162 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d27b4df2-4887-4ed6-af4a-5046eccb68cf req-807c8233-6a9f-4fa2-a534-78475f3416f0 service nova] Creating event network-changed:6384c608-f4ac-48d8-89c8-9495330fed94 for instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e on nch2-master-1 Oct 15 21:13:10.148443 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d27b4df2-4887-4ed6-af4a-5046eccb68cf req-807c8233-6a9f-4fa2-a534-78475f3416f0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050849 Oct 15 21:13:10.149439 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1195/4642] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:10.215936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:10.229165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:10.238788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:10.242313 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-602d781b-d7a4-494e-84a2-75a78a57adf1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 1.244007 Oct 15 21:13:10.243380 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3448/4643] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:08 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 1246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:11.272220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:11.272932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:11.287834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:11.287834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:11.288109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:11.288746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:11.289241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:11.289720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:11.921332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:11.935145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:11.939159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:11.941775 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2cae286b-6521-4357-907b-faff77e7da62 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 0.675999 Oct 15 21:13:11.963408 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1196/4644] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:11 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:12.969811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:12.970447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:12.983969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:12.985545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:12.988273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:12.989488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:12.989978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:12.993792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:13.761992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:13.780908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:13.783594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:13.786382 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37dab5f2-b051-4b69-becc-e93f3d82b825 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 0.821130 Oct 15 21:13:13.787718 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3449/4645] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:12 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 823 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:13.886796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:13.887375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:13.900200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:13.901030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:13.901491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:13.905956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:13.907002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:13.907800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:14.692892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:14.759531 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4b234bc0-fe04-4c1c-b5b4-f4d059404841 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.874408 Oct 15 21:13:14.762641 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1197/4646] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:13 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 880 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:14.791515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:14.792158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:14.804389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:14.805798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:14.806262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:14.808173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:14.808716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:14.811259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:14.816477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:14.817338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:14.828762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:14.829823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:14.830527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:14.831956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:14.832631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:14.833432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:15.824191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:15.865865 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-abc1a630-bea8-4b33-91b1-46d5ec3937a0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.078092 Oct 15 21:13:15.889720 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3450/4647] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:14 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1097 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:15.898574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:15.899202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:15.914780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:15.915292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:15.915739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:15.916454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:15.922408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:15.923188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:15.931391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:15.945180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:15.946854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:15.954860 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4df6b04-4165-4562-a024-8fb404b0a062 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 1.144073 Oct 15 21:13:15.955966 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1198/4648] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:14 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 1146 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:16.457090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:16.474770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ebebec8-3432-4c76-be35-bed6e8cfe716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.581578 Oct 15 21:13:16.485431 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3451/4649] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:15 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:16.500108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:16.500897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:16.507983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:16.508564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:16.509126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:16.510335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:16.511152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:16.511866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:16.986437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:16.988127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:16.996263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:16.997442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:16.998137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:17.000656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:17.001415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:17.002060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:17.053529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:17.063643 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1521f53b-f682-45bc-94ed-13d09143ed8e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.568422 Oct 15 21:13:17.081065 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1199/4650] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:16 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:17.241133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-920752135", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:17.325185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] validate_networks() for [('c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:13:18.295593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:18.308640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:18.310483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:18.311857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-54ae807b-0ca0-4a82-9d14-ad657de0c328 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 1.331681 Oct 15 21:13:18.329272 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3452/4651] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:16 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 1347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:18.335476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ecff0490-862b-435e-8bad-205ac856dbef req-a3f54841-d01a-4803-8e5d-b2b906e38840 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e", "name": "network-vif-plugged", "status": "completed", "tag": "6384c608-f4ac-48d8-89c8-9495330fed94"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:18.351103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ecff0490-862b-435e-8bad-205ac856dbef req-a3f54841-d01a-4803-8e5d-b2b906e38840 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:18.351965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ecff0490-862b-435e-8bad-205ac856dbef req-a3f54841-d01a-4803-8e5d-b2b906e38840 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:18.352919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ecff0490-862b-435e-8bad-205ac856dbef req-a3f54841-d01a-4803-8e5d-b2b906e38840 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:18.382970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ecff0490-862b-435e-8bad-205ac856dbef req-a3f54841-d01a-4803-8e5d-b2b906e38840 service nova] Creating event network-vif-plugged:6384c608-f4ac-48d8-89c8-9495330fed94 for instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e on nch2-master-1 Oct 15 21:13:18.394075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ecff0490-862b-435e-8bad-205ac856dbef req-a3f54841-d01a-4803-8e5d-b2b906e38840 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062802 Oct 15 21:13:18.396055 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3453/4652] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:19.302744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:13:19.303854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:13:19.305440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:13:19.306236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:13:19.308298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:13:19.337560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:19.340342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:19.341172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:19.342977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:19.344967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:19.358941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:19.360127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:19.360575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:19.376986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:19.378651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:19.379591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:19.380600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:19.381301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:19.381917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:19.407706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:13:19.417083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for user 1aadd87b97c247cb95321664ba0c32a9 and project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:13:19.448600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:13:19.451408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:13:19.530237 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0edd99c0-d648-452d-b43c-5cd9ac689a05 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.292781 Oct 15 21:13:19.531860 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1200/4653] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:13:17 2022] POST /compute/v2.1/servers => generated 380 bytes in 2295 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:13:19.543573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5ab90fb-d723-4fb7-8da5-f20d1000abd8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:19.544216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5ab90fb-d723-4fb7-8da5-f20d1000abd8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:19.694964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5ab90fb-d723-4fb7-8da5-f20d1000abd8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:19.709831 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5ab90fb-d723-4fb7-8da5-f20d1000abd8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1214 microversion: 2.1 time: 0.170578 Oct 15 21:13:19.712271 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1201/4654] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:19 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1214 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:19.753322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:19.779087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:19.785445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:19.797487 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-752c03a6-6eda-484f-a8ee-2b849d45de09 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1362 microversion: 2.26 time: 0.449028 Oct 15 21:13:19.800467 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3454/4655] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:19 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1362 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:20.741953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:20.743293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:20.751076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:20.751584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:20.752029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:20.753183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:20.753882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:20.754489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:20.828609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:20.829371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:20.840203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:20.851723 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f4b6cf7-dd86-4f2d-bc34-d1fb3d42e3f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1210 microversion: 2.1 time: 0.114724 Oct 15 21:13:20.865984 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1202/4656] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:20 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1210 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:20.873976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:20.874837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:20.875567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:20.876800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:20.878059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:20.878719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.200829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:21.213880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:21.218794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:21.220110 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e5cd507-e68a-49ef-ad34-36ddf3299d1e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1546 microversion: 2.26 time: 0.396025 Oct 15 21:13:21.220877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3455/4657] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:20 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1546 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 15 21:13:21.252159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eee4ee0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.266233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.267088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.267935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.269124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:21.275966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.276825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.277473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.279048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.279890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.280746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.316048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.316641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.317147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.335640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.336238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.336550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.395525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:21.473515 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e549e8f8-2ca9-4746-a7d0-0c3a706cb932 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-797782988" status: 201 len: 0 microversion: 2.26 time: 0.228328 Oct 15 21:13:21.485857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c01e3cd-45cf-4cc1-a254-8b0ed8f94d06 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74ef9e940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.491917 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1203/4658] 10.222.0.16 () {62 vars in 1377 bytes} [Sat Oct 15 21:13:21 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-797782988 => generated 0 bytes in 245 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Oct 15 21:13:21.505972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c01e3cd-45cf-4cc1-a254-8b0ed8f94d06 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.507918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c01e3cd-45cf-4cc1-a254-8b0ed8f94d06 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.508385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c01e3cd-45cf-4cc1-a254-8b0ed8f94d06 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.552923 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c01e3cd-45cf-4cc1-a254-8b0ed8f94d06 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-797782988" status: 204 len: 0 microversion: 2.26 time: 0.070451 Oct 15 21:13:21.556024 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3456/4659] 10.222.0.16 () {60 vars in 1358 bytes} [Sat Oct 15 21:13:21 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-797782988 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:21.564441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eee4280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.573764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.574629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.575340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.592897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:21.602308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.605019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.606935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.608096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.609559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.610557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.661572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.662076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.662510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.693145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:21.732462 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1213a584-5928-4227-8b44-382e2d2bc0bc tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 204 len: 0 microversion: 2.26 time: 0.172322 Oct 15 21:13:21.733427 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1204/4660] 10.222.0.16 () {60 vars in 1295 bytes} [Sat Oct 15 21:13:21 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 0 bytes in 174 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:21.744023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9671d1f-65fd-4a99-bb3b-3ec29d47dd7c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74ee5c040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.764540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9671d1f-65fd-4a99-bb3b-3ec29d47dd7c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.770465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9671d1f-65fd-4a99-bb3b-3ec29d47dd7c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.770961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9671d1f-65fd-4a99-bb3b-3ec29d47dd7c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.798128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9671d1f-65fd-4a99-bb3b-3ec29d47dd7c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 200 len: 12 microversion: 2.26 time: 0.058628 Oct 15 21:13:21.804285 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3457/4661] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 21:13:21 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 12 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:21.810061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74ee26160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.831318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.831318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.831318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.831318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:21.846951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.847442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.848099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.848993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.849639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.850233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.879565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.880196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:21.883314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.884041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.884683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.894733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.895530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.896056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.896395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.896574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.897617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.900245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.902354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.902903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.936718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:21.957011 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c686b1e-06b7-41f9-b708-e0cc33f32c4e tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-369098813" status: 201 len: 0 microversion: 2.26 time: 0.151807 Oct 15 21:13:21.963230 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1205/4662] 10.222.0.16 () {62 vars in 1377 bytes} [Sat Oct 15 21:13:21 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-369098813 => generated 0 bytes in 159 msecs (HTTP/1.1 201) 10 headers in 485 bytes (2 switches on core 0) Oct 15 21:13:21.969105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d26862d3-a706-4e67-a691-36433a9421f1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74f05ff70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:21.977315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d26862d3-a706-4e67-a691-36433a9421f1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:21.978154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d26862d3-a706-4e67-a691-36433a9421f1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:21.978820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d26862d3-a706-4e67-a691-36433a9421f1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:21.988935 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d26862d3-a706-4e67-a691-36433a9421f1 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 200 len: 35 microversion: 2.26 time: 0.024275 Oct 15 21:13:21.995588 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1206/4663] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 21:13:21 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 35 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:22.002205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eee43a0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.011497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.012244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.012952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.013776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:22.020827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.021694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.022381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.023320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.024026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.024673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.051965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=93,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=2,nova_cell1:RELEASE=2,nova_cell1:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:22.055848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.056305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:22.057010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.057714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.072907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a98e2663-0294-4363-ada2-8a1b73bbd805 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1272 microversion: 2.1 time: 0.198392 Oct 15 21:13:22.074015 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3458/4664] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:21 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1272 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:22.092387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:22.116528 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-93ca7328-faf6-4452-b921-51a5194fea6b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-369098813" status: 204 len: 0 microversion: 2.26 time: 0.119304 Oct 15 21:13:22.121689 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1207/4665] 10.222.0.16 () {60 vars in 1361 bytes} [Sat Oct 15 21:13:21 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-369098813 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:22.128053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1d972e8-9a47-402a-acf3-18ccbd0fe4aa tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eddc310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.135942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d972e8-9a47-402a-acf3-18ccbd0fe4aa tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.136770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d972e8-9a47-402a-acf3-18ccbd0fe4aa tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.137627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d972e8-9a47-402a-acf3-18ccbd0fe4aa tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.148132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1d972e8-9a47-402a-acf3-18ccbd0fe4aa tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 200 len: 12 microversion: 2.26 time: 0.025076 Oct 15 21:13:22.154276 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3459/4666] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 21:13:22 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 12 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:22.161613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eee44c0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.172081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.173360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.174180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.174897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:22.181878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.182804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.183498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.184558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.185273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.185945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.228435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.229393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.230061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.247144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:22.267048 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b3e8a3a-8564-4c45-8e51-739e82d4873c tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 204 len: 0 microversion: 2.26 time: 0.111618 Oct 15 21:13:22.268378 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1208/4667] 10.222.0.16 () {60 vars in 1295 bytes} [Sat Oct 15 21:13:22 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:22.278288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74e836dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.292896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.292896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.293313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.293770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:22.313758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.314277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.314727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.318081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.321226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.321877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.371699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.373361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.374148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.387274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.388418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.389751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.432189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:22.461204 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd840c60-ca67-491e-9ca2-3b8dbe698ced tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-167696578" status: 201 len: 0 microversion: 2.26 time: 0.186990 Oct 15 21:13:22.463371 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3460/4668] 10.222.0.16 () {62 vars in 1377 bytes} [Sat Oct 15 21:13:22 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-167696578 => generated 0 bytes in 189 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Oct 15 21:13:22.475031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eeb9f70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.490501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.492365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.493870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.494629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:22.502529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.503652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.504325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.505354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.506371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.507753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.568486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.570324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.571151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.586836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.587827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.589341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.636448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:22.660847 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d8ad450-1d61-4c8d-8bf5-1ca1b097d183 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-2011154873" status: 201 len: 0 microversion: 2.26 time: 0.192064 Oct 15 21:13:22.661957 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1209/4669] 10.222.0.16 () {62 vars in 1380 bytes} [Sat Oct 15 21:13:22 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-2011154873 => generated 0 bytes in 194 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Oct 15 21:13:22.675789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74ee425e0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.688178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.689267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.691163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.692461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:22.699667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.705512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.706625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.707621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.708321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.709709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.740255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=1,nova_cell1:RELEASE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:22.756742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.757850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.758318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.796265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:22.838538 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ccd44d0-b75a-48da-9ec3-2c8bc84b14c6 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 204 len: 0 microversion: 2.26 time: 0.171343 Oct 15 21:13:22.839402 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3461/4670] 10.222.0.16 () {60 vars in 1295 bytes} [Sat Oct 15 21:13:22 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 0 bytes in 173 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:22.853362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5312fbcb-813c-4b92-bac4-e79ff8d09ca4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eee4550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.878059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5312fbcb-813c-4b92-bac4-e79ff8d09ca4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.880002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5312fbcb-813c-4b92-bac4-e79ff8d09ca4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.880610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5312fbcb-813c-4b92-bac4-e79ff8d09ca4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.897003 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5312fbcb-813c-4b92-bac4-e79ff8d09ca4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 200 len: 12 microversion: 2.26 time: 0.052310 Oct 15 21:13:22.903040 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1210/4671] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 21:13:22 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 12 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:22.912377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe754590670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:22.929683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.932939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.933996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.934706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:22.944961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.945887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.946640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.948368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.949175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.950946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:22.993744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:22.994309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:22.994769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.029624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:23.079342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22cb9fd3-017f-48b5-9616-5499d7b33be4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 204 len: 0 microversion: 2.26 time: 0.173009 Oct 15 21:13:23.080909 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3462/4672] 10.222.0.16 () {60 vars in 1295 bytes} [Sat Oct 15 21:13:22 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 0 bytes in 177 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:23.094456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eef91f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:23.110818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:23.111368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.111862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.113202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:23.118186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.118936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:23.135069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.135839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.138643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.139172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.139601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.140933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.141321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.141855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.142000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.142669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.143145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.143588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.220464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.221018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.221489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.233001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.234110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.235167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.266574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:23.305437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c54a8cbe-ff1e-4876-b63b-34a0b25bbb87 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1272 microversion: 2.1 time: 0.197359 Oct 15 21:13:23.309756 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3463/4673] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:23 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1272 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:23.322335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:23.352605 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cb46a17a-7fd7-4a61-a2f0-3fa6023080ff tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-1784920981" status: 201 len: 0 microversion: 2.26 time: 0.263333 Oct 15 21:13:23.354107 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1211/4674] 10.222.0.16 () {62 vars in 1380 bytes} [Sat Oct 15 21:13:23 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-1784920981 => generated 0 bytes in 266 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Oct 15 21:13:23.368756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74ed4b040>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:23.388206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.389319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.389983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.391015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:23.401339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.402782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.403640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.404668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.406138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.406830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.481438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.482022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.482542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.493373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.494007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.494522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.527440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:23.552347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9c588703-2026-40ae-8fcd-34af238ccfc7 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-852415557" status: 201 len: 0 microversion: 2.26 time: 0.190983 Oct 15 21:13:23.563473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3464/4675] 10.222.0.16 () {62 vars in 1377 bytes} [Sat Oct 15 21:13:23 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags/tempest-tag-852415557 => generated 0 bytes in 204 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Oct 15 21:13:23.565956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Action: 'update_all', calling method: .version_select at 0x7fe74ee848b0>, body: {"tags": ["tempest-tag-58847197", "tempest-tag-1616515524"]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:23.584198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.585285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.585751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.586771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:23.594522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.595034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.595861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.596608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.597170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.599076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.653133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.653802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.654293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.692523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:23.720516 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-419c62d6-9d1c-4d5d-807c-704b8ba4cb36 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 200 len: 60 microversion: 2.26 time: 0.160975 Oct 15 21:13:23.721768 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1212/4676] 10.222.0.16 () {62 vars in 1312 bytes} [Sat Oct 15 21:13:23 2022] PUT /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 60 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:23.733404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-232d6e24-5008-46a0-9260-33bb40b7c24f tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe7583efb80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:23.771155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-232d6e24-5008-46a0-9260-33bb40b7c24f tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.773093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-232d6e24-5008-46a0-9260-33bb40b7c24f tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.776207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-232d6e24-5008-46a0-9260-33bb40b7c24f tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.799221 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-232d6e24-5008-46a0-9260-33bb40b7c24f tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 200 len: 60 microversion: 2.26 time: 0.071162 Oct 15 21:13:23.805818 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3465/4677] 10.222.0.16 () {60 vars in 1292 bytes} [Sat Oct 15 21:13:23 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 60 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:23.817899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '.version_select at 0x7fe74eef98b0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:23.832414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.833541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.833748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.834481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:23.864395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.866085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.868617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.871703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.872450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.873205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.916797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:23.918219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:23.921205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:23.944891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'flavor' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:23.991331 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-159d6a83-a2c8-4b90-9bea-6e1148eeba39 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags" status: 204 len: 0 microversion: 2.26 time: 0.183616 Oct 15 21:13:23.994201 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1213/4678] 10.222.0.16 () {60 vars in 1295 bytes} [Sat Oct 15 21:13:23 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e/tags => generated 0 bytes in 187 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:24.007066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:24.008734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:24.050871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:24.052538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:24.053278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:24.055324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:24.055977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:24.056656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:24.103570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:13:24.286158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2a741dbf-e7e1-4300-864c-051a70f98743 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 204 len: 0 microversion: 2.26 time: 0.285470 Oct 15 21:13:24.286972 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3466/4679] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Oct 15 21:13:23 2022] DELETE /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 15 21:13:24.298770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:24.301370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:24.321123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:24.323198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:24.324911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:24.327034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:24.328922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:24.331675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:24.340508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:24.341601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:24.364988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:24.365891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:24.366999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:24.372502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:24.372841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:24.373683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:25.114871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:25.132001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:25.134387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:25.135728 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2aabc674-dc5a-486a-9bc2-8857381ab901 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1552 microversion: 2.26 time: 0.842504 Oct 15 21:13:25.155237 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1214/4680] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:24 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1552 bytes in 845 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:25.158646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:25.167596 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-868022b1-1561-473f-a9bb-ef4a3c00b428 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 0.833456 Oct 15 21:13:25.181858 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3467/4681] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:24 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:26.164286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:26.165315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:26.181509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:26.182117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:26.182651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:26.183460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:26.184315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:26.185575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:26.192986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:26.194031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:26.196985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:26.215133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:26.221604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:26.222162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:26.222994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:26.223440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:26.223887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:27.321864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:27.343604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:27.347824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:27.349902 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-97484d98-8a12-4290-9763-9ad7338d5d28 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1552 microversion: 2.26 time: 1.190579 Oct 15 21:13:27.350473 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1215/4682] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:26 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1552 bytes in 1192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:27.378041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:27.404186 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2e2048d-f1bc-4444-96df-29d1d6101ab4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 1.214994 Oct 15 21:13:27.405102 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3468/4683] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:26 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 1218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:27.846103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c81eeb5d-8295-4887-a1c9-45a26bfebf6e req-27dc171c-5a07-4fd1-b7d9-4087e1f50641 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e", "name": "network-vif-deleted", "tag": "6384c608-f4ac-48d8-89c8-9495330fed94"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:27.896390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:27.907261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c81eeb5d-8295-4887-a1c9-45a26bfebf6e req-27dc171c-5a07-4fd1-b7d9-4087e1f50641 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:27.908111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c81eeb5d-8295-4887-a1c9-45a26bfebf6e req-27dc171c-5a07-4fd1-b7d9-4087e1f50641 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:27.910918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c81eeb5d-8295-4887-a1c9-45a26bfebf6e req-27dc171c-5a07-4fd1-b7d9-4087e1f50641 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:27.931416 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c81eeb5d-8295-4887-a1c9-45a26bfebf6e req-27dc171c-5a07-4fd1-b7d9-4087e1f50641 service nova] Creating event network-vif-deleted:6384c608-f4ac-48d8-89c8-9495330fed94 for instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e on nch2-master-1 Oct 15 21:13:27.946480 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c81eeb5d-8295-4887-a1c9-45a26bfebf6e req-27dc171c-5a07-4fd1-b7d9-4087e1f50641 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104834 Oct 15 21:13:27.947777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:27.948498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:27.950604 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1216/4684] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:27.956456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:27.957404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:27.958107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:27.970868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:27.972062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:27.972776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.090006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lazy-loading 'services' on Instance uuid 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:28.192870 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0f462f2-7b17-41a3-9e44-8863edad9db0 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments/83707e8c-d087-4efb-9ab5-b1c112500f85" status: 202 len: 0 microversion: 2.1 time: 0.248730 Oct 15 21:13:28.194311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3469/4685] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:13:27 2022] DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments/83707e8c-d087-4efb-9ab5-b1c112500f85 => generated 0 bytes in 251 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:13:28.204450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:28.207958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:28.220291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.221292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.222178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.223139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.223863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.224556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.283457 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a546026a-603c-4c02-8b73-779291b1e70d tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084482 Oct 15 21:13:28.284243 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1217/4686] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:13:28 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:28.391355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:28.392259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:28.409737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.417982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.418710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.419991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.420678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.424837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.428627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:28.429344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:28.445481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.464248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.466954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.471847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.476185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.477425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.516313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:28.517728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:28.518263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:28.520758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lazy-loading 'fault' on Instance uuid f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:13:28.568816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=89,nova_cell1:DELETE=4,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=3,nova_cell1:RELEASE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:28.634554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:28.648902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:28.651838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca0dba28facd46128c0015cbdf7edd67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be4ac2274794d4392c4f08bc0d2395e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:28.658626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7d4cbda8-8ece-413a-a7de-b2e7aa3b9bb4 tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 200 len: 1352 microversion: 2.26 time: 0.276782 Oct 15 21:13:28.659495 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3470/4687] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:28 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 1352 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 15 21:13:29.158017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:29.169252 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-eb7c0fae-3a07-45ba-96f5-0a2b63dc9869 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 0.744283 Oct 15 21:13:29.170438 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1218/4688] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:28 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:29.296719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:29.299118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:29.316901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:29.317470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:29.318095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:29.319026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:29.319497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:29.320126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:29.366641 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc3a55d7-521a-4859-96b2-636de2361690 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074378 Oct 15 21:13:29.367787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3471/4689] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:13:29 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:29.690288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:29.690949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] [instance: f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:29.695824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:29.701659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:29.704066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:29.704797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:29.705275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:29.706246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:29.706926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:29.707396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:29.727182 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] HTTP exception thrown: Instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e could not be found. Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Request method failure captured: Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e HTTP/1.1 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyJWI1lAvFvDyz4YflPT-yu5_pzSNSGBIZ0sOEzKVRpuUknE1P_NaRZa9huIRsQms0glTZ9N6ZstPNw6WZhuo4USIC87YmbfRfHjej7jP6vW7fciCxJCSUNhNjbPYmavVGBmFPcY3Qd5II-9efkqC5iGvK-KA-0aZZkZUcObinmXE5I Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Openstack-Nova-Api-Version: 2.26 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 0be4ac2274794d4392c4f08bc0d2395e Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerTagsTestJSON-527945825 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0be4ac2274794d4392c4f08bc0d2395e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0be4ac2274794d4392c4f08bc0d2395e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0be4ac2274794d4392c4f08bc0d2395e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0be4ac2274794d4392c4f08bc0d2395e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerTagsTestJSON-527945825 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 0be4ac2274794d4392c4f08bc0d2395e Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerTagsTestJSON-527945825 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerTagsTestJSON-527945825-project Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: ca0dba28facd46128c0015cbdf7edd67 Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerTagsTestJSON-527945825-project Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e could not be found. Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e'} Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e could not be found. Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e could not be found. Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:29.729190 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e could not be found. Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] Returning 404 to user: Instance f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d45eaac0-f7d1-484f-9625-60039284e66b tempest-ServerTagsTestJSON-527945825 tempest-ServerTagsTestJSON-527945825-project] 10.222.0.16 "GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e" status: 404 len: 111 microversion: 2.26 time: 0.045549 Oct 15 21:13:29.735525 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1219/4690] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Oct 15 21:13:29 2022] GET /compute/v2.1/servers/f7cd68b2-5b49-4ad9-a0d4-f89b17de6b2e => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 15 21:13:30.197296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:30.199144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:30.208550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.209202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.209740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.210793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.211515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.212048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.354065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5189b11a-21b3-4ee8-9d61-0bfd2fda7400 req-780bd851-6db9-4da7-bc01-cb30ba24a002 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa", "tag": "f1ee217d-8ce5-4ef4-aa47-ea76d019efff"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:30.365731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5189b11a-21b3-4ee8-9d61-0bfd2fda7400 req-780bd851-6db9-4da7-bc01-cb30ba24a002 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.366492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5189b11a-21b3-4ee8-9d61-0bfd2fda7400 req-780bd851-6db9-4da7-bc01-cb30ba24a002 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.366922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5189b11a-21b3-4ee8-9d61-0bfd2fda7400 req-780bd851-6db9-4da7-bc01-cb30ba24a002 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.383316 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5189b11a-21b3-4ee8-9d61-0bfd2fda7400 req-780bd851-6db9-4da7-bc01-cb30ba24a002 service nova] Creating event network-changed:f1ee217d-8ce5-4ef4-aa47-ea76d019efff for instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa on nch1-master-1 Oct 15 21:13:30.389794 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5189b11a-21b3-4ee8-9d61-0bfd2fda7400 req-780bd851-6db9-4da7-bc01-cb30ba24a002 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039186 Oct 15 21:13:30.391198 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1220/4691] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:13:30.397185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:30.399637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:30.406157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.406930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.407580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.408264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.412400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.413671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.452118 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aae66fb7-4861-4337-9dc3-5d5c77826342 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.059357 Oct 15 21:13:30.453490 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1221/4692] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:13:30 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72/os-volume_attachments => generated 25 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:13:30.534629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:30.536827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:30.544471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.545401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.546220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.547444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.548075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.548704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.580997 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Cannot 'reboot' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action HTTP/1.1 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 28 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: {"reboot": {"type": "HARD"}} Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: exception: Cannot 'reboot' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '76691e91-f327-49c4-88ce-d6932390510f', 'body': {'reboot': {'type': 'HARD'}}} Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1036, in _action_reboot Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.compute_api.reboot(context, instance, reboot_type) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3552, in reboot Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._hard_reboot(context, instance) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 76691e91-f327-49c4-88ce-d6932390510f in vm_state rescued. Cannot _hard_reboot while the instance is in this state. Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1040, in _action_reboot Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:13:30.582760 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'reboot' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'reboot' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 409 to user: Cannot 'reboot' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ebdb236e-4b4d-4501-86aa-498e2a90bc5f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action" status: 409 len: 145 microversion: 2.1 time: 0.054779 Oct 15 21:13:30.587289 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1222/4693] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:13:30 2022] POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action => generated 145 bytes in 56 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:13:30.595256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:30.598332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:30.604549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.605062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.605853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.606732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.607363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.607784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.642000 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Cannot 'rebuild' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action HTTP/1.1 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 65 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: exception: Cannot 'rebuild' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '76691e91-f327-49c4-88ce-d6932390510f', 'body': {'rebuild': {'imageRef': '77903f56-0555-4c00-82ae-bbf42e11a2a9'}}} Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1223, in _action_rebuild Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.compute_api.rebuild(context, Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 76691e91-f327-49c4-88ce-d6932390510f in vm_state rescued. Cannot rebuild while the instance is in this state. Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:13:30.643976 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1231, in _action_rebuild Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'rebuild' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'rebuild' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 409 to user: Cannot 'rebuild' instance 76691e91-f327-49c4-88ce-d6932390510f while it is in vm_state rescued {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:13:30.648457 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7f473707-7da5-45d1-a340-9836b0dbd744 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action" status: 409 len: 146 microversion: 2.1 time: 0.055417 Oct 15 21:13:30.654890 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1223/4694] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:13:30 2022] POST /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f/action => generated 146 bytes in 61 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:13:30.657344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:30.658099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:30.664972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.665791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.666295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.667063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.667593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.668075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.699954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:13:30.774882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:30.788064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8fa78c52-53b8-4f0e-80e0-707e02306b37 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 0.597470 Oct 15 21:13:30.790268 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3472/4695] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:30 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:30.834516 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c8f456e9-7c88-4b55-9768-36f037214716 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 204 len: 0 microversion: 2.1 time: 0.181638 Oct 15 21:13:30.835243 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1224/4696] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:13:30 2022] DELETE /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:13:30.845274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:30.845919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:30.853534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.854164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.854703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.855684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:30.856387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:30.856913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:30.865938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=92,nova_cell1:DELETE=2,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=3,nova_cell1:RELEASE=1,nova_cell1:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:31.539261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:31.556070 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d588ea0-5ef7-4fea-be4d-26c6712d401f tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1689 microversion: 2.1 time: 0.715282 Oct 15 21:13:31.557431 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3473/4697] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:30 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1689 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:31.815544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:31.817579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:31.846003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:31.846846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:31.847471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:31.848699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:31.849402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:31.850101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:32.475271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:32.485345 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fe7d065b-07da-430f-bc49-80fc8853fe81 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 0.676461 Oct 15 21:13:32.486783 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1225/4698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:31 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:32.580722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:32.581311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:32.590098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:32.590886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:32.591670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:32.593267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:32.593991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:32.594616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:33.337690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:33.353799 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4cd4d59-2b5b-4de3-9999-c5fe7c045740 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1689 microversion: 2.1 time: 0.777675 Oct 15 21:13:33.354698 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3474/4699] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:32 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1689 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:33.510563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:33.511809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:33.519064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:33.519558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:33.520205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:33.520924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:33.521585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:33.522020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:33.995524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:34.004555 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-97d79223-93e0-4e77-afb6-8075010244ed tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 0.497719 Oct 15 21:13:34.005164 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1226/4700] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:33 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:34.384248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:34.388882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:34.401894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:34.402588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:34.403337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:34.405012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:34.406479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:34.407236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:34.943938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:34.956825 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-401be1a2-27c2-4e63-8774-c60d30bc2158 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1689 microversion: 2.1 time: 0.579096 Oct 15 21:13:34.993403 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3475/4701] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:34 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1689 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:35.011806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-1e192844-8c1c-4f13-b46f-c37cc8eb73e1 req-12b2d423-d636-4614-a6bc-8cf79ea4f586 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa", "name": "network-vif-plugged", "status": "completed", "tag": "f1ee217d-8ce5-4ef4-aa47-ea76d019efff"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:35.035651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:35.035883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:35.049689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1e192844-8c1c-4f13-b46f-c37cc8eb73e1 req-12b2d423-d636-4614-a6bc-8cf79ea4f586 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:35.052119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1e192844-8c1c-4f13-b46f-c37cc8eb73e1 req-12b2d423-d636-4614-a6bc-8cf79ea4f586 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:35.053461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1e192844-8c1c-4f13-b46f-c37cc8eb73e1 req-12b2d423-d636-4614-a6bc-8cf79ea4f586 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:35.055959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:35.056474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:35.058376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:35.061733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:35.062290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:35.064525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:35.089504 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-1e192844-8c1c-4f13-b46f-c37cc8eb73e1 req-12b2d423-d636-4614-a6bc-8cf79ea4f586 service nova] Creating event network-vif-plugged:f1ee217d-8ce5-4ef4-aa47-ea76d019efff for instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa on nch1-master-1 Oct 15 21:13:35.136288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-1e192844-8c1c-4f13-b46f-c37cc8eb73e1 req-12b2d423-d636-4614-a6bc-8cf79ea4f586 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.132827 Oct 15 21:13:35.142306 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1227/4702] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:35.992157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:35.993495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:36.006742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:36.007909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:36.008642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:36.009743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:36.010441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:36.011094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:36.183278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:36.209248 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce7ecd9d-f1fa-4509-b971-958b065ddd8b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1314 microversion: 2.1 time: 1.180739 Oct 15 21:13:36.209852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3476/4703] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:35 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1314 bytes in 1183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:36.684300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:36.699580 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c3ffbc4-f504-4ccd-ba12-a1738fd79d77 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1689 microversion: 2.1 time: 0.714357 Oct 15 21:13:36.702523 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1228/4704] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:35 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1689 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:37.237168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:37.237168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:37.248555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:37.249622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:37.292168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:37.293375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:37.296263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:37.296996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:37.729986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:37.731237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:37.743222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:37.743638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:37.744075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:37.748132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:37.748614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:37.749697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:37.973798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:37.986534 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78d19c1b-ef86-4234-85e3-2851f9533df9 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1622 microversion: 2.1 time: 0.259955 Oct 15 21:13:37.987887 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1229/4705] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:37 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1622 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:38.085382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:38.098962 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30b3b4f9-2f4e-419e-9dc0-4e0b8735b53e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1502 microversion: 2.1 time: 0.866568 Oct 15 21:13:38.100004 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3477/4706] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:37 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1502 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:38.125305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "default"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:38.126230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:38.133197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:38.134120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:38.134753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:38.135690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:38.136352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:38.136956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:38.744557 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.network.security_group_api [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Removing security group 244087ac-197f-4a3f-8454-ad22462b7a23 from port f1ee217d-8ce5-4ef4-aa47-ea76d019efff Oct 15 21:13:39.011575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:39.012987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:39.022297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:39.023208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:39.023906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:39.024897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:39.026617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:39.027298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:39.157637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:39.167622 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c99f3b9-4231-412c-8f12-d703c6778703 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 200 len: 1622 microversion: 2.1 time: 0.159972 Oct 15 21:13:39.189629 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3478/4707] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:39 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 1622 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:39.196756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e6b60da7-74d3-45c3-92b8-bbaa53002a4c req-e7aeabdb-3829-400f-a1d4-e8206dc34d3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8cbc03dc-b1bc-422b-8a26-c1375e5a3c72", "name": "network-vif-deleted", "tag": "b013b704-2ff3-4eb1-a130-54222bd18fef"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:39.215068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e6b60da7-74d3-45c3-92b8-bbaa53002a4c req-e7aeabdb-3829-400f-a1d4-e8206dc34d3a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:39.216801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e6b60da7-74d3-45c3-92b8-bbaa53002a4c req-e7aeabdb-3829-400f-a1d4-e8206dc34d3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:39.219256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e6b60da7-74d3-45c3-92b8-bbaa53002a4c req-e7aeabdb-3829-400f-a1d4-e8206dc34d3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:39.289489 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e6b60da7-74d3-45c3-92b8-bbaa53002a4c req-e7aeabdb-3829-400f-a1d4-e8206dc34d3a service nova] Creating event network-vif-deleted:b013b704-2ff3-4eb1-a130-54222bd18fef for instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 on nch2-master-1 Oct 15 21:13:39.312615 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e6b60da7-74d3-45c3-92b8-bbaa53002a4c req-e7aeabdb-3829-400f-a1d4-e8206dc34d3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119401 Oct 15 21:13:39.317580 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3479/4708] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:39.924937 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d4b4bfc8-aa59-460a-96f6-e542c916068f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa/action" status: 202 len: 0 microversion: 2.1 time: 1.804670 Oct 15 21:13:39.929368 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1230/4709] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:13:38 2022] POST /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa/action => generated 0 bytes in 1807 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:13:39.936703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:39.937821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:39.945642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:39.946426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:39.947479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:39.948413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:39.949085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:39.949914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:40.073320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:40.089743 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d402e0f-d504-4676-9dd5-5f291b94dbe7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1460 microversion: 2.1 time: 0.158630 Oct 15 21:13:40.090947 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3480/4710] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:39 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1460 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:40.117915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:40.118427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:40.138755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:40.139660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:40.140346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:40.141274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:40.141915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:40.142866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:40.194176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:40.194991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:40.202961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:40.208846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:40.209617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:40.210636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:40.211298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:40.211935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:40.246979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 could not be found. Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 HTTP/1.1 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 could not be found. Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8cbc03dc-b1bc-422b-8a26-c1375e5a3c72'} Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 could not be found. Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 could not be found. Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:40.248771 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 could not be found. Oct 15 21:13:40.256129 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:40.256901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 404 to user: Instance 8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:13:40.257973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0eb3a37e-87f4-41e1-b7f5-54f0c28298fe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72" status: 404 len: 111 microversion: 2.1 time: 0.067829 Oct 15 21:13:40.259011 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3481/4711] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:40 2022] GET /compute/v2.1/servers/8cbc03dc-b1bc-422b-8a26-c1375e5a3c72 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:13:40.268991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-171f7841-dd24-434d-af46-3fca719fe602 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '.version_select at 0x7fe74e7c0ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:40.314821 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-171f7841-dd24-434d-af46-3fca719fe602 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1395606911" status: 202 len: 0 microversion: 2.1 time: 0.051672 Oct 15 21:13:40.315845 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3482/4712] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:13:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1395606911 => generated 0 bytes in 53 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:13:41.195436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:41.211601 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-be0cb2af-b75b-4cff-8795-daa50be87e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.098043 Oct 15 21:13:41.212182 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1231/4713] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:40 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1099 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:41.249645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:41.252391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:41.282840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:41.288497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:41.306064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:41.308690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:41.309243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:41.311237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:41.445189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0351cf3f-c810-4e6a-a89e-fe2dd9ae32b5 req-4e056fa5-6964-48cd-9c12-239527545ab2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa", "tag": "f1ee217d-8ce5-4ef4-aa47-ea76d019efff"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:41.457862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0351cf3f-c810-4e6a-a89e-fe2dd9ae32b5 req-4e056fa5-6964-48cd-9c12-239527545ab2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:41.458578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0351cf3f-c810-4e6a-a89e-fe2dd9ae32b5 req-4e056fa5-6964-48cd-9c12-239527545ab2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:41.459240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0351cf3f-c810-4e6a-a89e-fe2dd9ae32b5 req-4e056fa5-6964-48cd-9c12-239527545ab2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:41.472696 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0351cf3f-c810-4e6a-a89e-fe2dd9ae32b5 req-4e056fa5-6964-48cd-9c12-239527545ab2 service nova] Creating event network-changed:f1ee217d-8ce5-4ef4-aa47-ea76d019efff for instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa on nch1-master-1 Oct 15 21:13:41.481203 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0351cf3f-c810-4e6a-a89e-fe2dd9ae32b5 req-4e056fa5-6964-48cd-9c12-239527545ab2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039625 Oct 15 21:13:41.482367 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1232/4714] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:13:41.867489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:41.876573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3dce1830-c845-4453-9e76-62c670b0d76b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.633862 Oct 15 21:13:41.877472 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3483/4715] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:41 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:41.903107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:41.903845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:41.910254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:41.911003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:41.911623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:41.912494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:41.913379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:41.913974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:42.481340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:42.509575 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-82975384-bc30-4f86-9c22-2323bac1e714 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.610265 Oct 15 21:13:42.511592 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1233/4716] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:41 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:42.535333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:42.536548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:42.543898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:42.544408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:42.544860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:42.545740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:42.546213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:42.546755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:43.083139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:43.097122 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a2d071f-8352-49a7-b15f-2524e2ed0989 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.565644 Oct 15 21:13:43.099859 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3484/4717] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:42 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:43.309963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-570055934", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:43.396012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] validate_networks() for [('c03d22eb-9d7c-48c6-94c0-1e5d2877f4dd', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:13:44.958875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:13:44.959497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:13:44.961141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:13:44.962036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:13:44.963877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:13:44.976832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:44.977563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:44.978366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:44.979223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:44.979800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:44.980480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:45.005142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:13:45.010804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Getting quotas for user 1aadd87b97c247cb95321664ba0c32a9 and project 38b9244bbab846b7baf42985cebc72c2. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:13:45.027359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:13:45.029928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:13:45.081830 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-72162d67-88aa-4571-beec-7df23b3174de tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.778533 Oct 15 21:13:45.083310 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1234/4718] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:13:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 1782 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:13:45.095041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f5d329e-fbab-49ce-8ff0-57d22e112526 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:45.096386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f5d329e-fbab-49ce-8ff0-57d22e112526 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:45.235243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f5d329e-fbab-49ce-8ff0-57d22e112526 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:45.287302 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f5d329e-fbab-49ce-8ff0-57d22e112526 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1214 microversion: 2.1 time: 0.196939 Oct 15 21:13:45.291884 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3485/4719] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:45 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1214 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:46.317019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:46.318433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:46.329269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:46.333869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:46.334554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:46.336170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:46.337112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:46.337852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:46.495362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:46.508183 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-084a8bb6-8d06-44eb-96c7-32b3b3cae8e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1218 microversion: 2.1 time: 0.195228 Oct 15 21:13:46.512233 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1235/4720] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:46 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1218 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:46.716057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:46.716057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:46.726223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:46.726690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:46.727271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:46.728594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:46.735958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:46.736671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:46.782545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:13:46.900871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ce1a165c-dc5d-4de1-8b2f-0bd84a9cda12 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "DELETE /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 204 len: 0 microversion: 2.1 time: 0.191710 Oct 15 21:13:46.902145 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3486/4721] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:13:46 2022] DELETE /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:13:46.918676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:46.919320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:46.928499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:46.929489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:46.929944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:46.931171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:46.931674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:46.932115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:47.212867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:47.221641 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-eae7abcf-75ce-47f7-8cbb-9bbaf5efcbcb tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1506 microversion: 2.1 time: 0.314831 Oct 15 21:13:47.225132 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1236/4722] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:46 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1506 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:47.537974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:47.538789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:47.551569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:47.552180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:47.557153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:47.558432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:47.558928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:47.559386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:47.662934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:47.671553 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c3c0f6b-459d-4315-9901-33de7244d5ef tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1274 microversion: 2.1 time: 0.138603 Oct 15 21:13:47.679131 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3487/4723] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:47 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1274 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:48.247474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:48.249151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:48.259279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:48.262599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:48.265215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:48.266982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:48.269824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:48.272587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:48.557658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:48.564439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:48.566770 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1506 microversion: 2.1 time: 0.323470 Oct 15 21:13:48.570154 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1237/4724] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:48 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1506 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:48.706842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:48.709576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:48.735658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:48.738340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:48.738961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:48.741572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:48.745683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:48.746782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:48.798815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:48.800213 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-390ed3e0-6a00-463b-88b5-ad626e285c33 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001778 Oct 15 21:13:48.808561 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1238/4725] 10.222.0.16 () {56 vars in 886 bytes} [Sat Oct 15 21:13:48 2022] GET /compute/ => generated 377 bytes in 10 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Oct 15 21:13:49.027479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:49.046054 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a6ab62c-b623-4163-9182-c6fcc1064589 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1274 microversion: 2.1 time: 0.352410 Oct 15 21:13:49.061776 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3488/4726] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:48 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1274 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:49.596889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:49.598727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:49.617288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:49.621556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:49.622013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:49.628774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:49.629452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:49.631141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:50.075325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:50.077692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:50.093794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:50.094542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:50.095089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:50.096065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:50.096750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:50.097398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:50.376181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:50.392621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55d1cf23-79ca-4878-b21a-21be08f26ea7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1272 microversion: 2.1 time: 0.322464 Oct 15 21:13:50.398185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3489/4727] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:50 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1272 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:50.428195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:50.437631 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-401d3e28-a978-490e-838a-71aac660845b tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1506 microversion: 2.1 time: 0.846204 Oct 15 21:13:50.439088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1239/4728] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:49 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1506 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:51.428277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:51.430599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:51.441604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:51.442426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:51.443268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:51.444268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:51.444945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:51.445741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:51.480130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:51.480750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:51.504869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:51.505670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:51.506279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:51.516205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:51.518625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:51.519097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:51.688332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:51.692469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:51.697887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddd40aea-8b6d-4244-b5bd-f01150712a46 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1272 microversion: 2.1 time: 0.276652 Oct 15 21:13:51.701879 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fabf73ea-d946-4e8b-9ebc-55077d84a3ff tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 200 len: 1464 microversion: 2.1 time: 0.242531 Oct 15 21:13:51.703457 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1240/4729] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:51 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 1464 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:51.709965 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3490/4730] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:51 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1272 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:52.270100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-addb85ae-544c-41e0-ad20-f488a0b1c624 req-82105abd-4649-4a59-b43b-9535f0dc6483 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76691e91-f327-49c4-88ce-d6932390510f", "name": "network-vif-deleted", "tag": "b51cdbdc-d3ee-4cf7-85b1-07fec0d15ffc"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:52.293957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-addb85ae-544c-41e0-ad20-f488a0b1c624 req-82105abd-4649-4a59-b43b-9535f0dc6483 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.294755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-addb85ae-544c-41e0-ad20-f488a0b1c624 req-82105abd-4649-4a59-b43b-9535f0dc6483 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.296323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-addb85ae-544c-41e0-ad20-f488a0b1c624 req-82105abd-4649-4a59-b43b-9535f0dc6483 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.315541 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-addb85ae-544c-41e0-ad20-f488a0b1c624 req-82105abd-4649-4a59-b43b-9535f0dc6483 service nova] Creating event network-vif-deleted:b51cdbdc-d3ee-4cf7-85b1-07fec0d15ffc for instance 76691e91-f327-49c4-88ce-d6932390510f on nch2-master-1 Oct 15 21:13:52.331972 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-addb85ae-544c-41e0-ad20-f488a0b1c624 req-82105abd-4649-4a59-b43b-9535f0dc6483 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.594642 Oct 15 21:13:52.335905 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1241/4731] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:52.723549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:52.724200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:52.734497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:52.735839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: 76691e91-f327-49c4-88ce-d6932390510f] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:52.740935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.741713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.742341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.743339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.743971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.746746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.750473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.751913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.753243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.754358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.758382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.759297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.782707 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Instance 76691e91-f327-49c4-88ce-d6932390510f could not be found. Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f HTTP/1.1 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 76691e91-f327-49c4-88ce-d6932390510f could not be found. Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '76691e91-f327-49c4-88ce-d6932390510f'} Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 76691e91-f327-49c4-88ce-d6932390510f could not be found. Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 76691e91-f327-49c4-88ce-d6932390510f could not be found. Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:52.785605 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 76691e91-f327-49c4-88ce-d6932390510f could not be found. Oct 15 21:13:52.792321 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:13:52.794213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 404 to user: Instance 76691e91-f327-49c4-88ce-d6932390510f could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:13:52.798284 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1bd733da-b347-4cd3-895b-0967e9263357 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f" status: 404 len: 111 microversion: 2.1 time: 0.070925 Oct 15 21:13:52.801027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:52.804696 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1242/4732] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:52 2022] GET /compute/v2.1/servers/76691e91-f327-49c4-88ce-d6932390510f => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:13:52.814398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:52.817433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:52.825377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.826581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.827936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.829089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:52.830006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:52.830947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:52.887855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:13:53.069785 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5ca06c16-5044-4a4c-80a0-6aefff83a207 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 204 len: 0 microversion: 2.1 time: 0.263166 Oct 15 21:13:53.075805 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1243/4733] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:13:52 2022] DELETE /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:13:53.086586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:53.087305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:53.106207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:53.106632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:53.107127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:53.108442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:53.108913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:53.109381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:53.172714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:53.215525 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83a72c66-4874-43a4-89e9-f4addc25facd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 0.496296 Oct 15 21:13:53.216462 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3491/4734] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:52 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:53.540617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:53.550931 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fe9297a9-afb8-4e03-8048-fa43d1abe9cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.470049 Oct 15 21:13:53.557354 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1244/4735] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:53 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:53.942506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc89d80c-fdc9-4b50-931b-126858a3a6a8 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Calling method '.version_select at 0x7fe74f1f8940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:54.021806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc89d80c-fdc9-4b50-931b-126858a3a6a8 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b/defaults" status: 200 len: 371 microversion: 2.1 time: 0.260422 Oct 15 21:13:54.029773 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3492/4736] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:13:53 2022] GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b/defaults => generated 371 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:54.035751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7955185c-04da-40f6-884b-1133d9858324 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Calling method '.version_select at 0x7fe74ee56550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:54.171830 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7955185c-04da-40f6-884b-1133d9858324 tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b" status: 200 len: 371 microversion: 2.1 time: 0.140179 Oct 15 21:13:54.181208 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1245/4737] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:13:54 2022] GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b => generated 371 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:54.188309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f4b481d-cf67-49a9-93cb-16f46b8814cc tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Calling method '.version_select at 0x7fe74e836e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:54.243346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:54.244933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:54.255809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:54.256312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:54.256754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:54.260790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:54.261577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:54.263608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:54.328941 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f4b481d-cf67-49a9-93cb-16f46b8814cc tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b/defaults" status: 200 len: 371 microversion: 2.1 time: 0.145979 Oct 15 21:13:54.329626 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3493/4738] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:13:54 2022] GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b/defaults => generated 371 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:54.345115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-555c7f7f-5933-42fa-a577-0def5d8116df tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Calling method '.version_select at 0x7fe74e0f7940>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:54.610197 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-555c7f7f-5933-42fa-a577-0def5d8116df tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b" status: 200 len: 371 microversion: 2.1 time: 0.269922 Oct 15 21:13:54.612142 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3494/4739] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:13:54 2022] GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b => generated 371 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:13:54.618847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:54.619750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:54.628112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:54.629105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:54.629725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:54.631204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:54.632771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:54.633446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:54.991119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:55.011876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:55.024411 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f46e139-8792-403a-bc0a-6cff5672ecc9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 0.784968 Oct 15 21:13:55.026092 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1246/4740] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:54 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:55.033713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-29954608-e8c6-436f-8a4b-03eb96e690be req-920a88b3-a282-419f-8b95-7cada6f5ef1f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6b62f776-94e1-4e10-9095-ed834dbbb229", "tag": "b1d14b84-ac66-4849-a397-f7e2801d4ebb"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:55.054076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-29954608-e8c6-436f-8a4b-03eb96e690be req-920a88b3-a282-419f-8b95-7cada6f5ef1f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:55.054741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-29954608-e8c6-436f-8a4b-03eb96e690be req-920a88b3-a282-419f-8b95-7cada6f5ef1f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:55.054872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-29954608-e8c6-436f-8a4b-03eb96e690be req-920a88b3-a282-419f-8b95-7cada6f5ef1f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:55.068867 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-29954608-e8c6-436f-8a4b-03eb96e690be req-920a88b3-a282-419f-8b95-7cada6f5ef1f service nova] Creating event network-changed:b1d14b84-ac66-4849-a397-f7e2801d4ebb for instance 6b62f776-94e1-4e10-9095-ed834dbbb229 on nch2-master-1 Oct 15 21:13:55.078513 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-29954608-e8c6-436f-8a4b-03eb96e690be req-920a88b3-a282-419f-8b95-7cada6f5ef1f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050472 Oct 15 21:13:55.080528 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1247/4741] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:55.098097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-281960fe-b027-491f-a49c-2fe02769e92e tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] Calling method '.version_select at 0x7fe74eef91f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:55.236090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:55.255654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14048e13-a0ee-4f51-a6c7-41ae3c1009cc tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.640799 Oct 15 21:13:55.256812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3495/4742] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:54 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:55.650828 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-281960fe-b027-491f-a49c-2fe02769e92e tempest-QuotasTestJSON-496796251 tempest-QuotasTestJSON-496796251-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b?user_id=76fe12490e6847a795b9871e47182d08" status: 200 len: 371 microversion: 2.1 time: 0.561345 Oct 15 21:13:55.651640 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1248/4743] 10.222.0.16 () {58 vars in 1323 bytes} [Sat Oct 15 21:13:55 2022] GET /compute/v2.1/os-quota-sets/4b7eb1d81f374478b3db39266029033b?user_id=76fe12490e6847a795b9871e47182d08 => generated 371 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:56.051811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:56.053515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:56.070487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:56.071323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:56.072030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:56.073245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:56.074064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:56.074720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:56.281187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:56.281767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:56.291302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:56.291814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:56.292265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:56.292995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:56.293495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:56.293932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:57.048801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:57.056656 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9eaa94fc-2fff-4c17-8e6a-54a4dccf9716 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 1.012519 Oct 15 21:13:57.069129 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3496/4744] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:56 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:57.077614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:57.089268 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-111e2d7e-6cec-4dc9-a7fa-bf513dfa6efe tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1518 microversion: 2.1 time: 0.813170 Oct 15 21:13:57.089859 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1249/4745] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:56 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1518 bytes in 815 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:58.070521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f4be97cc-c15d-4d32-85d4-68f04cc57dc8 req-7591748c-c8bb-46e9-b720-0b29ade8f5e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b4ab84bd-5610-44e0-86cf-2877eac80927", "name": "network-vif-deleted", "tag": "bdfe27f9-115c-4ed4-88ed-0bf0adaabfac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:13:58.077066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=70,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:58.086132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:58.086748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:58.102745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:13:58.106926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:58.109141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4be97cc-c15d-4d32-85d4-68f04cc57dc8 req-7591748c-c8bb-46e9-b720-0b29ade8f5e5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:58.109861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:58.110462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:58.111425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:58.112347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4be97cc-c15d-4d32-85d4-68f04cc57dc8 req-7591748c-c8bb-46e9-b720-0b29ade8f5e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:58.112982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:58.113628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:58.117262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f4be97cc-c15d-4d32-85d4-68f04cc57dc8 req-7591748c-c8bb-46e9-b720-0b29ade8f5e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:58.163797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f4be97cc-c15d-4d32-85d4-68f04cc57dc8 req-7591748c-c8bb-46e9-b720-0b29ade8f5e5 service nova] Creating event network-vif-deleted:bdfe27f9-115c-4ed4-88ed-0bf0adaabfac for instance b4ab84bd-5610-44e0-86cf-2877eac80927 on nch1-master-1 Oct 15 21:13:58.176208 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f4be97cc-c15d-4d32-85d4-68f04cc57dc8 req-7591748c-c8bb-46e9-b720-0b29ade8f5e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109292 Oct 15 21:13:58.180592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3497/4746] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:13:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:13:58.211201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:58.211934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:58.233382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:58.234886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:58.235344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:58.236079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:58.236970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:58.237451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:58.825012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c48346e56c47481cab7f248b67369a27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2b1936403344a4994393f2b098327a0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:58.838283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8bf85805-ea50-48cd-aeeb-5d3353049627 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 200 len: 1304 microversion: 2.1 time: 0.654022 Oct 15 21:13:58.865352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3498/4747] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:58 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 1304 bytes in 684 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:13:59.380530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:13:59.390022 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f92eb17-bec4-407b-b970-faa1a854ae3a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 1.308011 Oct 15 21:13:59.392836 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1250/4748] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:58 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 1311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:13:59.885795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:13:59.889941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] [instance: b4ab84bd-5610-44e0-86cf-2877eac80927] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:13:59.908702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:59.909819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:59.910499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:59.911895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:13:59.913575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:13:59.914623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:13:59.938504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] HTTP exception thrown: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 could not be found. Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Request method failure captured: Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 HTTP/1.1 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyF0ISydZ_lplBCYAc3tgpZ_9_r_jPz6nAPFd0EKVOCWHNBnqmdSshvF93gqBGBx9iQVEK6Rc8ViWT1npDSvRcPgPEt50M2tp2bI1fp48XvKWWj5V7JXyV9rr-EaklR0n7KLuq3E26GJPtyITHl7XvHXi3dPf_h_SkUtqch2pXPQV3k Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2b1936403344a4994393f2b098327a0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2b1936403344a4994393f2b098327a0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2b1936403344a4994393f2b098327a0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e2b1936403344a4994393f2b098327a0 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-734140759 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c48346e56c47481cab7f248b67369a27 Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerRescueNegativeTestJSON-734140759-project Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 could not be found. Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b4ab84bd-5610-44e0-86cf-2877eac80927'} Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 could not be found. Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 could not be found. Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:59.942352 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 could not be found. Oct 15 21:13:59.947840 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:13:59.951995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] Returning 404 to user: Instance b4ab84bd-5610-44e0-86cf-2877eac80927 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:13:59.953332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-224f43ac-8ac6-4eae-97f8-3aee9d662ec5 tempest-ServerRescueNegativeTestJSON-734140759 tempest-ServerRescueNegativeTestJSON-734140759-project] 10.222.0.16 "GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927" status: 404 len: 111 microversion: 2.1 time: 0.073772 Oct 15 21:13:59.960788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3499/4749] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:13:59 2022] GET /compute/v2.1/servers/b4ab84bd-5610-44e0-86cf-2877eac80927 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:14:00.416714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:00.421868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:00.449117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:00.449646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:00.450092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:00.461830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:00.462343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:00.462792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:01.188464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:01.222723 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-431a1960-6c2d-49b2-b96e-e0eb05827142 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 0.811312 Oct 15 21:14:01.240461 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1251/4750] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:00 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:02.165277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-eca336b6-fcaa-4cb1-a48c-0ffafd4fd41a req-fa893903-1dc9-4bd4-89dd-3610e54b0780 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6b62f776-94e1-4e10-9095-ed834dbbb229", "name": "network-vif-plugged", "status": "completed", "tag": "b1d14b84-ac66-4849-a397-f7e2801d4ebb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:02.207238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eca336b6-fcaa-4cb1-a48c-0ffafd4fd41a req-fa893903-1dc9-4bd4-89dd-3610e54b0780 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:02.207411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eca336b6-fcaa-4cb1-a48c-0ffafd4fd41a req-fa893903-1dc9-4bd4-89dd-3610e54b0780 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:02.208084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eca336b6-fcaa-4cb1-a48c-0ffafd4fd41a req-fa893903-1dc9-4bd4-89dd-3610e54b0780 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:02.225627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-eca336b6-fcaa-4cb1-a48c-0ffafd4fd41a req-fa893903-1dc9-4bd4-89dd-3610e54b0780 service nova] Creating event network-vif-plugged:b1d14b84-ac66-4849-a397-f7e2801d4ebb for instance 6b62f776-94e1-4e10-9095-ed834dbbb229 on nch2-master-1 Oct 15 21:14:02.240408 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-eca336b6-fcaa-4cb1-a48c-0ffafd4fd41a req-fa893903-1dc9-4bd4-89dd-3610e54b0780 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.341515 Oct 15 21:14:02.241525 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3500/4751] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:14:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:14:02.272291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:02.273671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:02.301310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:02.302447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:02.307325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:02.308106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:02.308591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:02.313965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:03.419778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:03.431567 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ee6de7bb-aa63-42c0-a1f8-8f5c89bb70c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 1.173831 Oct 15 21:14:03.433056 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1252/4752] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:02 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 1176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:04.459218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:04.460261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:04.471094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:04.472133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:04.473318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:04.474340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:04.474915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:04.475449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:04.603201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:05.582801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:05.598881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-db8c462e-eb61-4bf6-b2ce-77d27f9ad945 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1314 microversion: 2.1 time: 1.146184 Oct 15 21:14:05.599736 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3501/4753] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:04 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1314 bytes in 1148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:06.635836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:06.637758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:06.649848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:06.651965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:06.652723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:06.655902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:06.657030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:06.658738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:06.659571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:08.328932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:08.346655 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-562abb11-e7a6-40e3-beba-8b9f599edb45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1501 microversion: 2.1 time: 1.712860 Oct 15 21:14:08.347333 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1253/4754] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:06 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1501 bytes in 1720 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:08.372877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:08.374532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:08.386769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:08.391866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:08.392575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:08.397976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:08.399232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:08.399896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:08.621304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae83d09d-f583-4478-9261-a32268bc71a6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229/action" status: 202 len: 0 microversion: 2.1 time: 0.248633 Oct 15 21:14:08.624005 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3502/4755] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:14:08 2022] POST /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229/action => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:14:08.631519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:08.632120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:08.652069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:08.652574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:08.653280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:08.658335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:08.659685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:08.667021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:09.573223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:09.586631 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8251dd2d-f3e5-4bfe-854a-83f483ae7327 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1506 microversion: 2.1 time: 0.958820 Oct 15 21:14:09.593125 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1254/4756] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:08 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1506 bytes in 963 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:10.616628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:10.624011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:10.658401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:10.658895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:10.679553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:10.682399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:10.683083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:10.683706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:12.032022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:12.040536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5a8660d-1ab0-4b6c-b403-fe1988d4c905 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1486 microversion: 2.1 time: 1.428166 Oct 15 21:14:12.053722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3503/4757] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:10 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1486 bytes in 1442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:12.081601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:12.082159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:12.091303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:12.091846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:12.092297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:12.093120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:12.093591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:12.094033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:12.129557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lazy-loading 'flavor' on Instance uuid 6b62f776-94e1-4e10-9095-ed834dbbb229 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:14:12.257325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:12.447686 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8158b38b-68d7-4f84-ba4d-4ffbfbb9c259 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229/action" status: 202 len: 0 microversion: 2.1 time: 0.369995 Oct 15 21:14:12.448869 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1255/4758] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:14:12 2022] POST /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229/action => generated 0 bytes in 372 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:14:12.457925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:12.458854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:12.475095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:12.475591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:12.476297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:12.477311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:12.478046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:12.478708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:13.084403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:13.098693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45fbd71e-2829-4443-b8ff-c3ac233ed28d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1492 microversion: 2.1 time: 0.644595 Oct 15 21:14:13.100168 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3504/4759] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:12 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1492 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:14.129608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:14.133300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:14.143294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:14.143830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:14.144257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:14.145216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:14.145843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:14.146294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:15.233171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:15.246279 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b092e4d9-2a13-4900-9bbc-9058b2a5df92 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1513 microversion: 2.1 time: 1.126198 Oct 15 21:14:15.247254 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1256/4760] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:14 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1513 bytes in 1128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:14:16.272311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:16.272859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:16.285985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:16.286497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:16.287868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:16.287868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:16.288115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:16.288719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:17.050254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:17.058766 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45549de8-c054-43e8-9765-a4636e0b55c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1513 microversion: 2.1 time: 0.790731 Oct 15 21:14:17.064610 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3505/4761] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:16 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1513 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:14:18.068162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-f1c0edd3-63fe-43d7-afc7-505222123e6c req-df19a9d1-6875-48c9-bdae-e8c7a8de4952 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6b62f776-94e1-4e10-9095-ed834dbbb229", "name": "network-vif-plugged", "status": "completed", "tag": "b1d14b84-ac66-4849-a397-f7e2801d4ebb"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:18.084436 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f1c0edd3-63fe-43d7-afc7-505222123e6c req-df19a9d1-6875-48c9-bdae-e8c7a8de4952 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:18.084927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f1c0edd3-63fe-43d7-afc7-505222123e6c req-df19a9d1-6875-48c9-bdae-e8c7a8de4952 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:18.085384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f1c0edd3-63fe-43d7-afc7-505222123e6c req-df19a9d1-6875-48c9-bdae-e8c7a8de4952 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:18.094290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:18.095129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:18.109974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:18.110337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:18.112493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:18.113424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:18.114731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:18.116684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:18.117903 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-f1c0edd3-63fe-43d7-afc7-505222123e6c req-df19a9d1-6875-48c9-bdae-e8c7a8de4952 service nova] Creating event network-vif-plugged:b1d14b84-ac66-4849-a397-f7e2801d4ebb for instance 6b62f776-94e1-4e10-9095-ed834dbbb229 on nch2-master-1 Oct 15 21:14:18.126805 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-f1c0edd3-63fe-43d7-afc7-505222123e6c req-df19a9d1-6875-48c9-bdae-e8c7a8de4952 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062391 Oct 15 21:14:18.135757 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1257/4762] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:14:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:14:18.462495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:18.475858 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0384aee1-8a21-44bf-a1c8-43da59f00f41 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.387679 Oct 15 21:14:18.478995 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3506/4763] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:18 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:19.518357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:19.520106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:19.551249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:19.551742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:19.552178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:19.552889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:19.553428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:19.553790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:19.971005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:19.989747 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e24a8422-18a4-4d3b-946e-97403a51db1c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.469036 Oct 15 21:14:19.993310 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1258/4764] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:19 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:21.015605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:21.016408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:21.034529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:21.037668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:21.038202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:21.040407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:21.047927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:21.050085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:21.788249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:21.817155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0a99e45-1556-405b-afc1-deead1267275 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.803199 Oct 15 21:14:21.821419 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3507/4765] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:21 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:22.845737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:22.849054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:22.862179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:22.864534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:22.868343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:22.869432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:22.876592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:22.877357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:23.497757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:23.513710 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa235b3f-b577-4d53-aa8a-e5a629ab0eb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.667558 Oct 15 21:14:23.514998 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1259/4766] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:22 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:24.554197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:24.554818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:24.564314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:24.571111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:24.573700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:24.575042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:24.575518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:24.582413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:24.595107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:25.511738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:25.512783 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71e840b2-6fa4-499d-bfe4-ce135dae2cc4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.962045 Oct 15 21:14:25.513593 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3508/4767] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:24 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 964 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:26.552220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:26.553653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:26.568793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:26.569697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:26.570797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:26.573108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:26.573933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:26.576810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:27.430599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:27.455476 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-795cdd24-ce0b-4a25-a1ef-936d5e1b913d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.905382 Oct 15 21:14:27.456143 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1260/4768] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:26 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 910 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:28.479192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:28.479819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:28.504015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=24 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:28.505541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:28.511241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:28.515861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:28.516883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:28.518454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:28.540803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:30.400173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:30.411317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fdeb9c34-155e-491f-a448-fb608306ea3b tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 1.935920 Oct 15 21:14:30.422978 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3509/4769] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:28 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 1948 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:31.437810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:31.438444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:31.462358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=22 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:31.463928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:31.474115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:31.477761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:31.478591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:31.479311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:31.479516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:32.198288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:32.207367 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7e95d194-02b5-437b-b732-a5925d26217f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.775265 Oct 15 21:14:32.208024 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1261/4770] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:31 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:33.234717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:33.235767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:33.250526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:33.251781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:33.253679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:33.256501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:33.257066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:33.257603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:33.748151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:33.761156 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1bd9b2aa-97af-4bc2-ac2d-896292293f4c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.531159 Oct 15 21:14:33.766239 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3510/4771] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:33 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:34.790529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:34.791387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:34.801902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:34.802314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:34.802757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:34.826279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:34.826907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:34.827367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:35.510130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:35.522585 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1977aa07-283b-4597-a80e-4d632bb310b1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.735750 Oct 15 21:14:35.529612 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1262/4772] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:34 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:36.554680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:36.556116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:36.570137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:36.571995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:36.573844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:36.576780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:36.577739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:36.579482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:37.019200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:37.031836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-085c8221-cba5-4a4f-917d-147202368801 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.481348 Oct 15 21:14:37.036593 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3511/4773] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:36 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:38.061011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:38.063024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:38.084079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:38.093911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:38.096009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:38.100430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:38.103713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:38.105201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:39.566289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:39.582552 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1efd51c2-d9ec-4bb7-b9be-68f0e2f7333a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 1.525879 Oct 15 21:14:39.585335 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1263/4774] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:38 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 1530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:40.612025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:40.614276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:40.631489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:40.631902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:40.632365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:40.640425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:40.641130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:40.641610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:41.586347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:41.606179 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-849d702c-20cf-47fc-bc66-3a9e3a115570 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.995147 Oct 15 21:14:41.608506 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3512/4775] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:40 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 1000 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:42.629532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:42.630158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:42.654270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:42.660974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:42.663075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:42.672362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:42.680708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:42.688501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:42.704577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=46 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:43.671940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:43.688159 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6d702c18-a073-4db5-a7c6-c00a1df5f7dc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 1.056750 Oct 15 21:14:43.691800 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1264/4776] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:42 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 1067 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:44.516048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c66e96af-58ee-41d7-810e-a24de95373b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: .version_select at 0x7fe74fa9f940>, body: {"keypair": {"name": "tempest-keypair-689819752", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLzdCKpbDkPDy76gOLf2QEkPvW+5KjPUeAZM6qlgtHA1Qw9L0uqSOIPZROy64GKoxXr75Uw+Ps2PVzsyI0HiZWIO52XdXSaGuo6GiN8kb/ME9V1htK+FJCevcG8ud6B+cQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:44.577770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c66e96af-58ee-41d7-810e-a24de95373b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:14:44.621803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c66e96af-58ee-41d7-810e-a24de95373b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:14:44.708978 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c66e96af-58ee-41d7-810e-a24de95373b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.462936 Oct 15 21:14:44.715753 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3513/4777] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:14:44 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:14:44.718921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:44.719548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:44.737578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:44.738101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:44.738553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:44.739850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:44.740324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:44.740757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:45.762519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:45.784599 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b461c513-a89d-48e1-97c5-99ff3dda7cb6 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 1.070308 Oct 15 21:14:45.785919 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1265/4778] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:44 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 1073 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:14:46.815822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:46.818780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:46.827074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:46.829301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:46.834352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:46.835491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:46.836186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:46.836855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:47.661484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:47.690066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-88d83cbd-ce57-48ed-9b1a-f78e2bf7da34 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 0.883040 Oct 15 21:14:47.697252 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3514/4779] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:46 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:48.720674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:48.721553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:48.749852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:48.750443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:48.750999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:48.752892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:48.755411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:48.755918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:50.044597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:50.065096 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7684f058-e1ca-4db2-9184-78e62ae81727 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1508 microversion: 2.1 time: 1.347768 Oct 15 21:14:50.076322 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1266/4780] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:48 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1508 bytes in 1360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:50.239921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1087921870", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d51b163a-0bf6-4a5b-be5f-78f88001c0b8"}], "security_groups": [{"name": "tempest-securitygroup--2118654908"}], "key_name": "tempest-keypair-689819752", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:50.643721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] validate_networks() for [('d51b163a-0bf6-4a5b-be5f-78f88001c0b8', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:14:51.099676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:51.100451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:51.115107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:51.115627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:51.116078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:51.116815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:51.117317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:51.117753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:52.531093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:52.539942 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a295be74-0142-4192-b898-3dc841012102 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1505 microversion: 2.1 time: 1.443981 Oct 15 21:14:52.544457 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1267/4781] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:51 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1505 bytes in 1449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:53.410600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:14:53.411711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:14:53.412218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:14:53.413110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:14:53.413677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:14:53.429343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:53.429844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:53.430290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:53.431039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:53.431496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:53.436171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:53.500897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:14:53.517398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:14:53.555106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:14:53.559340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:14:53.572523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:53.573219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:53.583227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:53.583732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:53.584195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:53.584898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:53.602437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:53.602951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:53.689379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5769bb3d-dcd8-4f3f-8623-443c03e7a93f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.453423 Oct 15 21:14:53.692938 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3515/4782] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:14:50 2022] POST /compute/v2.1/servers => generated 402 bytes in 3458 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:14:53.702165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0016236c-84a7-4bdf-a578-b34f25ba202b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:53.706585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0016236c-84a7-4bdf-a578-b34f25ba202b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:53.927409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0016236c-84a7-4bdf-a578-b34f25ba202b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:53.937118 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0016236c-84a7-4bdf-a578-b34f25ba202b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1241 microversion: 2.1 time: 0.237883 Oct 15 21:14:53.941165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3516/4783] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:53 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1241 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:54.270370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:54.293608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0f9df6d4-2d8d-428e-a085-e34c6e05cd78 req-5cbde025-ca37-44d2-a8f4-91cfc5127dca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6b62f776-94e1-4e10-9095-ed834dbbb229", "name": "network-vif-unplugged", "status": "completed", "tag": "b1d14b84-ac66-4849-a397-f7e2801d4ebb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:54.301598 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3efd5835-54a4-44d7-b4a5-25658c1fd26f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1505 microversion: 2.1 time: 0.733689 Oct 15 21:14:54.304528 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1268/4784] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:53 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1505 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:14:54.342961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f9df6d4-2d8d-428e-a085-e34c6e05cd78 req-5cbde025-ca37-44d2-a8f4-91cfc5127dca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:54.344590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f9df6d4-2d8d-428e-a085-e34c6e05cd78 req-5cbde025-ca37-44d2-a8f4-91cfc5127dca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:54.346851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f9df6d4-2d8d-428e-a085-e34c6e05cd78 req-5cbde025-ca37-44d2-a8f4-91cfc5127dca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:54.406206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0f9df6d4-2d8d-428e-a085-e34c6e05cd78 req-5cbde025-ca37-44d2-a8f4-91cfc5127dca service nova] Creating event network-vif-unplugged:b1d14b84-ac66-4849-a397-f7e2801d4ebb for instance 6b62f776-94e1-4e10-9095-ed834dbbb229 on nch2-master-1 Oct 15 21:14:54.435368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0f9df6d4-2d8d-428e-a085-e34c6e05cd78 req-5cbde025-ca37-44d2-a8f4-91cfc5127dca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.147225 Oct 15 21:14:54.441883 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3517/4785] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:14:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:14:54.966020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:54.966652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:54.976738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:54.977061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:54.989509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:54.995977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:54.996256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:54.996717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:55.328212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:55.329353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:55.344985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:55.351049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:55.351805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:55.353251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:55.373331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:55.374129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:55.391511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=40 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:55.840044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:55.899208 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d77b1247-ba00-481a-a9e0-43bc37199af0 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1245 microversion: 2.1 time: 0.931277 Oct 15 21:14:55.922385 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1269/4786] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:54 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1245 bytes in 962 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:56.233821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:56.248331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7b7b13a-d584-4811-ba30-e27effe575c1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 200 len: 1448 microversion: 2.1 time: 0.924300 Oct 15 21:14:56.250291 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3518/4787] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:55 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 1448 bytes in 928 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:14:56.274988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:56.277564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:56.288786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:56.290983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:56.291878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:56.293137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:56.293813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:56.294463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:56.946688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:56.947343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:56.969180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:56.970112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:56.970700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:56.971904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:56.972738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:56.973663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:56.998112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:57.020502 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-75471f8c-30e7-4605-b58d-d00ea75178f4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.747644 Oct 15 21:14:57.047116 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1270/4788] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:56 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:57.065184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:57.065911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:57.076015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:57.078387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:57.079050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:57.079814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:57.080285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:57.080717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:57.151596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:57.170782 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1209dcd-cae0-4efe-a4ba-3e54f0b18d0a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1299 microversion: 2.1 time: 0.229514 Oct 15 21:14:57.174576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3519/4789] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:56 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1299 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:57.621464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:57.630793 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fafa4378-1481-46da-a684-33138b143634 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.576655 Oct 15 21:14:57.636456 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1271/4790] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:57 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:57.662697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:57.663326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:57.673418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:57.675789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:57.676397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:57.677308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:57.680164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:57.680787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:58.199266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:58.200129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:58.210663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:58.212852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:58.217115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:58.217115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:58.217115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:58.217115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:58.437016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:58.448401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c1c7c6af-35b2-47ab-842e-3ff100080c42 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.789867 Oct 15 21:14:58.451292 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3520/4791] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:57 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:58.483433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:58.484035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:58.508130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:58.518139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:58.518770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:58.520125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:58.520912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:58.522808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:58.548710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:58.564668 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b74e7df0-0170-4b01-aee9-78a35a25cc47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1299 microversion: 2.1 time: 0.371608 Oct 15 21:14:58.567396 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1272/4792] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:58 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1299 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:14:59.257648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:14:59.269648 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0e5c753-a498-462b-996e-a680bb9ac8ab tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.796452 Oct 15 21:14:59.273889 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3521/4793] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:58 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:14:59.295684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3b98f5c1-23b1-4b84-8536-37e630a5c3fd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:14:59.341524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b98f5c1-23b1-4b84-8536-37e630a5c3fd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:59.342077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b98f5c1-23b1-4b84-8536-37e630a5c3fd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:59.342524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b98f5c1-23b1-4b84-8536-37e630a5c3fd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:59.414496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3b98f5c1-23b1-4b84-8536-37e630a5c3fd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Going to try to stop instance {{(pid=138689) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:14:59.526719 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3b98f5c1-23b1-4b84-8536-37e630a5c3fd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.233686 Oct 15 21:14:59.528337 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1273/4794] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:14:59 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 239 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:14:59.537377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:59.538554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:59.542776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=50,nova_api:INSERT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:14:59.568677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:59.569774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:59.570991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:59.574479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:59.577217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:59.578070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:59.593447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:14:59.594516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:14:59.632890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:59.645996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:59.647825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:14:59.652254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:14:59.661250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:14:59.665160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:00.468115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:00.477592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f619f66-8f8e-4b82-bd6e-e923011939f8 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.944714 Oct 15 21:15:00.486789 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3522/4795] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:59 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 955 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:00.589608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:00.602523 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a32cf56d-5e7f-49f2-847f-483b9cb217bf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 1.013818 Oct 15 21:15:00.605862 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1274/4796] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:14:59 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:01.513180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:01.513180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:01.518605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-2033465649", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:01.527338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:01.535957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:01.536462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:01.536917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:01.537663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:01.538164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:01.538780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:01.634038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] validate_networks() for None {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:15:02.808385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:02.829394 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-281688c4-8714-47d8-8d73-c1308235ed91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 1.323207 Oct 15 21:15:02.830176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3523/4797] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:01 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 1325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:02.835114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:02.844823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:02.855626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:02.871518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:02.872194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:02.873356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:02.874013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:02.874647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:03.150571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:15:03.151744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:15:03.152482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:15:03.153138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:15:03.153814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:15:03.170580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:03.172594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:03.173314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:03.177494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:03.179592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:03.180269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:03.232440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:03.249214 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0bd37a82-296f-4120-8c51-acab8f41d951 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 0.415942 Oct 15 21:15:03.250171 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3524/4798] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:02 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:03.251084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Getting quotas for project b0007ee63f7340f68f40f6caf5054654. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:15:03.290062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Getting quotas for user 1d86728459fe4fefabd2555de8608ddc and project b0007ee63f7340f68f40f6caf5054654. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:15:03.334295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:15:03.353672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:15:03.462946 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5b71ab19-dcb1-4c60-9ec8-e40f497ffc41 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.948241 Oct 15 21:15:03.464947 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1275/4799] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:15:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 1951 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:15:03.476531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0302c23-93e1-4b18-a9bb-56ae78c24e66 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:03.477279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0302c23-93e1-4b18-a9bb-56ae78c24e66 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:03.491797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:03.632130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0302c23-93e1-4b18-a9bb-56ae78c24e66 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:03.641704 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0302c23-93e1-4b18-a9bb-56ae78c24e66 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1215 microversion: 2.1 time: 0.169473 Oct 15 21:15:03.647132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3525/4800] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:03 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1215 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:03.854955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:03.855926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:03.873774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:03.878987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:03.879606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:03.884373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:03.888033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:03.891214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:04.278429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:04.279777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:04.291989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:04.297503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:04.297980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:04.298721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:04.299197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:04.299632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:04.386193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:04.407869 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5bf45d81-b128-4abc-88e2-dba961a6c3dd tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.558435 Oct 15 21:15:04.408669 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1276/4801] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:03 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:04.562513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-f445fed4-9839-49b9-804c-adcb05c0fa31 req-9855aa28-e2e7-42eb-9c63-27af8148f6c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c0812e6-ee5e-470c-a73a-5eaaca23354c", "tag": "9b5226b2-acbc-4977-8451-6c4b8fb6bf30"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:04.592976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f445fed4-9839-49b9-804c-adcb05c0fa31 req-9855aa28-e2e7-42eb-9c63-27af8148f6c0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:04.593488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f445fed4-9839-49b9-804c-adcb05c0fa31 req-9855aa28-e2e7-42eb-9c63-27af8148f6c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:04.597310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f445fed4-9839-49b9-804c-adcb05c0fa31 req-9855aa28-e2e7-42eb-9c63-27af8148f6c0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:04.621677 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-f445fed4-9839-49b9-804c-adcb05c0fa31 req-9855aa28-e2e7-42eb-9c63-27af8148f6c0 service nova] Creating event network-changed:9b5226b2-acbc-4977-8451-6c4b8fb6bf30 for instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c on nch1-master-1 Oct 15 21:15:04.638206 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-f445fed4-9839-49b9-804c-adcb05c0fa31 req-9855aa28-e2e7-42eb-9c63-27af8148f6c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.081817 Oct 15 21:15:04.639176 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1277/4802] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:15:04.676579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:04.677924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:04.702504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:04.702913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:04.708225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:04.710654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:04.713770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:04.714472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:04.962075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:04.987548 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f29fddb4-4033-4fb6-b59f-e8761eeea40f tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1211 microversion: 2.1 time: 0.314595 Oct 15 21:15:04.993216 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1278/4803] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:04 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1211 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:05.158727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:05.169524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df640a06-3752-47d9-b499-2e49dc12cb40 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 0.895839 Oct 15 21:15:05.170798 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3526/4804] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:04 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:05.436479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:05.437191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:05.446725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:05.452628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:05.453324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:05.454609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:05.455392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:05.455938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:06.020313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:06.021628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:06.032490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:06.035630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:06.039621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:06.042266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:06.043345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:06.043886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:06.293475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:06.307907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fe28a60-f3d1-45ee-b06a-a3fa51526213 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1273 microversion: 2.1 time: 0.291914 Oct 15 21:15:06.313862 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3527/4805] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:06 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1273 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:06.321680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:06.322313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:06.328244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:06.336754 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f0d83296-c3a1-43e8-b124-0fe52645193a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.905145 Oct 15 21:15:06.339316 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1279/4806] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:05 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 909 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:06.351224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:06.351730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:06.352183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:06.352888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:06.357424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:06.359070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:07.334780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:07.335402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:07.346253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:07.346747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:07.347657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:07.350263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:07.350843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:07.351991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:07.489749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:07.501240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf61684d-43d8-4e77-abee-12a2b6e440b4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 1.186163 Oct 15 21:15:07.514896 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3528/4807] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:06 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 1200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:07.525372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:07.526944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:07.534077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:07.535270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:07.535918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:07.537280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:07.538203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:07.538857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:07.541478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:07.549497 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fef656e7-4b40-49ec-b113-7c7b088614fb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1273 microversion: 2.1 time: 0.218613 Oct 15 21:15:07.561314 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1280/4808] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:07 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1273 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:07.990241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:08.002417 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2cb7a371-d6cc-4008-ad2e-c677f0a2b355 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.485947 Oct 15 21:15:08.003026 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3529/4809] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:07 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:08.530721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:08.531837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:08.546967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:08.550682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:08.551394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:08.552904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:08.556122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:08.559809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:08.578083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:08.578700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:08.583965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:08.592509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:08.593015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:08.593488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:08.594210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:08.594672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:08.601936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:08.859625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:08.870825 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b45081e6-9b2a-45d3-8799-8b3c5b66b332 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1273 microversion: 2.1 time: 0.298708 Oct 15 21:15:08.890667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3530/4810] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:08 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1273 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:09.040566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:09.044110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:09.064773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:09.066615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:09.076413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:09.082303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:09.084684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:09.085436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:09.156892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:09.170428 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-33a94251-ca4a-429a-b4a3-44bf56e8c5c4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 0.643988 Oct 15 21:15:09.172665 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1281/4811] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:08 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:09.898269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:09.898902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:09.900404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:09.912867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:09.914270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:09.914766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:09.918135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:09.918639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:09.919087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:09.976331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f41e7ff-330a-4faf-a13b-66822d3b9790 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1681 microversion: 2.1 time: 0.949678 Oct 15 21:15:09.991213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3531/4812] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:09 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1681 bytes in 965 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:10.145907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:10.159272 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a38064ab-a12c-4a2e-9bf0-72b1f6010532 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1273 microversion: 2.1 time: 0.264875 Oct 15 21:15:10.163113 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1282/4813] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:09 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1273 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:10.201245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:10.202292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:10.217299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:10.219578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:10.220221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:10.226733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:10.227302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:10.227758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:11.007689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:11.008441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:11.025422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:11.026128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:11.026813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:11.027742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:11.029303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:11.031554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:11.609305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:11.622910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-758ee97c-643d-4bdc-91c9-aab27a69e5ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 1.426105 Oct 15 21:15:11.625185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3532/4814] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:10 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 1429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:11.631172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:11.632470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:11.642229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:11.643131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:11.643964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:11.644920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:11.645565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:11.646932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:12.251774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:12.262774 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8984b7f0-2509-4e09-bdaf-2dec845053d4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1658 microversion: 2.1 time: 1.259236 Oct 15 21:15:12.263349 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1283/4815] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:11 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1658 bytes in 1261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:12.288389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-868714d0-4f1d-4a18-8524-e7e8d7754f4e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:12.297974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-868714d0-4f1d-4a18-8524-e7e8d7754f4e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:12.298806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-868714d0-4f1d-4a18-8524-e7e8d7754f4e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:12.302016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-868714d0-4f1d-4a18-8524-e7e8d7754f4e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:12.335768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-868714d0-4f1d-4a18-8524-e7e8d7754f4e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Going to try to start instance {{(pid=138689) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 21:15:12.419343 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-868714d0-4f1d-4a18-8524-e7e8d7754f4e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.135059 Oct 15 21:15:12.419974 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1284/4816] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:15:12 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 137 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:15:12.430354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:12.431962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:12.444768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:12.445658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:12.446667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:12.448020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:12.448813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:12.449002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:12.449757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:12.457626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8e142a15-1c9d-4db9-895b-0956c3bba707 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 0.831264 Oct 15 21:15:12.464687 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3533/4817] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:11 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:12.646756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:12.647408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:12.667706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:12.668384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:12.668970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:12.670004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:12.670473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:12.670933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:13.163061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:13.177521 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-59027f9e-7648-45b1-8729-9bb3bd166b5e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1667 microversion: 2.1 time: 0.752185 Oct 15 21:15:13.182117 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1285/4818] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:12 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1667 bytes in 758 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:13.366929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:13.380584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9fbca50-4073-4e43-805c-d9c9f27a10dd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 0.737208 Oct 15 21:15:13.381630 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3534/4819] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:12 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:13.397771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e973afc2-b5b0-4d7c-94cb-85d2c49c1e6c req-d58299fe-88a2-41ad-811c-6847788d4703 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c0812e6-ee5e-470c-a73a-5eaaca23354c", "name": "network-vif-plugged", "status": "completed", "tag": "9b5226b2-acbc-4977-8451-6c4b8fb6bf30"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:13.413485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:13.414634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:13.440871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e973afc2-b5b0-4d7c-94cb-85d2c49c1e6c req-d58299fe-88a2-41ad-811c-6847788d4703 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:13.441638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e973afc2-b5b0-4d7c-94cb-85d2c49c1e6c req-d58299fe-88a2-41ad-811c-6847788d4703 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:13.442285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e973afc2-b5b0-4d7c-94cb-85d2c49c1e6c req-d58299fe-88a2-41ad-811c-6847788d4703 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:13.456653 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e973afc2-b5b0-4d7c-94cb-85d2c49c1e6c req-d58299fe-88a2-41ad-811c-6847788d4703 service nova] Creating event network-vif-plugged:9b5226b2-acbc-4977-8451-6c4b8fb6bf30 for instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c on nch1-master-1 Oct 15 21:15:13.473438 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e973afc2-b5b0-4d7c-94cb-85d2c49c1e6c req-d58299fe-88a2-41ad-811c-6847788d4703 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081290 Oct 15 21:15:13.474704 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1286/4820] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:13.498791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:13.500911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:13.518056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:13.518787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:13.519413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:13.528703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:13.530601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:13.531322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:13.638400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=14 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:14.093094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:14.098051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=100 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:14.108983 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83f77165-4162-4424-8e17-809e4e515524 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 0.620780 Oct 15 21:15:14.110993 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3535/4821] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:13 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:14.211051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:14.212129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:14.221531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:14.222341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:14.222955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:14.223889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:14.224932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:14.225590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:14.409595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:14.411013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:14.426451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:14.427482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:14.428557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:14.430435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:14.432163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:14.432982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:15.238852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:15.247610 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7c13425d-3608-432a-99fa-4222e31d3e0d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1667 microversion: 2.1 time: 1.041488 Oct 15 21:15:15.248195 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1287/4822] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:14 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1667 bytes in 1043 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:15.269819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-65e17b6e-924a-49db-bd3e-dd9f97a2595c req-61c451bf-46eb-4f9a-9c1a-2ed539954aa2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7028d90b-941e-4a2e-9535-2cc75855726b", "tag": "41e3788a-44c6-41e5-a03f-90f4f395ccba"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:15.281965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-65e17b6e-924a-49db-bd3e-dd9f97a2595c req-61c451bf-46eb-4f9a-9c1a-2ed539954aa2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:15.282520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-65e17b6e-924a-49db-bd3e-dd9f97a2595c req-61c451bf-46eb-4f9a-9c1a-2ed539954aa2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:15.283017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-65e17b6e-924a-49db-bd3e-dd9f97a2595c req-61c451bf-46eb-4f9a-9c1a-2ed539954aa2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:15.296462 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-65e17b6e-924a-49db-bd3e-dd9f97a2595c req-61c451bf-46eb-4f9a-9c1a-2ed539954aa2 service nova] Creating event network-changed:41e3788a-44c6-41e5-a03f-90f4f395ccba for instance 7028d90b-941e-4a2e-9535-2cc75855726b on nch2-master-1 Oct 15 21:15:15.303257 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-65e17b6e-924a-49db-bd3e-dd9f97a2595c req-61c451bf-46eb-4f9a-9c1a-2ed539954aa2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037282 Oct 15 21:15:15.304064 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1288/4823] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:15.315895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:15.316020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:15.323447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:15.324131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:15.324643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:15.325425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:15.326037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:15.326593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:15.398658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:15.412926 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71f50ed1-a292-4822-809d-c6278b1e9694 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1367 microversion: 2.1 time: 1.010802 Oct 15 21:15:15.417911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3536/4824] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:14 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1367 bytes in 1016 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:15.851118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:15.872678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff11173f-a500-468d-8840-0a3a773ce753 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 0.564758 Oct 15 21:15:15.888668 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1289/4825] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:15 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:16.279295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:16.279962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:16.312086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:16.316409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:16.316810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:16.324676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:16.325427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:16.332417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:16.447875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:16.448562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:16.467065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:16.468369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:16.471421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:16.472233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:16.472851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:16.475566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:16.991127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:17.010808 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-92898647-09ac-437b-ba16-c879299c34e4 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.732329 Oct 15 21:15:17.014929 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3537/4826] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:16 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:17.027463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:17.029449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:17.043559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.044595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.046919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.047786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.050026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.050588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.061485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:17.073945 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-72d1e087-7c61-47fb-810d-ca01a5ba9c47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1557 microversion: 2.1 time: 0.635333 Oct 15 21:15:17.076560 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1290/4827] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:16 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1557 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:17.087255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:17.093390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:17.101536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.102032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.102469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.103784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.109060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.113650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.602334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:17.618034 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec6c7ad2-6334-485c-97b6-453694e54778 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 0.601680 Oct 15 21:15:17.629085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3538/4828] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:17 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:17.687395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:17.692166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:17.750108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.751606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.758797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.762660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.763824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.765167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.802603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:17.820555 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bf02f318-c7d9-459b-a6f3-0544212d0dc1 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.742648 Oct 15 21:15:17.835178 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1291/4829] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:17 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 758 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:17.858612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:17.859566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:17.867051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.867699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.868442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:17.870044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:17.871510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:17.872087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:18.013836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-59c88869-bc0a-42bf-99b0-48a12bf71eb9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.383353 Oct 15 21:15:18.015528 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3539/4830] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:15:17 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-interface => generated 285 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:18.536101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:18.551056 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f0ce010-4b89-4396-a62a-df01dd2a83ae tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.696321 Oct 15 21:15:18.566333 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1292/4831] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:17 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:18.577862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:18.578923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:18.586161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:18.586980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:18.587585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:18.588552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:18.589228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:18.590009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:18.655978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:18.656899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:18.677868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:18.678685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:18.681403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:18.687248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:18.688113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:18.688557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:19.682538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:19.699421 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7953e7b-60e5-48fa-83f8-6c43150e50a7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.127306 Oct 15 21:15:19.706898 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3540/4832] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:18 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:19.731864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:19.732610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:19.751753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:19.752446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:19.753961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:19.755423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:19.757881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:19.758338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:19.783648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:19.795387 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-04fec553-955e-44f7-9716-5568d782ec39 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 1.143069 Oct 15 21:15:19.798613 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1293/4833] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:18 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 1147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:20.714837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:20.730792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fc041d2d-8f84-47aa-a866-c96c0fe27799 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 1.002746 Oct 15 21:15:20.733101 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3541/4834] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:19 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 1007 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:20.764157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:20.765339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:20.781342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:20.782102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:20.782700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:20.783825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:20.787848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:20.788873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:20.827125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lazy-loading 'flavor' on Instance uuid 9585dc89-1627-4fc5-9817-abee53538924 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:15:20.833624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:20.834355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:20.857242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:20.857808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:20.858270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:20.859133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:20.861020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:20.867621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:21.013632 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f5526f14-f524-4ef9-9b40-d32ee1dee645 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.256559 Oct 15 21:15:21.014953 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1294/4835] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:15:20 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 259 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:15:21.034841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:21.042016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:21.062170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:21.063985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:21.064485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:21.067831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:21.068239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:21.070375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:21.596319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:21.612864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:21.624892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d526ef24-68b3-4a2f-8638-64029854b97d tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 0.797813 Oct 15 21:15:21.627140 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ead5bb97-07d6-425c-8438-e762cacbf508 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1679 microversion: 2.1 time: 0.598403 Oct 15 21:15:21.627867 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3542/4836] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:20 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:21.644241 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1295/4837] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:21 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1679 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:22.006583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9df88469-a069-411d-8c69-8560a34c7c99 req-507c04d1-841e-4f7b-b691-5199add17d96 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c0812e6-ee5e-470c-a73a-5eaaca23354c", "tag": "9b5226b2-acbc-4977-8451-6c4b8fb6bf30"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:22.029456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9df88469-a069-411d-8c69-8560a34c7c99 req-507c04d1-841e-4f7b-b691-5199add17d96 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:22.029957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9df88469-a069-411d-8c69-8560a34c7c99 req-507c04d1-841e-4f7b-b691-5199add17d96 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:22.030404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9df88469-a069-411d-8c69-8560a34c7c99 req-507c04d1-841e-4f7b-b691-5199add17d96 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:22.052521 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9df88469-a069-411d-8c69-8560a34c7c99 req-507c04d1-841e-4f7b-b691-5199add17d96 service nova] Creating event network-changed:9b5226b2-acbc-4977-8451-6c4b8fb6bf30 for instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c on nch1-master-1 Oct 15 21:15:22.059582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9df88469-a069-411d-8c69-8560a34c7c99 req-507c04d1-841e-4f7b-b691-5199add17d96 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056332 Oct 15 21:15:22.060911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3543/4838] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:22.652133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:22.663312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:22.664315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:22.667090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:22.688707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:22.690271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:22.690851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:22.692335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:22.692945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:22.693617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:22.694629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:22.697785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:22.699006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:22.699360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:22.700518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:22.701769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:22.755184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:23.677615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:23.703896 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1bbe94d4-cc0d-4196-8f2e-0cc7344189be tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 1.046500 Oct 15 21:15:23.722153 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3544/4839] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:22 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:23.733385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4245461c-248f-4d7d-aeb2-1271eb43ac26 req-f9896488-6248-4d40-8a0c-4be879a426ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7028d90b-941e-4a2e-9535-2cc75855726b", "name": "network-vif-plugged", "status": "completed", "tag": "41e3788a-44c6-41e5-a03f-90f4f395ccba"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:23.750748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:23.760870 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2184c3bc-623c-466f-bc34-23f17512e208 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1662 microversion: 2.1 time: 1.113354 Oct 15 21:15:23.761158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4245461c-248f-4d7d-aeb2-1271eb43ac26 req-f9896488-6248-4d40-8a0c-4be879a426ef service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:23.761899 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1296/4840] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:22 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1662 bytes in 1115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:23.762272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4245461c-248f-4d7d-aeb2-1271eb43ac26 req-f9896488-6248-4d40-8a0c-4be879a426ef service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:23.762881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4245461c-248f-4d7d-aeb2-1271eb43ac26 req-f9896488-6248-4d40-8a0c-4be879a426ef service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:23.788960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:23.791249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:23.812957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:23.813488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:23.821090 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4245461c-248f-4d7d-aeb2-1271eb43ac26 req-f9896488-6248-4d40-8a0c-4be879a426ef service nova] Creating event network-vif-plugged:41e3788a-44c6-41e5-a03f-90f4f395ccba for instance 7028d90b-941e-4a2e-9535-2cc75855726b on nch2-master-1 Oct 15 21:15:23.826373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:23.827221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:23.827706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:23.829623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:23.834352 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4245461c-248f-4d7d-aeb2-1271eb43ac26 req-f9896488-6248-4d40-8a0c-4be879a426ef service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105701 Oct 15 21:15:23.838042 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3545/4841] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:24.017263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b3c13b10-430a-4fd9-b9de-422bafa7d8df tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action" status: 202 len: 0 microversion: 2.1 time: 0.232479 Oct 15 21:15:24.021647 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1297/4842] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:15:23 2022] POST /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924/action => generated 0 bytes in 238 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:15:24.034816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:24.036060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:24.052777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:24.054361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:24.055200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:24.056663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:24.060610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:24.061491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:24.445929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:24.459200 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4eb70e7-2c01-42e4-88fa-60f6f1fbff10 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1668 microversion: 2.1 time: 0.435740 Oct 15 21:15:24.459696 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3546/4843] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:24 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1668 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:24.741108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:24.742036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:24.754766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:24.755504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:24.756309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:24.762895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:24.763686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:24.764294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:25.146816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:25.161901 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea06fcb5-29ba-4b9c-b329-73aa4c8c32af tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1315 microversion: 2.1 time: 0.428096 Oct 15 21:15:25.162618 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1298/4844] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:24 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1315 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:25.487565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:25.488222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:25.506743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:25.507597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:25.508257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:25.510052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:25.516700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:25.517605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:26.189970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:26.191169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:26.200694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:26.206668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:26.207363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:26.208343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:26.209576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:26.211692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:26.832794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:26.857628 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6201df66-f56c-43e5-b77b-5c446a4adfeb tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1668 microversion: 2.1 time: 1.375252 Oct 15 21:15:26.859054 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3547/4845] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:25 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1668 bytes in 1377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:26.925659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:26.935074 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-097f45e8-3be7-4fca-8fbc-8bd239a61bd0 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1465 microversion: 2.1 time: 0.750733 Oct 15 21:15:26.937772 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1299/4846] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:26 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1465 bytes in 754 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:27.023359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:27.026014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:27.040394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:27.041209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:27.043419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:27.045405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:27.047883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:27.050196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:27.094554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:15:27.251659 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d360741-3910-4506-ae9f-f00df314c2eb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 204 len: 0 microversion: 2.1 time: 0.233448 Oct 15 21:15:27.252274 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3548/4847] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:15:27 2022] DELETE /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:27.260991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:27.261775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:27.274195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:27.275227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:27.275790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:27.276373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:27.276834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:27.283242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:27.843579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:27.876220 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9e4a9373-a737-4204-9de3-9f8056ea1e1a tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1471 microversion: 2.1 time: 0.611541 Oct 15 21:15:27.885519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:27.886943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:27.901714 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1300/4848] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:27 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1471 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:27.913662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:27.913662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:27.913662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:27.914030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:27.919188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:27.919680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:28.251059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:28.264519 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ddab38f-0213-4d0d-bc5c-4d9129597dd9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1668 microversion: 2.1 time: 0.382137 Oct 15 21:15:28.265264 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3549/4849] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:27 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1668 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:28.926866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:28.928468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:28.955140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:28.955766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:28.956257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:28.957015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:28.961887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:28.964872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:29.224437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:29.237006 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5af9eb7f-d761-45c4-9295-f9c617d9b511 tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1471 microversion: 2.1 time: 0.318712 Oct 15 21:15:29.241054 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1301/4850] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:28 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1471 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:29.292735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:29.293387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:29.305672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:29.317816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:29.318502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:29.319431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:29.320043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:29.320633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:29.631115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:29.646237 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-253d6135-f279-4b61-a90e-41d8b4f597d3 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.358114 Oct 15 21:15:29.647492 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3550/4851] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:29 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:29.675907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:29.676777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:29.689169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-dbaa9f62-14d7-431b-b6d3-d8dc2a4c5725 req-1b7e32b3-7359-42b2-8220-f515f4a427da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7028d90b-941e-4a2e-9535-2cc75855726b", "name": "network-vif-deleted", "tag": "41e3788a-44c6-41e5-a03f-90f4f395ccba"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:29.694479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:29.700993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:29.703963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:29.706716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:29.707662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:29.708311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:29.709648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:29.726429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dbaa9f62-14d7-431b-b6d3-d8dc2a4c5725 req-1b7e32b3-7359-42b2-8220-f515f4a427da service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:29.727209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dbaa9f62-14d7-431b-b6d3-d8dc2a4c5725 req-1b7e32b3-7359-42b2-8220-f515f4a427da service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:29.728053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-dbaa9f62-14d7-431b-b6d3-d8dc2a4c5725 req-1b7e32b3-7359-42b2-8220-f515f4a427da service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:29.767396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-dbaa9f62-14d7-431b-b6d3-d8dc2a4c5725 req-1b7e32b3-7359-42b2-8220-f515f4a427da service nova] Creating event network-vif-deleted:41e3788a-44c6-41e5-a03f-90f4f395ccba for instance 7028d90b-941e-4a2e-9535-2cc75855726b on nch2-master-1 Oct 15 21:15:29.778437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-dbaa9f62-14d7-431b-b6d3-d8dc2a4c5725 req-1b7e32b3-7359-42b2-8220-f515f4a427da service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093029 Oct 15 21:15:29.783508 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3551/4852] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:15:30.240559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:30.257174 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35ce1d8e-a10f-4386-b4d7-dd5e49db36b9 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1671 microversion: 2.1 time: 0.585007 Oct 15 21:15:30.259700 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1302/4853] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:29 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1671 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:30.275809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:30.289676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:30.291689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:30.292448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:30.308866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:30.311607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:30.312372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:30.314999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:30.315708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:30.316793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:30.331757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:30.332287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:30.332724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:30.333487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:30.336549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:30.337064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:30.426701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:15:30.465579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:30.466499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:30.466955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:30.467431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lazy-loading 'fault' on Instance uuid 7028d90b-941e-4a2e-9535-2cc75855726b {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:15:30.467768 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.api [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Working on deleting snapshot 8daa26e8-3373-4890-8d13-192c3ed2605f from shelved instance... Oct 15 21:15:30.766027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d86728459fe4fefabd2555de8608ddc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0007ee63f7340f68f40f6caf5054654', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:30.768601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:30.777365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40af5ed1-151b-40ca-af02-1c4c66901afa tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 200 len: 1305 microversion: 2.1 time: 0.518665 Oct 15 21:15:30.779567 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3552/4854] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:30 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 1305 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:15:31.804011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:31.804631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] [instance: 7028d90b-941e-4a2e-9535-2cc75855726b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:31.823028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:31.823536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:31.823990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:31.824867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:31.825385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:31.825820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:31.860961 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] HTTP exception thrown: Instance 7028d90b-941e-4a2e-9535-2cc75855726b could not be found. Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Request method failure captured: Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b HTTP/1.1 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLTgmwbaktzX1_02ML53uuDHtOmUjaxdsrpUa8RV60eesDl0pDSTJsKkvVZgpjW-L5F0sLpaP8g69c5EjOT6ztGE6fxV2N6h9P3FsEjNwePOWtJ4jAmIbepmH9Xda3rFkSPlI_fPeLOuIOVHJh7xqLf05yzRqkmbGNRms_Qa334wnE Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b0007ee63f7340f68f40f6caf5054654 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesNegativeTest-833509769 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b0007ee63f7340f68f40f6caf5054654"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b0007ee63f7340f68f40f6caf5054654"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b0007ee63f7340f68f40f6caf5054654"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b0007ee63f7340f68f40f6caf5054654"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesNegativeTest-833509769 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b0007ee63f7340f68f40f6caf5054654 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesNegativeTest-833509769 Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesNegativeTest-833509769-project Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1d86728459fe4fefabd2555de8608ddc Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesNegativeTest-833509769-project Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 7028d90b-941e-4a2e-9535-2cc75855726b could not be found. Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '7028d90b-941e-4a2e-9535-2cc75855726b'} Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7028d90b-941e-4a2e-9535-2cc75855726b could not be found. Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7028d90b-941e-4a2e-9535-2cc75855726b could not be found. Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:31.863341 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7028d90b-941e-4a2e-9535-2cc75855726b could not be found. Oct 15 21:15:31.867208 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:31.870491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] Returning 404 to user: Instance 7028d90b-941e-4a2e-9535-2cc75855726b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:15:31.871702 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a501616c-3c0c-4623-a47e-5ff88ae87edb tempest-VolumesNegativeTest-833509769 tempest-VolumesNegativeTest-833509769-project] 10.222.0.16 "GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b" status: 404 len: 111 microversion: 2.1 time: 0.071756 Oct 15 21:15:31.873215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3553/4855] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:31 2022] GET /compute/v2.1/servers/7028d90b-941e-4a2e-9535-2cc75855726b => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:15:31.967643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:31.992615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Doing local delete in cell 680a3962-acf9-40a3-9cba-9f0a72927170(cell1) {{(pid=138689) _delete /opt/stack/nova/nova/compute/api.py:2546}} Oct 15 21:15:31.994284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:31.995166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:31.995803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:31.996238 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.api [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Oct 15 21:15:32.002299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] deallocate_for_instance() {{(pid=138689) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1780}} Oct 15 21:15:33.771215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Updating instance_info_cache with network_info: [] {{(pid=138689) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 15 21:15:33.778483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-bd18d9ba-8887-41a2-ba5f-cb1e59876540 req-753eaa71-03a3-488e-914d-de74e83b0613 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6b62f776-94e1-4e10-9095-ed834dbbb229", "name": "network-vif-deleted", "tag": "b1d14b84-ac66-4849-a397-f7e2801d4ebb"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:33.800107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bd18d9ba-8887-41a2-ba5f-cb1e59876540 req-753eaa71-03a3-488e-914d-de74e83b0613 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:33.804854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bd18d9ba-8887-41a2-ba5f-cb1e59876540 req-753eaa71-03a3-488e-914d-de74e83b0613 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:33.806907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-bd18d9ba-8887-41a2-ba5f-cb1e59876540 req-753eaa71-03a3-488e-914d-de74e83b0613 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:33.829545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_external_events [req-bd18d9ba-8887-41a2-ba5f-cb1e59876540 req-753eaa71-03a3-488e-914d-de74e83b0613 service nova] Unable to find a host for instance 6b62f776-94e1-4e10-9095-ed834dbbb229. Dropping event network-vif-deleted {{(pid=138688) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 15 21:15:33.830460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-bd18d9ba-8887-41a2-ba5f-cb1e59876540 req-753eaa71-03a3-488e-914d-de74e83b0613 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.056722 Oct 15 21:15:33.841468 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3554/4856] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:33 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 68 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 15 21:15:34.586789 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c5acbc6-b905-4558-8f4c-41866e417544 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 204 len: 0 microversion: 2.1 time: 4.299675 Oct 15 21:15:34.591122 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1303/4857] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:15:30 2022] DELETE /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 0 bytes in 4305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:34.598977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:34.599681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 6b62f776-94e1-4e10-9095-ed834dbbb229] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:34.607676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:34.612321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:34.612777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:34.613548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:34.614016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:34.614452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:34.649902 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] HTTP exception thrown: Instance 6b62f776-94e1-4e10-9095-ed834dbbb229 could not be found. Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Request method failure captured: Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 HTTP/1.1 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCkMhMzgxIb6i7VoC650pGF8tykO4OjlOTVaW1cx9EsLvnp80UEJNdjmkJ9gxWxEKdx-BZyz41tnpRESXgt8mdrfybGIjbfLbEf7abh0rD68HzJxm6T7JWNMVWGCBvXV4emyeRKn3nLIRCE_kdy1202kOCKfFOyewmw-b1yDpMy11U Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1aadd87b97c247cb95321664ba0c32a9 Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6b62f776-94e1-4e10-9095-ed834dbbb229 could not be found. Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6b62f776-94e1-4e10-9095-ed834dbbb229'} Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6b62f776-94e1-4e10-9095-ed834dbbb229 could not be found. Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6b62f776-94e1-4e10-9095-ed834dbbb229 could not be found. Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:34.652200 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6b62f776-94e1-4e10-9095-ed834dbbb229 could not be found. Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Returning 404 to user: Instance 6b62f776-94e1-4e10-9095-ed834dbbb229 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:15:34.656128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2207f155-2e06-454b-8337-cfeb0481931d tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229" status: 404 len: 111 microversion: 2.1 time: 0.059210 Oct 15 21:15:34.667537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:34.668099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:34.671255 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3555/4858] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:34 2022] GET /compute/v2.1/servers/6b62f776-94e1-4e10-9095-ed834dbbb229 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:34.685863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:34.686814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:34.687919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:34.690826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:34.692740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:34.693518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:34.763009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:15:34.872951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=18,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:15:34.938992 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-eeced7cc-1cff-41e2-a417-3a5fb9a87651 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "DELETE /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 204 len: 0 microversion: 2.1 time: 0.276841 Oct 15 21:15:34.940679 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1304/4859] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:15:34 2022] DELETE /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 0 bytes in 279 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:34.950210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:34.951587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:34.963467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:34.965753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:34.966283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:34.967193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:34.968000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:34.968556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:35.598607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:35.612302 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35dbaada-2295-49fa-ab45-023f21478d91 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1466 microversion: 2.1 time: 0.667492 Oct 15 21:15:35.633313 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3556/4860] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:34 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1466 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:36.671985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:36.676596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:36.699452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:36.702034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:36.703250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:36.705025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:36.709828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:36.710444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:36.950765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:36.959548 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d21b7ac3-725a-456a-b93c-bd9b07e4378a tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1466 microversion: 2.1 time: 0.303345 Oct 15 21:15:36.960137 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1305/4861] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:36 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1466 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:37.611403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fb379543-0405-4122-8886-ae6514e247d8 req-ffa51ada-5cc2-4bdd-8632-dd09233a135c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa", "name": "network-vif-deleted", "tag": "f1ee217d-8ce5-4ef4-aa47-ea76d019efff"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:37.651631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fb379543-0405-4122-8886-ae6514e247d8 req-ffa51ada-5cc2-4bdd-8632-dd09233a135c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:37.652133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fb379543-0405-4122-8886-ae6514e247d8 req-ffa51ada-5cc2-4bdd-8632-dd09233a135c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:37.652751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fb379543-0405-4122-8886-ae6514e247d8 req-ffa51ada-5cc2-4bdd-8632-dd09233a135c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:37.732372 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fb379543-0405-4122-8886-ae6514e247d8 req-ffa51ada-5cc2-4bdd-8632-dd09233a135c service nova] Creating event network-vif-deleted:f1ee217d-8ce5-4ef4-aa47-ea76d019efff for instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa on nch1-master-1 Oct 15 21:15:37.741365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fb379543-0405-4122-8886-ae6514e247d8 req-ffa51ada-5cc2-4bdd-8632-dd09233a135c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.136238 Oct 15 21:15:37.743311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3557/4862] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:37.987241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:37.987877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:38.006938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:38.011242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:38.011876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:38.012692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:38.013153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:38.013627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:38.146607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:38.147153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:38.148750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:38.149376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lazy-loading 'fault' on Instance uuid 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:15:38.379203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:38.396304 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-140a5648-7be9-4f4e-a943-748bb467f2c5 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 200 len: 1304 microversion: 2.1 time: 0.410515 Oct 15 21:15:38.405146 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1306/4863] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:37 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 1304 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:39.423699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:39.424627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:39.443386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:39.444028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:39.444536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:39.445492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:39.446642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:39.447789 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:39.501478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] HTTP exception thrown: Instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa could not be found. Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Request method failure captured: Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa HTTP/1.1 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCkMhMzgxIb6i7VoC650pGF8tykO4OjlOTVaW1cx9EsLvnp80UEJNdjmkJ9gxWxEKdx-BZyz41tnpRESXgt8mdrfybGIjbfLbEf7abh0rD68HzJxm6T7JWNMVWGCBvXV4emyeRKn3nLIRCE_kdy1202kOCKfFOyewmw-b1yDpMy11U Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1aadd87b97c247cb95321664ba0c32a9 Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa could not be found. Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa'} Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa could not be found. Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa could not be found. Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:39.504532 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa could not be found. Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Returning 404 to user: Instance 08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:15:39.511050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d52d53b-c91e-4307-935d-a9e045d3c21c tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa" status: 404 len: 111 microversion: 2.1 time: 0.088938 Oct 15 21:15:39.512877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3558/4864] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:39 2022] GET /compute/v2.1/servers/08ad4c0f-25e9-4ba0-a55e-86aa294ac8fa => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:39.516281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:39.516832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:39.529571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:39.530443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:39.531135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:39.531948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:39.533776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:39.534296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:39.622485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:15:39.866822 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a496cdb6-8211-42c0-b0e9-9ff25809af45 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "DELETE /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 204 len: 0 microversion: 2.1 time: 0.354618 Oct 15 21:15:39.868386 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1307/4865] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:15:39 2022] DELETE /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:39.877129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:39.877767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:39.885345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:39.885883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:39.886304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:39.887026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:39.887491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:39.887925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:40.474837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:40.497298 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a9438fb-f701-48b1-b093-d9f821c2ef0e tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1506 microversion: 2.1 time: 0.624547 Oct 15 21:15:40.501001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3559/4866] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:39 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1506 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:41.740915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9f9981bd-6062-43cb-b92f-796bf108fbbb req-bac91346-a74e-422e-abcb-8044d17730f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "854dff46-3f20-4740-aef1-35486a5f9738", "name": "network-vif-deleted", "tag": "ed98b007-0df9-43dd-9145-95a58eb49e8e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:41.787965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9f9981bd-6062-43cb-b92f-796bf108fbbb req-bac91346-a74e-422e-abcb-8044d17730f8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:41.789504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9f9981bd-6062-43cb-b92f-796bf108fbbb req-bac91346-a74e-422e-abcb-8044d17730f8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:41.791433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9f9981bd-6062-43cb-b92f-796bf108fbbb req-bac91346-a74e-422e-abcb-8044d17730f8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:41.829875 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9f9981bd-6062-43cb-b92f-796bf108fbbb req-bac91346-a74e-422e-abcb-8044d17730f8 service nova] Creating event network-vif-deleted:ed98b007-0df9-43dd-9145-95a58eb49e8e for instance 854dff46-3f20-4740-aef1-35486a5f9738 on nch1-master-1 Oct 15 21:15:41.865918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9f9981bd-6062-43cb-b92f-796bf108fbbb req-bac91346-a74e-422e-abcb-8044d17730f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.132870 Oct 15 21:15:41.868794 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3560/4867] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:42.181994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:42.182808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:42.229386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:42.232701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:42.238340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:42.239764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:42.240582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:42.241215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:42.353110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:42.353550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:42.353996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:42.354684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lazy-loading 'fault' on Instance uuid 854dff46-3f20-4740-aef1-35486a5f9738 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:15:42.512634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:42.530784 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-42699873-93c3-49db-b13d-c34098d71fcc tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 200 len: 1303 microversion: 2.1 time: 1.010875 Oct 15 21:15:42.532489 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1308/4868] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:41 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 1303 bytes in 1014 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:43.563109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:43.564206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 854dff46-3f20-4740-aef1-35486a5f9738] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:43.598604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:43.599101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:43.599542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:43.607017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:43.611531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:43.612315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:43.649406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] HTTP exception thrown: Instance 854dff46-3f20-4740-aef1-35486a5f9738 could not be found. Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Request method failure captured: Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 HTTP/1.1 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyCkMhMzgxIb6i7VoC650pGF8tykO4OjlOTVaW1cx9EsLvnp80UEJNdjmkJ9gxWxEKdx-BZyz41tnpRESXgt8mdrfybGIjbfLbEf7abh0rD68HzJxm6T7JWNMVWGCBvXV4emyeRKn3nLIRCE_kdy1202kOCKfFOyewmw-b1yDpMy11U Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1aadd87b97c247cb95321664ba0c32a9 Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 854dff46-3f20-4740-aef1-35486a5f9738 could not be found. Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '854dff46-3f20-4740-aef1-35486a5f9738'} Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 854dff46-3f20-4740-aef1-35486a5f9738 could not be found. Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 854dff46-3f20-4740-aef1-35486a5f9738 could not be found. Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:43.652227 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 854dff46-3f20-4740-aef1-35486a5f9738 could not be found. Oct 15 21:15:43.661546 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:15:43.672407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Returning 404 to user: Instance 854dff46-3f20-4740-aef1-35486a5f9738 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:15:43.676567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eaf6532d-486c-40d7-9491-c0d9a6b40735 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738" status: 404 len: 111 microversion: 2.1 time: 0.120551 Oct 15 21:15:43.677638 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3561/4869] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:43 2022] GET /compute/v2.1/servers/854dff46-3f20-4740-aef1-35486a5f9738 => generated 111 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:43.686702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:43.692683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:43.715442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:43.716331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:43.716927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:43.718460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:43.719144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:43.719762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:43.788973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:15:43.901816 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a14d4cd0-19e6-4842-abd7-295342ed1469 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 204 len: 0 microversion: 2.1 time: 0.218752 Oct 15 21:15:43.904311 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1309/4870] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:15:43 2022] DELETE /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 21:15:43.913677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:43.914585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:43.927642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:43.928256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:43.928714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:43.929499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:43.933002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:43.934521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:44.229809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:44.243529 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8d14a312-8153-4271-943d-69371a900984 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1496 microversion: 2.1 time: 0.335367 Oct 15 21:15:44.245088 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3562/4871] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:43 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1496 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:45.269558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:45.270845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:45.278462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:45.279963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:45.281887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:45.282839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:45.288477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:45.289251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:45.533371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cd03dd90-a2b5-4d4f-b6b4-2e932223e38a req-73ea6ae6-be97-470c-92a3-72278610eee9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6420b15-803c-41f2-b430-b770b2f46e64", "name": "network-vif-deleted", "tag": "fdaa8af5-5d45-4b21-be18-4bc6b852d171"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:45.591066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cd03dd90-a2b5-4d4f-b6b4-2e932223e38a req-73ea6ae6-be97-470c-92a3-72278610eee9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:45.604053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cd03dd90-a2b5-4d4f-b6b4-2e932223e38a req-73ea6ae6-be97-470c-92a3-72278610eee9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:45.618230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cd03dd90-a2b5-4d4f-b6b4-2e932223e38a req-73ea6ae6-be97-470c-92a3-72278610eee9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.025s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:45.623038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:45.646750 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-192bab5e-694c-401a-8ec5-0539f83218c0 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 200 len: 1454 microversion: 2.1 time: 0.376281 Oct 15 21:15:45.648171 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1310/4872] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:45 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 1454 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:45.677498 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cd03dd90-a2b5-4d4f-b6b4-2e932223e38a req-73ea6ae6-be97-470c-92a3-72278610eee9 service nova] Creating event network-vif-deleted:fdaa8af5-5d45-4b21-be18-4bc6b852d171 for instance b6420b15-803c-41f2-b430-b770b2f46e64 on nch1-master-1 Oct 15 21:15:45.712266 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cd03dd90-a2b5-4d4f-b6b4-2e932223e38a req-73ea6ae6-be97-470c-92a3-72278610eee9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.178821 Oct 15 21:15:45.715435 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3563/4873] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:15:46.674960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:46.676550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: b6420b15-803c-41f2-b430-b770b2f46e64] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:46.691828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:46.694184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:46.694872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:46.698104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:46.698726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:46.701010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:46.741628 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] HTTP exception thrown: Instance b6420b15-803c-41f2-b430-b770b2f46e64 could not be found. Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Request method failure captured: Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 HTTP/1.1 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyCkMhMzgxIb6i7VoC650pGF8tykO4OjlOTVaW1cx9EsLvnp80UEJNdjmkJ9gxWxEKdx-BZyz41tnpRESXgt8mdrfybGIjbfLbEf7abh0rD68HzJxm6T7JWNMVWGCBvXV4emyeRKn3nLIRCE_kdy1202kOCKfFOyewmw-b1yDpMy11U Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1aadd87b97c247cb95321664ba0c32a9 Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance b6420b15-803c-41f2-b430-b770b2f46e64 could not be found. Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'b6420b15-803c-41f2-b430-b770b2f46e64'} Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6420b15-803c-41f2-b430-b770b2f46e64 could not be found. Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6420b15-803c-41f2-b430-b770b2f46e64 could not be found. Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:46.747052 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6420b15-803c-41f2-b430-b770b2f46e64 could not be found. Oct 15 21:15:46.750490 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:46.752575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Returning 404 to user: Instance b6420b15-803c-41f2-b430-b770b2f46e64 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:15:46.756346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dd0efb0f-f364-48c9-935a-e2b1efda2b6f tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64" status: 404 len: 111 microversion: 2.1 time: 0.083882 Oct 15 21:15:46.760850 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1311/4874] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:46 2022] GET /compute/v2.1/servers/b6420b15-803c-41f2-b430-b770b2f46e64 => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:46.785975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:46.786615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:46.817858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:46.818367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:46.818821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:46.829231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:46.830491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:46.835045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:46.900642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:15:47.047003 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e8674ae0-114b-4532-875d-5a429a258493 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 204 len: 0 microversion: 2.1 time: 0.282118 Oct 15 21:15:47.051403 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3564/4875] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:15:46 2022] DELETE /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:15:47.059767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:47.060591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:47.072895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:47.074852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:47.075488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:47.076405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:47.077532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:47.078680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:47.478678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:47.502346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-738c178b-6504-460c-b47a-bfeb11e65a72 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1677 microversion: 2.1 time: 0.446537 Oct 15 21:15:47.504917 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1312/4876] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:47 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1677 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:48.535984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:48.540471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:48.570828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:48.571562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:48.571909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:48.572718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:48.579792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:48.580340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:49.884124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:49.944561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-519aab77-5bed-4d36-a778-3fd4440f0e00 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1677 microversion: 2.1 time: 1.417953 Oct 15 21:15:49.952095 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3565/4877] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:48 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1677 bytes in 1426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:50.975833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:50.976491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:50.994606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:50.995121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:50.995571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:50.996301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:50.996766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:50.997232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:51.100449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aadd87b97c247cb95321664ba0c32a9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '38b9244bbab846b7baf42985cebc72c2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:15:51.134139 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c3ccbc44-3dc8-4c9d-bef9-5811a25d80b7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 200 len: 1609 microversion: 2.1 time: 0.162103 Oct 15 21:15:51.138629 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1313/4878] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:50 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 1609 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:15:51.377204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-87b60497-6579-4313-87ea-670df5a15a14 req-b8462d5c-6d4c-4bb2-991a-00f786e2bf40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9585dc89-1627-4fc5-9817-abee53538924", "name": "network-vif-deleted", "tag": "8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:51.409166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-87b60497-6579-4313-87ea-670df5a15a14 req-b8462d5c-6d4c-4bb2-991a-00f786e2bf40 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:51.409698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-87b60497-6579-4313-87ea-670df5a15a14 req-b8462d5c-6d4c-4bb2-991a-00f786e2bf40 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:51.410085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-87b60497-6579-4313-87ea-670df5a15a14 req-b8462d5c-6d4c-4bb2-991a-00f786e2bf40 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:51.437521 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-87b60497-6579-4313-87ea-670df5a15a14 req-b8462d5c-6d4c-4bb2-991a-00f786e2bf40 service nova] Creating event network-vif-deleted:8e40b9b6-e4bd-4fbd-a0f3-6eddefc95699 for instance 9585dc89-1627-4fc5-9817-abee53538924 on nch2-master-1 Oct 15 21:15:51.444424 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-87b60497-6579-4313-87ea-670df5a15a14 req-b8462d5c-6d4c-4bb2-991a-00f786e2bf40 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074065 Oct 15 21:15:51.446933 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3566/4879] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:15:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:52.169173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:52.170631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] [instance: 9585dc89-1627-4fc5-9817-abee53538924] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:52.184373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:52.184875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:52.186423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:52.207252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:52.208150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:52.208767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:52.231429 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] HTTP exception thrown: Instance 9585dc89-1627-4fc5-9817-abee53538924 could not be found. Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Request method failure captured: Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 HTTP/1.1 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyCkMhMzgxIb6i7VoC650pGF8tykO4OjlOTVaW1cx9EsLvnp80UEJNdjmkJ9gxWxEKdx-BZyz41tnpRESXgt8mdrfybGIjbfLbEf7abh0rD68HzJxm6T7JWNMVWGCBvXV4emyeRKn3nLIRCE_kdy1202kOCKfFOyewmw-b1yDpMy11U Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38b9244bbab846b7baf42985cebc72c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 38b9244bbab846b7baf42985cebc72c2 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-995038554 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1aadd87b97c247cb95321664ba0c32a9 Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-995038554-project Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 9585dc89-1627-4fc5-9817-abee53538924 could not be found. Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '9585dc89-1627-4fc5-9817-abee53538924'} Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9585dc89-1627-4fc5-9817-abee53538924 could not be found. Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9585dc89-1627-4fc5-9817-abee53538924 could not be found. Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:15:52.233412 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9585dc89-1627-4fc5-9817-abee53538924 could not be found. Oct 15 21:15:52.237331 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:15:52.242636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Returning 404 to user: Instance 9585dc89-1627-4fc5-9817-abee53538924 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:15:52.250802 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-67c8f9ef-7bac-40e3-a1a1-f592e2c4dfb7 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924" status: 404 len: 111 microversion: 2.1 time: 0.086971 Oct 15 21:15:52.251581 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1314/4880] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:15:52 2022] GET /compute/v2.1/servers/9585dc89-1627-4fc5-9817-abee53538924 => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:15:52.261727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5ceee93-8687-46ed-8631-7f1057525141 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] Calling method '.version_select at 0x7fe758517160>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:52.300307 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5ceee93-8687-46ed-8631-7f1057525141 tempest-ServerActionsTestJSON-995038554 tempest-ServerActionsTestJSON-995038554-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2116871727" status: 202 len: 0 microversion: 2.1 time: 0.043665 Oct 15 21:15:52.301352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3567/4881] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:15:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2116871727 => generated 0 bytes in 45 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:15:53.698333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:15:53.698882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:53.736406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:53.736891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:53.737376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:53.745077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:53.745571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:53.746227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:53.802990 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9af50d3c-cef7-4de9-9dee-927f404be623 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/ips" status: 200 len: 150 microversion: 2.1 time: 0.108947 Oct 15 21:15:53.809787 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1315/4882] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:15:53 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/ips => generated 150 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:15:55.784308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "970a9591-f727-42c1-abca-2a937aab458b"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:15:55.791596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:15:55.807783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:55.808555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:55.809005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:55.809761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:15:55.810307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:15:55.811014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:15:56.238300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c365e07-63d1-4540-ab42-0d9addce675e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.463677 Oct 15 21:15:56.239657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3568/4883] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:15:55 2022] POST /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments => generated 194 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:01.217975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d8cd2e4-0cf1-4e32-8ed9-6a6173637382 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: .version_select at 0x7fe74ed32c10>, body: {"keypair": {"name": "tempest-keypair-1990827327", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDTPXxCLlTcAqvSTsvpL9Oi9mb2deocr6FLIaGVEF2s1oIvpr+3sLo0T6Fxw3H/w5fxgL4x0DZtbMQ4TZfNmaMhkzyDJ29tho6MmvuKfpFodZakMlHWrpN35Pm+pnFmcUA=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:01.248598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7d8cd2e4-0cf1-4e32-8ed9-6a6173637382 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:16:01.262698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7d8cd2e4-0cf1-4e32-8ed9-6a6173637382 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:16:01.316188 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d8cd2e4-0cf1-4e32-8ed9-6a6173637382 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.102096 Oct 15 21:16:01.317599 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1316/4884] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:16:01 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:05.986922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:05.996341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2032118843", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d51b163a-0bf6-4a5b-be5f-78f88001c0b8"}], "security_groups": [{"name": "tempest-securitygroup--1139582872"}], "key_name": "tempest-keypair-1990827327", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:06.222702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=68,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:06.398512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] validate_networks() for [('d51b163a-0bf6-4a5b-be5f-78f88001c0b8', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:16:08.010041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:16:08.010677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:16:08.011133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:16:08.011723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:16:08.012551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:16:08.028141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:08.028817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:08.031488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:08.032875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:08.034365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:08.036009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:08.064948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:16:08.072582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:16:08.096449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:16:08.101653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:16:08.192570 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-db0e031f-1eac-4c55-8dc5-55bb5b6a8ca1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.207779 Oct 15 21:16:08.194800 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3569/4885] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:16:05 2022] POST /compute/v2.1/servers => generated 402 bytes in 2211 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:16:08.207291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec7120d3-8e39-464c-81a5-c50a18ab0cb5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:08.212177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec7120d3-8e39-464c-81a5-c50a18ab0cb5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:08.232329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=80,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:08.239759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=4,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:08.725266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec7120d3-8e39-464c-81a5-c50a18ab0cb5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:08.734386 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec7120d3-8e39-464c-81a5-c50a18ab0cb5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1242 microversion: 2.1 time: 0.533038 Oct 15 21:16:08.739500 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1317/4886] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:08 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1242 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:09.783138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:09.784670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:09.811780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:09.813576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:09.814043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:09.817092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:09.817771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:09.818593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:10.181139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:10.220436 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-defcf2c3-f3b8-466f-a442-a405631742e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1294 microversion: 2.1 time: 0.448305 Oct 15 21:16:10.229882 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3570/4887] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:09 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1294 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:11.257404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:11.257942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:11.274359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:11.274870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:11.275312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:11.276047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:11.276507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:11.276941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:11.454953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:11.472150 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce2212cc-124e-4737-9f87-7946681cb863 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1300 microversion: 2.1 time: 0.218745 Oct 15 21:16:11.472738 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1318/4888] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:11 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1300 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:12.502261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:12.502822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:12.521011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:12.521518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:12.521965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:12.523216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:12.523683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:12.524180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:13.162990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:13.179334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f286e9c1-333e-4a65-9358-7f138d8bffe2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 0.680259 Oct 15 21:16:13.180288 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3571/4889] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:12 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:14.218520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:14.223386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:14.238515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:14.239905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:14.240588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:14.241910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:14.243017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:14.243686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:16.165593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:16.183373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-024626cc-5a60-4842-a7b5-c361aaf13409 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 1.971330 Oct 15 21:16:16.198847 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1319/4890] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:14 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 1988 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:17.227678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:17.229187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:17.246760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:17.247868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:17.248567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:17.249610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:17.251007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:17.252253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:18.048358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:18.051925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:18.060126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69412fc4-0733-4e35-8631-462f42585aed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 0.839638 Oct 15 21:16:18.066085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3572/4891] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:17 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:18.540674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-17b4bcbe-6fc3-4904-8613-6da5afef2757 req-aeddb354-c182-4291-9c58-44e04da916ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "265795dc-9437-48ae-b7bd-626db1aaa6a4", "tag": "3fbbbcfe-1609-4659-baf8-bb604b2d15f8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:18.558740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-17b4bcbe-6fc3-4904-8613-6da5afef2757 req-aeddb354-c182-4291-9c58-44e04da916ad service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:18.559223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-17b4bcbe-6fc3-4904-8613-6da5afef2757 req-aeddb354-c182-4291-9c58-44e04da916ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:18.561581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-17b4bcbe-6fc3-4904-8613-6da5afef2757 req-aeddb354-c182-4291-9c58-44e04da916ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:18.587208 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-17b4bcbe-6fc3-4904-8613-6da5afef2757 req-aeddb354-c182-4291-9c58-44e04da916ad service nova] Creating event network-changed:3fbbbcfe-1609-4659-baf8-bb604b2d15f8 for instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 on nch2-master-1 Oct 15 21:16:18.599253 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-17b4bcbe-6fc3-4904-8613-6da5afef2757 req-aeddb354-c182-4291-9c58-44e04da916ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068628 Oct 15 21:16:18.606911 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1320/4892] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:16:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (3 switches on core 0) Oct 15 21:16:19.089467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:19.090741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:19.094569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:19.113903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:19.114699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:19.115316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:19.117179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:19.117816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:19.118425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:21.140145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:21.151566 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d6e8ca70-fef6-4e1d-92f6-2b17b6a62084 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 2.066134 Oct 15 21:16:21.152181 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3573/4893] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:19 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 2068 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:22.178499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:22.179163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:22.187166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:22.203740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:22.204240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:22.204668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:22.212787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:22.213812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:22.213812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:23.029546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:23.044132 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5f0dcb6-cd2e-413a-b1dd-f48f90fd27b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 0.869111 Oct 15 21:16:23.045762 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1321/4894] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:22 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:24.072661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:24.074395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:24.083861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:24.084727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:24.087379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:24.090754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:24.091505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:24.092178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:26.460379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:26.500296 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7a68d73-17cb-4d24-9f4e-6ec4e137be1d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 2.432430 Oct 15 21:16:26.504942 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3574/4895] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:24 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 2441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:27.528026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:27.528756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:27.548994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:27.555385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:27.555949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:27.557666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:27.559826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:27.562234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:28.923154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:28.933893 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7146d837-94fb-4d51-8220-a3bcff89f018 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 1.410769 Oct 15 21:16:28.942297 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1322/4896] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:27 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 1420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:29.967527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:29.968705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:29.989244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:29.990005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:29.991851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:29.992906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:29.993238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:29.993924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:30.860479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:30.875048 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c3d0b10-26b8-4855-acdd-85d257902355 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 0.913269 Oct 15 21:16:30.877868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3575/4897] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:29 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 917 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:31.899700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:31.900330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:31.923352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:31.927352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:31.928840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:31.931540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:31.935209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:31.944520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:32.441656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-71d2a195-d44b-4351-86ff-94c2e999dea1 req-2d413dba-2418-45fe-b70c-baf4f3d06c7a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "265795dc-9437-48ae-b7bd-626db1aaa6a4", "name": "network-vif-plugged", "status": "completed", "tag": "3fbbbcfe-1609-4659-baf8-bb604b2d15f8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:32.457668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-71d2a195-d44b-4351-86ff-94c2e999dea1 req-2d413dba-2418-45fe-b70c-baf4f3d06c7a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:32.458161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-71d2a195-d44b-4351-86ff-94c2e999dea1 req-2d413dba-2418-45fe-b70c-baf4f3d06c7a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:32.459190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-71d2a195-d44b-4351-86ff-94c2e999dea1 req-2d413dba-2418-45fe-b70c-baf4f3d06c7a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:32.505138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:32.526720 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-71d2a195-d44b-4351-86ff-94c2e999dea1 req-2d413dba-2418-45fe-b70c-baf4f3d06c7a service nova] Creating event network-vif-plugged:3fbbbcfe-1609-4659-baf8-bb604b2d15f8 for instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 on nch2-master-1 Oct 15 21:16:32.529367 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-409b844a-5c16-4f8a-a42c-dba24c981c1e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 0.618417 Oct 15 21:16:32.530335 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1323/4898] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:31 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:32.536743 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-71d2a195-d44b-4351-86ff-94c2e999dea1 req-2d413dba-2418-45fe-b70c-baf4f3d06c7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098813 Oct 15 21:16:32.543541 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3576/4899] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:16:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:33.555528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:33.556158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:33.564660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:33.577030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:33.577657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:33.585513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:33.586107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:33.586565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:34.472218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:34.486440 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f86fd1fe-fc02-43a0-9b99-37b9a09bede4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1368 microversion: 2.1 time: 0.935748 Oct 15 21:16:34.487064 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1324/4900] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:33 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1368 bytes in 937 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:35.511821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:35.512455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:35.535065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:35.536451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:35.537483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:35.538388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:35.541460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:35.542104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:36.422356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:36.476861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=45 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:36.508739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72e9c91d-e88a-46df-a35d-65e1c7d9b81e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1557 microversion: 2.1 time: 0.992616 Oct 15 21:16:36.511981 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3577/4901] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:35 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1557 bytes in 1005 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:36.538072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:36.539189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:36.550194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:36.550713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:36.558158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:36.558941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:36.559422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:36.559852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:37.072063 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f0df0b5-0e8e-4038-a278-a106f6802274 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.538272 Oct 15 21:16:37.072949 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1325/4902] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:16:36 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/os-interface => generated 284 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:37.144752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerAddressesTestJSON-server-495684803", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7a81ed0e-7270-44c1-bf44-6941426e3445"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:37.193740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=32,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:37.250875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] validate_networks() for [('7a81ed0e-7270-44c1-bf44-6941426e3445', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:16:39.072076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-02c56f0b-37a1-4a16-9e76-58dc1ff0dafb tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-aggregate-scenario-32601073", "availability_zone": "foo_zone"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:39.097816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:16:39.100665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:16:39.102246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:16:39.103573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:16:39.105077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.aggregate [None req-02c56f0b-37a1-4a16-9e76-58dc1ff0dafb tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Generated uuid a3865ab2-cea7-45d0-9e4d-d9f2545d9976 for aggregate {{(pid=138689) create /opt/stack/nova/nova/objects/aggregate.py:298}} Oct 15 21:16:39.105592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:16:39.127060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=20 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:39.129285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:39.129838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:39.133176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:39.134556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:39.136744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:39.137618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:39.145115 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-02c56f0b-37a1-4a16-9e76-58dc1ff0dafb tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 207 microversion: 2.1 time: 0.307814 Oct 15 21:16:39.147616 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1326/4903] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Oct 15 21:16:38 2022] POST /compute/v2.1/os-aggregates => generated 207 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:39.171244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f3175e9-2c8e-4999-b7cc-c3e6177cd522 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"meta_key": "meta_value"}}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:39.208209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Getting quotas for project 8d0dcf5fc5e0456a9e14fc7edbda53cc. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:16:39.232180 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f3175e9-2c8e-4999-b7cc-c3e6177cd522 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 317 microversion: 2.1 time: 0.072247 Oct 15 21:16:39.233810 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1327/4904] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 21:16:39 2022] POST /compute/v2.1/os-aggregates/19/action => generated 317 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:39.242182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Getting quotas for user 655b3dac7ef04774a94c1294ad38b292 and project 8d0dcf5fc5e0456a9e14fc7edbda53cc. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:16:39.248297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:39.277422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:39.280933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:39.282729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:39.288689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:39.291853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:39.293895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:39.297074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:16:39.306686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:16:39.344077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=43 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:39.378076 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da161fb0-fe1c-4b9a-a6ec-a9bef891795f tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.135711 Oct 15 21:16:39.383111 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1328/4905] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Oct 15 21:16:39 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:39.391657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2650b418-8aff-433e-8f9e-f70cca4d2223 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:39.415273 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2650b418-8aff-433e-8f9e-f70cca4d2223 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 296 microversion: 2.1 time: 0.026114 Oct 15 21:16:39.417390 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1329/4906] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Oct 15 21:16:39 2022] GET /compute/v2.1/os-aggregates => generated 296 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:39.432437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-feb6c5c6-5a66-45df-bfc2-03e8fda7ca1e tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:39.464835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-feb6c5c6-5a66-45df-bfc2-03e8fda7ca1e tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:39.465534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-feb6c5c6-5a66-45df-bfc2-03e8fda7ca1e tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:39.466039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-feb6c5c6-5a66-45df-bfc2-03e8fda7ca1e tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:39.473391 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f32b99e9-6056-4d07-9700-805ca97f236e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.663984 Oct 15 21:16:39.481741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3578/4907] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:16:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 2673 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:16:39.487694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c0e1cd5-0c4a-4c64-8e63-b668249d3d32 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:39.489283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c0e1cd5-0c4a-4c64-8e63-b668249d3d32 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:39.782124 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-feb6c5c6-5a66-45df-bfc2-03e8fda7ca1e tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 308 microversion: 2.1 time: 0.355779 Oct 15 21:16:39.786068 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1330/4908] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 21:16:39 2022] POST /compute/v2.1/os-aggregates/19/action => generated 308 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:39.796880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-663c7ac7-c33b-480f-8030-52ead56d07e9 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:39.822928 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-663c7ac7-c33b-480f-8030-52ead56d07e9 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/19" status: 200 len: 308 microversion: 2.1 time: 0.030755 Oct 15 21:16:39.825343 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1331/4909] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:16:39 2022] GET /compute/v2.1/os-aggregates/19 => generated 308 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:16:39.868413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ac703919-9fc5-4841-8875-5e4ba2b14be0 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-renamed-aggregate-scenario-1920627127", "availability_zone": "foo_zone"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:39.869865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c0e1cd5-0c4a-4c64-8e63-b668249d3d32 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:39.887149 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c0e1cd5-0c4a-4c64-8e63-b668249d3d32 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1216 microversion: 2.1 time: 0.403607 Oct 15 21:16:39.889165 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3579/4910] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:39 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1216 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:40.292086 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ac703919-9fc5-4841-8875-5e4ba2b14be0 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-aggregates/19" status: 200 len: 342 microversion: 2.1 time: 0.441468 Oct 15 21:16:40.293729 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1332/4911] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Oct 15 21:16:39 2022] PUT /compute/v2.1/os-aggregates/19 => generated 342 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:40.306619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19f38369-06fe-4281-9b1b-f667eacaf6dc tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"foo": "bar"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:40.364387 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19f38369-06fe-4281-9b1b-f667eacaf6dc tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 356 microversion: 2.1 time: 0.061507 Oct 15 21:16:40.379814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c001221-b183-49fa-a1f1-e3e241164c54 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:40.386961 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3580/4912] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 21:16:40 2022] POST /compute/v2.1/os-aggregates/19/action => generated 356 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:40.399039 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c001221-b183-49fa-a1f1-e3e241164c54 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/19" status: 200 len: 356 microversion: 2.1 time: 0.021373 Oct 15 21:16:40.414767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ff436c66-28fb-43b6-bcfe-0e5fe54c2666 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:40.417609 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1333/4913] 10.222.0.16 () {58 vars in 1152 bytes} [Sat Oct 15 21:16:40 2022] GET /compute/v2.1/os-aggregates/19 => generated 356 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:40.456706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff436c66-28fb-43b6-bcfe-0e5fe54c2666 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:40.460438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff436c66-28fb-43b6-bcfe-0e5fe54c2666 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:40.461028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ff436c66-28fb-43b6-bcfe-0e5fe54c2666 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:40.915946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:40.916986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:40.930566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:40.931239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:40.934121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:40.936234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:40.938328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:40.944600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:41.145071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:41.168219 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9ea80dcb-15b0-43c2-8a0e-6d6d19ffe44f tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1216 microversion: 2.1 time: 0.258245 Oct 15 21:16:41.169063 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1334/4914] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:40 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1216 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:16:41.409165 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ff436c66-28fb-43b6-bcfe-0e5fe54c2666 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 341 microversion: 2.1 time: 0.995371 Oct 15 21:16:41.410775 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3581/4915] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Oct 15 21:16:40 2022] POST /compute/v2.1/os-aggregates/19/action => generated 341 bytes in 1002 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:41.427589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bf25998-cb72-4750-912e-c3324a9d2a41 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:41.495076 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6bf25998-cb72-4750-912e-c3324a9d2a41 tempest-TestAggregatesBasicOps-2005509526 tempest-TestAggregatesBasicOps-2005509526-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/19" status: 200 len: 0 microversion: 2.1 time: 0.072875 Oct 15 21:16:41.500604 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1335/4916] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Oct 15 21:16:41 2022] DELETE /compute/v2.1/os-aggregates/19 => generated 0 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Oct 15 21:16:42.197724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:42.197870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6efa756a-5cba-4757-8292-a35b9089a615 req-949a3383-276b-4ea8-9ed3-91f2448dca44 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "265795dc-9437-48ae-b7bd-626db1aaa6a4", "tag": "3fbbbcfe-1609-4659-baf8-bb604b2d15f8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:42.198798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:42.207914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:42.216064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:42.218143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:42.219976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:42.220657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:42.224818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:42.236028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6efa756a-5cba-4757-8292-a35b9089a615 req-949a3383-276b-4ea8-9ed3-91f2448dca44 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:42.244520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6efa756a-5cba-4757-8292-a35b9089a615 req-949a3383-276b-4ea8-9ed3-91f2448dca44 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:42.247397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6efa756a-5cba-4757-8292-a35b9089a615 req-949a3383-276b-4ea8-9ed3-91f2448dca44 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:42.322570 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6efa756a-5cba-4757-8292-a35b9089a615 req-949a3383-276b-4ea8-9ed3-91f2448dca44 service nova] Creating event network-changed:3fbbbcfe-1609-4659-baf8-bb604b2d15f8 for instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 on nch2-master-1 Oct 15 21:16:42.373976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6efa756a-5cba-4757-8292-a35b9089a615 req-949a3383-276b-4ea8-9ed3-91f2448dca44 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.615590 Oct 15 21:16:42.374793 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3582/4917] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:16:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:43.196703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:43.210677 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aed8d5fa-f2b2-47a9-8e32-35235463cf97 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1286 microversion: 2.1 time: 1.015807 Oct 15 21:16:43.213382 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1336/4918] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:42 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1286 bytes in 1024 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:44.242391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:44.243262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:44.259146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:44.260289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:44.262168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:44.266778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:44.269181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:44.269869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:44.547351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:44.559173 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-637047f1-c467-41ad-960e-c65580e9b5d1 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1274 microversion: 2.1 time: 0.322881 Oct 15 21:16:44.569186 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3583/4919] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:44 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1274 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:45.591351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:45.592267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:45.612006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:45.612609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:45.613103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:45.613955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:45.615429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:45.622335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:46.311771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:46.322787 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c821ac64-b37f-4418-8def-e30b0c604e4e tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1274 microversion: 2.1 time: 0.736728 Oct 15 21:16:46.323382 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1337/4920] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:45 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1274 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:47.353192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:47.354821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:47.366846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:47.368899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:47.369581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:47.370852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:47.371616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:47.372230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:48.448891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:48.461433 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c040355f-8452-40b0-8a99-46d3f0f4953c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 1.114772 Oct 15 21:16:48.462147 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3584/4921] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:47 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 1116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:49.490537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:49.491293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:49.494786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:49.530663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:49.532002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:49.534707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:49.537274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:49.541081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:49.541567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:49.623153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7a7a13d3-5597-41d0-bceb-b80f7e9c1707 req-bb3366e7-d601-4bee-84d6-a78b07382f89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7496d5d-2f2b-4990-8185-9523feefff9c", "tag": "58b4cebd-4b78-4dbb-bc6e-575f20338778"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:49.652257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:49.654677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7a7a13d3-5597-41d0-bceb-b80f7e9c1707 req-bb3366e7-d601-4bee-84d6-a78b07382f89 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:49.655167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7a7a13d3-5597-41d0-bceb-b80f7e9c1707 req-bb3366e7-d601-4bee-84d6-a78b07382f89 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:49.655600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7a7a13d3-5597-41d0-bceb-b80f7e9c1707 req-bb3366e7-d601-4bee-84d6-a78b07382f89 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:49.710015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7a7a13d3-5597-41d0-bceb-b80f7e9c1707 req-bb3366e7-d601-4bee-84d6-a78b07382f89 service nova] Creating event network-changed:58b4cebd-4b78-4dbb-bc6e-575f20338778 for instance c7496d5d-2f2b-4990-8185-9523feefff9c on nch1-master-1 Oct 15 21:16:49.720534 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7a7a13d3-5597-41d0-bceb-b80f7e9c1707 req-bb3366e7-d601-4bee-84d6-a78b07382f89 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101636 Oct 15 21:16:49.724960 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3585/4922] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:16:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:50.336165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:50.353608 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-361ae7b4-bc84-4ca6-9626-18d532d2cfe6 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 0.869011 Oct 15 21:16:50.357284 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1338/4923] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:49 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:51.380668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:51.382014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:51.384336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:16:51.405935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:51.406439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:51.406876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:51.407782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:51.408247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:51.408677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:51.757424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:51.765914 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba3590d1-df21-4e58-b947-a1254020fb2b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 0.389683 Oct 15 21:16:51.766515 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3586/4924] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:51 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:52.795472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:52.796094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:52.814199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:52.815012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:52.815618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:52.816751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:52.817705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:52.818333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:53.845470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:53.886378 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f593e304-574f-4be9-817b-b57f1f3002de tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 1.094613 Oct 15 21:16:53.895520 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1339/4925] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:52 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 1104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:54.919381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:54.920119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:54.933540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:54.934277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:54.935018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:54.936354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:54.937071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:54.937778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:55.319715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:55.343275 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c19f71c5-1ccf-42ea-8483-8930cb8ead36 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 0.428141 Oct 15 21:16:55.364580 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3587/4926] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:54 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:56.370933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:56.372360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:56.399225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:56.399725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:56.409845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:56.411422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:56.411944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:56.412392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:56.869219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:56.882993 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5ca1c395-42f9-4040-b3e3-f02e0e6db08b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 0.515941 Oct 15 21:16:56.886119 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1340/4927] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:56 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:57.909566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:57.911858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:57.946685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:57.947869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:57.950342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:57.968083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:57.970151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:57.972191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:57.981928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b2c148ab-3511-43fb-9e49-d34ad0ee7b3a req-06385527-ded2-4afa-800c-a847ed0ed3b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7496d5d-2f2b-4990-8185-9523feefff9c", "name": "network-vif-plugged", "status": "completed", "tag": "58b4cebd-4b78-4dbb-bc6e-575f20338778"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:16:58.018444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b2c148ab-3511-43fb-9e49-d34ad0ee7b3a req-06385527-ded2-4afa-800c-a847ed0ed3b8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:58.018938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b2c148ab-3511-43fb-9e49-d34ad0ee7b3a req-06385527-ded2-4afa-800c-a847ed0ed3b8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:58.019382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b2c148ab-3511-43fb-9e49-d34ad0ee7b3a req-06385527-ded2-4afa-800c-a847ed0ed3b8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:58.051767 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b2c148ab-3511-43fb-9e49-d34ad0ee7b3a req-06385527-ded2-4afa-800c-a847ed0ed3b8 service nova] Creating event network-vif-plugged:58b4cebd-4b78-4dbb-bc6e-575f20338778 for instance c7496d5d-2f2b-4990-8185-9523feefff9c on nch1-master-1 Oct 15 21:16:58.070849 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b2c148ab-3511-43fb-9e49-d34ad0ee7b3a req-06385527-ded2-4afa-800c-a847ed0ed3b8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097994 Oct 15 21:16:58.071736 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1341/4928] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:16:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:16:58.793762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:16:58.825937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57741bd4-9c93-4f59-b0ab-c26925fc52b2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1316 microversion: 2.1 time: 0.917381 Oct 15 21:16:58.827565 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3588/4929] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:57 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1316 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:16:59.851897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:16:59.852686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:16:59.869924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:59.871370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:59.873723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:16:59.877710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:16:59.878423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:16:59.882320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.451026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:00.478035 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d7db2d0c-5fd9-4a68-ba84-ffbf7adf1567 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1504 microversion: 2.1 time: 0.632210 Oct 15 21:17:00.478969 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1342/4930] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:16:59 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1504 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:00.518404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:00.518946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:00.541863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.546735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.547261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.561727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.562231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.562680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.656925 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1b58a38b-ae61-4dfe-8038-e0087eac760b tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c/ips" status: 200 len: 108 microversion: 2.1 time: 0.153040 Oct 15 21:17:00.683428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:00.683939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:00.693592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3589/4931] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:17:00 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c/ips => generated 108 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:00.700633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.704644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.705137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.705910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.706455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.706852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.770946 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-da4be922-f82e-4276-bec6-0d59471f28b7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c/ips" status: 200 len: 108 microversion: 2.1 time: 0.091254 Oct 15 21:17:00.780429 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1343/4932] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:17:00 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c/ips => generated 108 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:00.784589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:00.786247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:00.799184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.806537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.812344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.813548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.814221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.816223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.853701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ec6fdaf-feb3-4eb9-b530-ea323fee4be7 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c/ips/tempest-ServerAddressesTestJSON-616929736-network" status: 200 len: 93 microversion: 2.1 time: 0.072818 Oct 15 21:17:00.855055 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3590/4933] 10.222.0.16 () {58 vars in 1398 bytes} [Sat Oct 15 21:17:00 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c/ips/tempest-ServerAddressesTestJSON-616929736-network => generated 93 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:17:00.867295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:00.867857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:00.878945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.882097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.883316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:00.884402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:00.887503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:00.888208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:01.002885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:17:01.132288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6481aaa2-fde3-4d26-a2fb-920e7d30b4be tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 204 len: 0 microversion: 2.1 time: 0.269032 Oct 15 21:17:01.134647 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1344/4934] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:17:00 2022] DELETE /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:17:01.144519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:01.145625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:01.171285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:01.172345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:01.173183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:01.174719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:01.175625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:01.177291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:01.742267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:01.758485 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-acb59170-19ae-4d78-a5ca-2324c499c264 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1510 microversion: 2.1 time: 0.617255 Oct 15 21:17:01.760941 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3591/4935] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:01 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1510 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:02.801758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:02.803138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:02.814664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:02.817463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:02.819237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:02.820189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:02.822330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:02.825750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:03.839860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:03.852834 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e6aaac1-f505-4f5b-8447-c7db10147d8c tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1510 microversion: 2.1 time: 1.057020 Oct 15 21:17:03.871455 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1345/4936] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:02 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1510 bytes in 1062 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:04.882591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:04.883311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:04.893637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:04.895492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:04.898621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:04.900740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:04.901509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:04.907014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:05.141498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-28bca4df-f01e-41d8-9847-8c215dcf8cc5 req-719a844c-d909-422e-968e-f981cdca385e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7496d5d-2f2b-4990-8185-9523feefff9c", "name": "network-vif-deleted", "tag": "58b4cebd-4b78-4dbb-bc6e-575f20338778"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:05.189672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-28bca4df-f01e-41d8-9847-8c215dcf8cc5 req-719a844c-d909-422e-968e-f981cdca385e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:05.192336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-28bca4df-f01e-41d8-9847-8c215dcf8cc5 req-719a844c-d909-422e-968e-f981cdca385e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:05.192849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-28bca4df-f01e-41d8-9847-8c215dcf8cc5 req-719a844c-d909-422e-968e-f981cdca385e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:05.243827 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-28bca4df-f01e-41d8-9847-8c215dcf8cc5 req-719a844c-d909-422e-968e-f981cdca385e service nova] Creating event network-vif-deleted:58b4cebd-4b78-4dbb-bc6e-575f20338778 for instance c7496d5d-2f2b-4990-8185-9523feefff9c on nch1-master-1 Oct 15 21:17:05.255568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '655b3dac7ef04774a94c1294ad38b292', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0dcf5fc5e0456a9e14fc7edbda53cc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:05.260110 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-28bca4df-f01e-41d8-9847-8c215dcf8cc5 req-719a844c-d909-422e-968e-f981cdca385e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.123701 Oct 15 21:17:05.260731 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1346/4937] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:05.270125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccce9111-637f-471f-9641-272c523c5876 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 200 len: 1468 microversion: 2.1 time: 0.393838 Oct 15 21:17:05.271904 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3592/4938] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:04 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 1468 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:06.298714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:06.299590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] [instance: c7496d5d-2f2b-4990-8185-9523feefff9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:06.319308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:06.335285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:06.341121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:06.343662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:06.345965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:06.352270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:06.373228 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] HTTP exception thrown: Instance c7496d5d-2f2b-4990-8185-9523feefff9c could not be found. Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Request method failure captured: Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c HTTP/1.1 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyM0ndhgNsBhmNVP9p_bdsyX-SpZd6V-r8qg1FyVlRiJMsmkAHO0UJlTcwLsa6B9htFkNNgoJ27ZOn8ymEaP8yELOk6TNlT8ZUkDsRUR_pT8SkpVZNoynEJCUuHCe83TMwaRuj2i7PmcgpBFmFje7x673o_z3_36UJvYhubN7SvL8kw Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 8d0dcf5fc5e0456a9e14fc7edbda53cc Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerAddressesTestJSON-810557019 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d0dcf5fc5e0456a9e14fc7edbda53cc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8d0dcf5fc5e0456a9e14fc7edbda53cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8d0dcf5fc5e0456a9e14fc7edbda53cc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8d0dcf5fc5e0456a9e14fc7edbda53cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerAddressesTestJSON-810557019 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 8d0dcf5fc5e0456a9e14fc7edbda53cc Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerAddressesTestJSON-810557019 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerAddressesTestJSON-810557019-project Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 655b3dac7ef04774a94c1294ad38b292 Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerAddressesTestJSON-810557019-project Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance c7496d5d-2f2b-4990-8185-9523feefff9c could not be found. Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'c7496d5d-2f2b-4990-8185-9523feefff9c'} Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7496d5d-2f2b-4990-8185-9523feefff9c could not be found. Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7496d5d-2f2b-4990-8185-9523feefff9c could not be found. Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:06.375384 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7496d5d-2f2b-4990-8185-9523feefff9c could not be found. Oct 15 21:17:06.379970 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:06.382211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] Returning 404 to user: Instance c7496d5d-2f2b-4990-8185-9523feefff9c could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:06.383850 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a1923bb2-d96c-4fd4-ac90-4092f0934db2 tempest-ServerAddressesTestJSON-810557019 tempest-ServerAddressesTestJSON-810557019-project] 10.222.0.16 "GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c" status: 404 len: 111 microversion: 2.1 time: 0.089427 Oct 15 21:17:06.385487 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1347/4939] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:06 2022] GET /compute/v2.1/servers/c7496d5d-2f2b-4990-8185-9523feefff9c => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:17:09.011492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesBackupsTest-instance-936265395", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:09.024502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=4,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:09.103074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:17:10.365166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:17:10.365799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:17:10.366060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:17:10.366476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:17:10.366893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:17:10.390312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:10.390815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:10.391261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:10.392003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:10.392463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:10.392890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:10.409082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=62 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:10.422043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Getting quotas for project 6b0324b37fb14723b8c4071057f61fb9. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:17:10.429082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Getting quotas for user a41804510e0746c781fa0f73bce0ecf2 and project 6b0324b37fb14723b8c4071057f61fb9. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:17:10.443263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:17:10.445943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:17:10.556810 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71f6f796-0239-42b9-b410-7728d0a2cd41 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.549233 Oct 15 21:17:10.564962 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3593/4940] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:17:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 1558 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:17:10.569479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7cef8eb0-90c4-40c4-b3bb-90664ba564ee tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:10.570094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7cef8eb0-90c4-40c4-b3bb-90664ba564ee tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:10.574745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=4,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:10.877365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7cef8eb0-90c4-40c4-b3bb-90664ba564ee tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:10.890881 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7cef8eb0-90c4-40c4-b3bb-90664ba564ee tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1213 microversion: 2.1 time: 0.325854 Oct 15 21:17:10.891469 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1348/4941] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:10 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1213 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:11.920359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:11.921803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:11.928843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:11.929624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:11.930227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:11.931132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:11.931758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:11.932352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:12.080509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:12.091333 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ce80bfe-4c75-48df-9013-2af03b4fcd3c tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1265 microversion: 2.1 time: 0.175837 Oct 15 21:17:12.110218 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3594/4942] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:11 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1265 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:13.122369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:13.123664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:13.133993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:13.135895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:13.137983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:13.139354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:13.141191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:13.143221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:13.176873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=78,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:13.367836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:13.376075 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-44e4b174-979d-42ca-b43e-95995824431f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1271 microversion: 2.1 time: 0.257597 Oct 15 21:17:13.401649 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1349/4943] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:13 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1271 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:14.416015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:14.416632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:14.424248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:14.429783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:14.430451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:14.434777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:14.435261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:14.436437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:14.681688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:14.700258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a923b35-3370-4e29-a971-e8338d4df959 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1271 microversion: 2.1 time: 0.288124 Oct 15 21:17:14.701761 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3595/4944] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:14 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1271 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:15.725070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:15.726610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:15.746233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:15.746998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:15.747826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:15.750051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:15.750613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:15.752129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:16.117123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:16.134881 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-53d4ed0f-1868-4c94-8e8f-39108afff6e7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.412489 Oct 15 21:17:16.137661 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1350/4945] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:15 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:17.156031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-54a4e2b9-24df-47a0-9c7a-c5a6779837f5 req-1264411d-8560-42f7-9e8c-2c4031c52401 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b697ac2b-8135-4136-9629-b01a9afbb399", "tag": "0efd2717-e25d-490c-b4f1-61a62b3ee75b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:17.165398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:17.174023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:17.190360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-54a4e2b9-24df-47a0-9c7a-c5a6779837f5 req-1264411d-8560-42f7-9e8c-2c4031c52401 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:17.191220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-54a4e2b9-24df-47a0-9c7a-c5a6779837f5 req-1264411d-8560-42f7-9e8c-2c4031c52401 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:17.191976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-54a4e2b9-24df-47a0-9c7a-c5a6779837f5 req-1264411d-8560-42f7-9e8c-2c4031c52401 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:17.208106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:17.209635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:17.212320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:17.213949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:17.224854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:17.225392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:17.234418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-54a4e2b9-24df-47a0-9c7a-c5a6779837f5 req-1264411d-8560-42f7-9e8c-2c4031c52401 service nova] Creating event network-changed:0efd2717-e25d-490c-b4f1-61a62b3ee75b for instance b697ac2b-8135-4136-9629-b01a9afbb399 on nch2-master-1 Oct 15 21:17:17.246431 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-54a4e2b9-24df-47a0-9c7a-c5a6779837f5 req-1264411d-8560-42f7-9e8c-2c4031c52401 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.148738 Oct 15 21:17:17.247458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3596/4946] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:17.648144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:17.656353 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de7fbdc8-baf1-4de4-8995-b683b93b6d6b tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.498106 Oct 15 21:17:17.669899 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1351/4947] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:17 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:18.682910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:18.684702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:18.697977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:18.703696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:18.704278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:18.705071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:18.705557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:18.713829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:19.405945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:19.416060 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e1b8077-4bd5-45dc-a789-c65a13b0fba7 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.737585 Oct 15 21:17:19.416651 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3597/4948] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:18 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:20.440560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:20.441430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:20.452715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:20.455989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:20.458108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:20.460526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:20.462618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:20.465245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:20.882483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:21.054350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:21.063450 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-efe9ac39-3f62-48ff-aeb2-ae6ef61bc3e4 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.629201 Oct 15 21:17:21.069432 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1352/4949] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:20 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:22.097121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:22.098407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:22.101565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:22.108404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:22.119999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:22.126394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:22.127021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:22.130847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:22.132315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:22.133722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:22.550636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:22.563180 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1cb014bf-b53e-46e8-92b2-3228cadbb208 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.471183 Oct 15 21:17:22.564065 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3598/4950] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:22 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:23.589947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:23.590562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:23.601330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:23.604043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:23.606871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:23.607895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:23.608764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:23.610852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:23.866432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:23.900500 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5e87c3cc-13dd-45f9-b0d3-6153666b1495 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.296172 Oct 15 21:17:23.907796 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1353/4951] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:23 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:23.946817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b6d84124-9fb9-44d2-b92f-49576dbd99a5 req-12541471-f9cc-47dc-8e56-09d2ea11ae65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b697ac2b-8135-4136-9629-b01a9afbb399", "name": "network-vif-plugged", "status": "completed", "tag": "0efd2717-e25d-490c-b4f1-61a62b3ee75b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:23.962779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6d84124-9fb9-44d2-b92f-49576dbd99a5 req-12541471-f9cc-47dc-8e56-09d2ea11ae65 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:23.964167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6d84124-9fb9-44d2-b92f-49576dbd99a5 req-12541471-f9cc-47dc-8e56-09d2ea11ae65 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:23.965258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b6d84124-9fb9-44d2-b92f-49576dbd99a5 req-12541471-f9cc-47dc-8e56-09d2ea11ae65 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:23.979906 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b6d84124-9fb9-44d2-b92f-49576dbd99a5 req-12541471-f9cc-47dc-8e56-09d2ea11ae65 service nova] Creating event network-vif-plugged:0efd2717-e25d-490c-b4f1-61a62b3ee75b for instance b697ac2b-8135-4136-9629-b01a9afbb399 on nch2-master-1 Oct 15 21:17:23.986220 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b6d84124-9fb9-44d2-b92f-49576dbd99a5 req-12541471-f9cc-47dc-8e56-09d2ea11ae65 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043812 Oct 15 21:17:23.987393 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3599/4952] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:17:24.935898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:24.936608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:24.960887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:24.961448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:24.962088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:24.966093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:24.966606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:24.967073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:25.551479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:25.560305 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b04b9092-b19d-4573-a9b0-4a2c8bf20186 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1313 microversion: 2.1 time: 0.628982 Oct 15 21:17:25.560465 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1354/4953] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:24 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1313 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:26.587255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:26.588764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:26.610513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:26.613885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:26.614632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:26.615731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:26.617474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:26.619651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:27.094219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:27.106252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec25b0d1-f3ef-4446-8d4c-75ad831138c5 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1462 microversion: 2.1 time: 0.526945 Oct 15 21:17:27.108185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3600/4954] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:26 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1462 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:27.137394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c2bfc4bf-132f-412e-b190-07d6f1ae77af", "device": "/dev/sdb"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:27.142373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:27.166633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:27.167256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:27.170925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:27.171851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:27.172658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:27.173309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:28.229352 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69fcaeb8-defa-426a-84c4-66364b21bf04 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "POST /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.096676 Oct 15 21:17:28.231552 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1355/4955] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:17:27 2022] POST /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399/os-volume_attachments => generated 194 bytes in 1100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:30.684538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerAddressesNegativeTestJSON-server-1008597336", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "3432a067-3042-4dba-955b-c153260eee79"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:30.799708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] validate_networks() for [('3432a067-3042-4dba-955b-c153260eee79', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:17:32.555094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:17:32.555713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:17:32.556279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:17:32.556970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:17:32.557724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:17:32.571676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:32.572697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:32.577264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:32.578363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:32.579060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:32.579928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:32.602284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Getting quotas for project 66100212a95b45afb4d16f48f0fca0e3. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:17:32.609803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Getting quotas for user 58294fd300b840db9d21d4c8de89a144 and project 66100212a95b45afb4d16f48f0fca0e3. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:17:32.625323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:17:32.628450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:17:32.688503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ab6342c-8fbb-4216-97c2-2d94566205cd tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.249114 Oct 15 21:17:32.698092 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3601/4956] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:17:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 2259 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:17:32.718906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4ff85600-827f-4d18-8b8e-03992911d8d5 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:32.725592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4ff85600-827f-4d18-8b8e-03992911d8d5 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:32.842163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4ff85600-827f-4d18-8b8e-03992911d8d5 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:32.850650 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4ff85600-827f-4d18-8b8e-03992911d8d5 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1225 microversion: 2.1 time: 0.151372 Oct 15 21:17:32.851217 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1356/4957] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:32 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1225 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:33.875146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:33.875912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:33.887755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:33.889971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:33.893231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:33.894257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:33.894819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:33.895326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:34.017190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:34.040813 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48b2267c-de13-4a55-9e09-ca7aab88f064 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1221 microversion: 2.1 time: 0.171355 Oct 15 21:17:34.043902 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3602/4958] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:33 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1221 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:35.069944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:35.070567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:35.083295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:35.083805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:35.084255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:35.084986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:35.092975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:35.093699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:35.291778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:35.330048 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de741d25-8343-40c2-8a4c-5ddd683962ae tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1283 microversion: 2.1 time: 0.264748 Oct 15 21:17:35.338211 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1357/4959] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:35 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1283 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:36.364603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:36.365386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:36.388942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:36.389565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:36.390018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:36.390828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:36.391297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:36.391728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:37.009125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:37.025957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77bb9fda-d4a3-42a2-9777-4843cafb9ab4 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.664372 Oct 15 21:17:37.028041 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3603/4960] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:36 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:38.050883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:38.052311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:38.069557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:38.070064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:38.070496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:38.071212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:38.071667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:38.072097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:38.635616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:38.648550 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9bf09b7b-e56d-4e2e-b2c2-28bdd21f1d05 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.600659 Oct 15 21:17:38.650764 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1358/4961] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:38 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:39.123362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2b5338cf-f74e-4b7a-8196-b4553365bbda req-808b1fc5-a80c-4d90-8a0f-67b56b5c61de service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dfe2893c-e662-47f4-bfeb-d2abf9e407a4", "tag": "e0086a84-25a4-435d-aa79-fd167f4f989a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:39.149325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=44,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:39.152937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2b5338cf-f74e-4b7a-8196-b4553365bbda req-808b1fc5-a80c-4d90-8a0f-67b56b5c61de service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:39.153653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2b5338cf-f74e-4b7a-8196-b4553365bbda req-808b1fc5-a80c-4d90-8a0f-67b56b5c61de service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:39.154257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2b5338cf-f74e-4b7a-8196-b4553365bbda req-808b1fc5-a80c-4d90-8a0f-67b56b5c61de service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:39.232075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2b5338cf-f74e-4b7a-8196-b4553365bbda req-808b1fc5-a80c-4d90-8a0f-67b56b5c61de service nova] Creating event network-changed:e0086a84-25a4-435d-aa79-fd167f4f989a for instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 on nch1-master-1 Oct 15 21:17:39.252068 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2b5338cf-f74e-4b7a-8196-b4553365bbda req-808b1fc5-a80c-4d90-8a0f-67b56b5c61de service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.134531 Oct 15 21:17:39.259118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3604/4962] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:39.676119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:39.676917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:39.700784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:39.701451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:39.702819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:39.711333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:39.711975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:39.712429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.395899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:40.400257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:40.412341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.414177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.414829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.416093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.416831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.417522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.430786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=52 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:40.448813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:40.475796 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9794bfbb-18a5-44bd-aa6c-db4158535983 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/ips" status: 200 len: 148 microversion: 2.1 time: 0.092018 Oct 15 21:17:40.476731 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3605/4963] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:17:40 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/ips => generated 148 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:40.495196 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-541f6ad8-1067-48c3-b898-2244004b6965 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.815774 Oct 15 21:17:40.498288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "970a9591-f727-42c1-abca-2a937aab458b"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:40.500587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:40.505090 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1359/4964] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:39 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:40.508691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.509469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.510062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.510967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.511866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.512649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.666499 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Invalid volume: volume 970a9591-f727-42c1-abca-2a937aab458b is already attached to instances: 3c0812e6-ee5e-470c-a73a-5eaaca23354c Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/os-volume_attachments HTTP/1.1 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 74 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: {"volumeAttachment": {"volumeId": "970a9591-f727-42c1-abca-2a937aab458b"}} Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid volume: volume 970a9591-f727-42c1-abca-2a937aab458b is already attached to instances: 3c0812e6-ee5e-470c-a73a-5eaaca23354c Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '265795dc-9437-48ae-b7bd-626db1aaa6a4', 'body': {'volumeAttachment': {'volumeId': '970a9591-f727-42c1-abca-2a937aab458b'}}} Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 970a9591-f727-42c1-abca-2a937aab458b is already attached to instances: 3c0812e6-ee5e-470c-a73a-5eaaca23354c Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:17:40.670030 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 970a9591-f727-42c1-abca-2a937aab458b is already attached to instances: 3c0812e6-ee5e-470c-a73a-5eaaca23354c Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 970a9591-f727-42c1-abca-2a937aab458b is already attached to instances: 3c0812e6-ee5e-470c-a73a-5eaaca23354c Oct 15 21:17:40.675248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:40.677630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 400 to user: Invalid volume: volume 970a9591-f727-42c1-abca-2a937aab458b is already attached to instances: 3c0812e6-ee5e-470c-a73a-5eaaca23354c {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:40.680373 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e0d8d875-d2e6-4214-87fb-36bcc014ae16 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.188089 Oct 15 21:17:40.680988 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3606/4965] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:17:40 2022] POST /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4/os-volume_attachments => generated 174 bytes in 196 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:17:40.692974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:40.694095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:40.703807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:40.707017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.707621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.708518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.709637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.710904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.711348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.738844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:40.740043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:40.748563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.749162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.749724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.750712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.751370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.751995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.759350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:17:40.950804 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f942c1a4-0828-45c5-884a-d65107554cf5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 204 len: 0 microversion: 2.1 time: 0.262975 Oct 15 21:17:40.957837 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1360/4966] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:17:40 2022] DELETE /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:17:40.969677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lazy-loading 'services' on Instance uuid b697ac2b-8135-4136-9629-b01a9afbb399 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:17:40.975382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:40.976639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:40.990085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.990847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.992734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:40.993913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:40.994622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:40.995259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:41.147404 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-acc9dc6f-138e-489a-95b9-ff037d51517f tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399/os-volume_attachments/c2bfc4bf-132f-412e-b190-07d6f1ae77af" status: 202 len: 0 microversion: 2.1 time: 0.430720 Oct 15 21:17:41.154265 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3607/4967] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:17:40 2022] DELETE /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399/os-volume_attachments/c2bfc4bf-132f-412e-b190-07d6f1ae77af => generated 0 bytes in 436 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:17:41.484233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:41.501452 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f0f484e8-ef9a-48cb-a3cb-323d7fc077b2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1681 microversion: 2.1 time: 0.540147 Oct 15 21:17:41.518527 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1361/4968] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:40 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1681 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:41.529804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:41.531167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:41.546626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:41.548269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:41.550049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:41.551009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:41.553994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:41.554974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:42.005595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:42.018099 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1d9fb1b8-9b91-41b0-bf71-2d956c697b06 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.492865 Oct 15 21:17:42.018709 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3608/4969] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:41 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:42.534544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:42.536334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:42.551207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:42.552008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:42.553754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:42.554853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:42.555530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:42.557005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:42.849823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:43.042408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:43.043113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:43.046437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:43.047372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:43.063101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:43.063613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:43.064061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:43.070732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:43.071241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:43.071861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:43.349790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:43.352548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:43.371800 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f9d205b2-25ff-49bc-917d-c0e4345f67a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1681 microversion: 2.1 time: 0.839838 Oct 15 21:17:43.372432 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1362/4970] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:42 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1681 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:43.500500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:43.513517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:43.528505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:43.529733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:43.530419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:43.531499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:43.532208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:43.533154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:43.590050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:17:43.743986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31a0f445-c9ab-46c6-be29-e0fd25b5ab31 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 204 len: 0 microversion: 2.1 time: 0.258775 Oct 15 21:17:43.745137 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1363/4971] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:17:43 2022] DELETE /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 21:17:43.755394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:43.756411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:43.770801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:43.772854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:43.774596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:43.776918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:43.777764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:43.778524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:43.837269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:43.870592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fc067bf-0b32-4c99-9527-f8bb2053ab4c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.831912 Oct 15 21:17:43.885070 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3609/4972] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:43 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:44.322557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:44.335033 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69b7f070-b03e-4f31-9389-96f181787c74 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1468 microversion: 2.1 time: 0.582169 Oct 15 21:17:44.337786 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1364/4973] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:43 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1468 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:44.398350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:44.400080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:44.418478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:44.421866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:44.424314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:44.425569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:44.429890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:44.430569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:44.896917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:44.897695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:44.914409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:44.918149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:44.918680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:44.920384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:44.921994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:44.924783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:45.311542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:45.328607 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c329c20c-7326-4c63-bd16-1d9147c0f365 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1681 microversion: 2.1 time: 0.934568 Oct 15 21:17:45.341867 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3610/4974] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:44 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1681 bytes in 949 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:45.367243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:45.367982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:45.381955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:45.382486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:45.382966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:45.383672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:45.384450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:45.385683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:45.727639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:45.751217 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e81a6d0b-5bdd-4b37-80ea-286fd780039c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.860289 Oct 15 21:17:45.752358 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1365/4975] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:44 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 862 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:45.913465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:45.929797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffd53c7e-e4c2-4eb8-8d4e-94d851f0fd0d tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1468 microversion: 2.1 time: 0.567874 Oct 15 21:17:45.930726 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3611/4976] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:45 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1468 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:46.355227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:46.355935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:46.369524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:46.370038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:46.370493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:46.371231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:46.371694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:46.372132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:46.401941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3df4d577-df4c-4c08-9d27-44c066fd88d3 req-860a9e3a-88be-4435-9054-451a514a32a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe2893c-e662-47f4-bfeb-d2abf9e407a4", "name": "network-vif-plugged", "status": "completed", "tag": "e0086a84-25a4-435d-aa79-fd167f4f989a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:46.421270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3df4d577-df4c-4c08-9d27-44c066fd88d3 req-860a9e3a-88be-4435-9054-451a514a32a4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:46.421768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3df4d577-df4c-4c08-9d27-44c066fd88d3 req-860a9e3a-88be-4435-9054-451a514a32a4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:46.422221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3df4d577-df4c-4c08-9d27-44c066fd88d3 req-860a9e3a-88be-4435-9054-451a514a32a4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:46.461134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3df4d577-df4c-4c08-9d27-44c066fd88d3 req-860a9e3a-88be-4435-9054-451a514a32a4 service nova] Creating event network-vif-plugged:e0086a84-25a4-435d-aa79-fd167f4f989a for instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 on nch1-master-1 Oct 15 21:17:46.467621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3df4d577-df4c-4c08-9d27-44c066fd88d3 req-860a9e3a-88be-4435-9054-451a514a32a4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069847 Oct 15 21:17:46.468937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3612/4977] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:46.780746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:46.781821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:46.795690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:46.796511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:46.797183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:46.798096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:46.799869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:46.800637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:47.035523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:47.052449 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c61913bf-32a0-4705-81f0-8ccbaaf55dfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1613 microversion: 2.1 time: 0.701904 Oct 15 21:17:47.053078 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1366/4978] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:46 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1613 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:47.059523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:47.060784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:47.073224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:47.073990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:47.074632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:47.076056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:47.077419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:47.078359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:47.592271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:47.600371 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ffff3ce1-e305-4661-8b06-76fa60ada97b tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1325 microversion: 2.1 time: 0.825526 Oct 15 21:17:47.610541 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3613/4979] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:46 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1325 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:47.681683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:47.691554 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2b6f1a9c-cf9a-413e-b26e-429c7098e837 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1468 microversion: 2.1 time: 0.637175 Oct 15 21:17:47.692504 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1367/4980] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:47 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1468 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:48.083967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:48.084766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:48.094008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.094974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.096413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.103663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.105610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.106559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.162408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-2e36e874-2ab7-4f22-a6a8-8fa336472a7f req-dc7533c6-6f68-4bdc-a026-783d650af341 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "265795dc-9437-48ae-b7bd-626db1aaa6a4", "name": "network-vif-deleted", "tag": "3fbbbcfe-1609-4659-baf8-bb604b2d15f8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:48.229732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e36e874-2ab7-4f22-a6a8-8fa336472a7f req-dc7533c6-6f68-4bdc-a026-783d650af341 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.234626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e36e874-2ab7-4f22-a6a8-8fa336472a7f req-dc7533c6-6f68-4bdc-a026-783d650af341 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.235311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e36e874-2ab7-4f22-a6a8-8fa336472a7f req-dc7533c6-6f68-4bdc-a026-783d650af341 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.277172 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-2e36e874-2ab7-4f22-a6a8-8fa336472a7f req-dc7533c6-6f68-4bdc-a026-783d650af341 service nova] Creating event network-vif-deleted:3fbbbcfe-1609-4659-baf8-bb604b2d15f8 for instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 on nch2-master-1 Oct 15 21:17:48.289596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-2e36e874-2ab7-4f22-a6a8-8fa336472a7f req-dc7533c6-6f68-4bdc-a026-783d650af341 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.131427 Oct 15 21:17:48.290819 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1368/4981] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:17:48.375099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:48.384290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc93b971-d44a-4f75-8de1-023d2767c5e5 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 200 len: 1613 microversion: 2.1 time: 0.305884 Oct 15 21:17:48.395459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3614/4982] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:48 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 1613 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:48.522690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e654bcf7-3e60-45a4-8828-33d1816ed305 req-daaf825b-9e07-42bc-af59-95bb08f4122c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b697ac2b-8135-4136-9629-b01a9afbb399", "name": "network-vif-unplugged", "status": "completed", "tag": "0efd2717-e25d-490c-b4f1-61a62b3ee75b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:48.569520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e654bcf7-3e60-45a4-8828-33d1816ed305 req-daaf825b-9e07-42bc-af59-95bb08f4122c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.574503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e654bcf7-3e60-45a4-8828-33d1816ed305 req-daaf825b-9e07-42bc-af59-95bb08f4122c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.577791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e654bcf7-3e60-45a4-8828-33d1816ed305 req-daaf825b-9e07-42bc-af59-95bb08f4122c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.629723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:48.630263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:48.650056 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e654bcf7-3e60-45a4-8828-33d1816ed305 req-daaf825b-9e07-42bc-af59-95bb08f4122c service nova] Creating event network-vif-unplugged:0efd2717-e25d-490c-b4f1-61a62b3ee75b for instance b697ac2b-8135-4136-9629-b01a9afbb399 on nch2-master-1 Oct 15 21:17:48.654089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.654674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.655108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.655849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.656308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.659100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.668803 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e654bcf7-3e60-45a4-8828-33d1816ed305 req-daaf825b-9e07-42bc-af59-95bb08f4122c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.153036 Oct 15 21:17:48.669691 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1369/4983] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:48 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:48.717653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:48.719111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:48.746409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.749053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.749473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.755698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.759454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.760093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.887855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:48.889759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:48.890373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:48.891598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lazy-loading 'fault' on Instance uuid b697ac2b-8135-4136-9629-b01a9afbb399 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:17:49.037561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a41804510e0746c781fa0f73bce0ecf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6b0324b37fb14723b8c4071057f61fb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:49.063228 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-86b84107-3443-42be-af53-ff5782016193 tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 200 len: 1303 microversion: 2.1 time: 0.350924 Oct 15 21:17:49.063822 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1370/4984] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:48 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 1303 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:17:49.314543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:49.326785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-346950a9-c4fc-42ce-9555-dc699167183a tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1522 microversion: 2.1 time: 0.702065 Oct 15 21:17:49.329354 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3615/4985] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:48 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1522 bytes in 706 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:49.356794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:49.358603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:49.381703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.384097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.387773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.389796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.397781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.398677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.422124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:49.423211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:49.430122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.431024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.431788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.432791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.434234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.435055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.449915 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] HTTP exception thrown: Instance is not a member of specified network Oct 15 21:17:49.453973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 HTTP/1.1 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '265795dc-9437-48ae-b7bd-626db1aaa6a4'} Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:49.455996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.459970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Request method failure captured: Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4/ips/invalid HTTP/1.1 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyNqnSymn72690h9nqqsl3UNZ0VGkvy5ePR0_na_i4P5ssaJB3nbQh38WcYbWW4EgDDsjKApL9CYeaeUq27YvGNvUMaqyuhJkRarg372JGuyZFCK4AgZcRfY3msMnSITfKLnp0Sd21au-jzrh6jUssJ8owpy3Syj4mplLFTQ-3TqGBk Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 66100212a95b45afb4d16f48f0fca0e3 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 66100212a95b45afb4d16f48f0fca0e3 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerAddressesNegativeTestJSON-1139617441-project Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 58294fd300b840db9d21d4c8de89a144 Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1139617441-project Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance is not a member of specified network Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'server_id': 'dfe2893c-e662-47f4-bfeb-d2abf9e407a4', 'id': 'invalid'} Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/ips.py", line 52, in show Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance is not a member of specified network Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance is not a member of specified network Oct 15 21:17:49.461082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.471382 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-814c2766-2ebc-40e9-9b49-decc02c9abdf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 404 len: 111 microversion: 2.1 time: 0.040757 Oct 15 21:17:49.471529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Returning 404 to user: Instance is not a member of specified network {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:49.471529 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c582db23-996e-4efa-8727-cdd606e6c28e tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4/ips/invalid" status: 404 len: 91 microversion: 2.1 time: 0.109929 Oct 15 21:17:49.471529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1371/4986] 10.222.0.16 () {58 vars in 1272 bytes} [Sat Oct 15 21:17:49 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4/ips/invalid => generated 91 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:17:49.471928 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3616/4987] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:49 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:17:49.472062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-96a99d2b-083a-4dde-a0e8-7d57620b5a62 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '.version_select at 0x7fe74edf2700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:49.472325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a8ab5ee-d014-4946-be2b-9feae9d0cc54 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:49.473123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a8ab5ee-d014-4946-be2b-9feae9d0cc54 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Failed to fetch instance by id 999 {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2981}} Oct 15 21:17:49.473929 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-4a8ab5ee-d014-4946-be2b-9feae9d0cc54 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] HTTP exception thrown: Instance 999 could not be found. Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a8ab5ee-d014-4946-be2b-9feae9d0cc54 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Request method failure captured: Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/999/ips HTTP/1.1 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyNqnSymn72690h9nqqsl3UNZ0VGkvy5ePR0_na_i4P5ssaJB3nbQh38WcYbWW4EgDDsjKApL9CYeaeUq27YvGNvUMaqyuhJkRarg372JGuyZFCK4AgZcRfY3msMnSITfKLnp0Sd21au-jzrh6jUssJ8owpy3Syj4mplLFTQ-3TqGBk Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 66100212a95b45afb4d16f48f0fca0e3 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 66100212a95b45afb4d16f48f0fca0e3 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerAddressesNegativeTestJSON-1139617441-project Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 58294fd300b840db9d21d4c8de89a144 Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1139617441-project Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 999 could not be found. Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '999'} Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 999 could not be found. Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/ips.py", line 37, in index Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self._compute_api, context, server_id) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 999 could not be found. Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 999 could not be found. Oct 15 21:17:49.476932 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:49.482518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a8ab5ee-d014-4946-be2b-9feae9d0cc54 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Returning 404 to user: Instance 999 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:49.484663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a8ab5ee-d014-4946-be2b-9feae9d0cc54 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/999/ips" status: 404 len: 78 microversion: 2.1 time: 0.018052 Oct 15 21:17:49.486072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3617/4988] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Oct 15 21:17:49 2022] GET /compute/v2.1/servers/999/ips => generated 78 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:17:49.503675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:49.504675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:49.506682 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-96a99d2b-083a-4dde-a0e8-7d57620b5a62 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1990827327" status: 202 len: 0 microversion: 2.1 time: 0.040978 Oct 15 21:17:49.510741 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1372/4989] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:17:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1990827327 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:17:49.529897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.529897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.530191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.530925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.531510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.531994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.573439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:17:49.686008 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb84bf13-71c4-4cd4-8008-f16c4c7a5bf6 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "DELETE /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 204 len: 0 microversion: 2.1 time: 0.194712 Oct 15 21:17:49.686701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3618/4990] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:17:49 2022] DELETE /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:17:49.696612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:49.697762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:49.710500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.713367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.714035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:49.714996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:49.725269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:49.726023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:50.052565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:50.071266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-025bb8e6-c568-4ca1-8e79-18318b9e3083 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1528 microversion: 2.1 time: 0.377193 Oct 15 21:17:50.072214 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1373/4991] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:49 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1528 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:50.088609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:50.089725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] [instance: b697ac2b-8135-4136-9629-b01a9afbb399] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:50.108535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:50.109728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:50.110404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:50.111892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:50.112576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:50.113247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:50.138904 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] HTTP exception thrown: Instance b697ac2b-8135-4136-9629-b01a9afbb399 could not be found. Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Request method failure captured: Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 HTTP/1.1 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyNSzkgj5joQ5da_GUgWzQZQzsnAsHYu1DbuHMxCChsOgYXi1uVAHB34JziUKZx8l_abbJqVGjLj6nMzITvLfhAI5qq4YGbjfnBwjIyKLvvjGBZVQdLZTAFjofVmFMrFhscSquOxE4EfBTzqxU267Qj2qd5E27ged73eOpCFgl54lEM Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6b0324b37fb14723b8c4071057f61fb9 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesBackupsTest-2098473723 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b0324b37fb14723b8c4071057f61fb9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6b0324b37fb14723b8c4071057f61fb9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6b0324b37fb14723b8c4071057f61fb9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6b0324b37fb14723b8c4071057f61fb9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesBackupsTest-2098473723 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6b0324b37fb14723b8c4071057f61fb9 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesBackupsTest-2098473723 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesBackupsTest-2098473723-project Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a41804510e0746c781fa0f73bce0ecf2 Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesBackupsTest-2098473723-project Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b697ac2b-8135-4136-9629-b01a9afbb399 could not be found. Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b697ac2b-8135-4136-9629-b01a9afbb399'} Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b697ac2b-8135-4136-9629-b01a9afbb399 could not be found. Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b697ac2b-8135-4136-9629-b01a9afbb399 could not be found. Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:50.141012 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b697ac2b-8135-4136-9629-b01a9afbb399 could not be found. Oct 15 21:17:50.144823 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:17:50.158624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] Returning 404 to user: Instance b697ac2b-8135-4136-9629-b01a9afbb399 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:50.159510 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53dae6cf-3857-49b6-a41f-ddb9f7f854ae tempest-VolumesBackupsTest-2098473723 tempest-VolumesBackupsTest-2098473723-project] 10.222.0.16 "GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399" status: 404 len: 111 microversion: 2.1 time: 0.075243 Oct 15 21:17:50.163187 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3619/4992] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:50 2022] GET /compute/v2.1/servers/b697ac2b-8135-4136-9629-b01a9afbb399 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:17:50.355136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-99f130d1-3ea4-4dac-90e2-3e59793d0b57 req-39bfafbf-d1b6-4101-8229-bbc785108d3d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b697ac2b-8135-4136-9629-b01a9afbb399", "name": "network-vif-deleted", "tag": "0efd2717-e25d-490c-b4f1-61a62b3ee75b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:50.369647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-99f130d1-3ea4-4dac-90e2-3e59793d0b57 req-39bfafbf-d1b6-4101-8229-bbc785108d3d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:50.370114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-99f130d1-3ea4-4dac-90e2-3e59793d0b57 req-39bfafbf-d1b6-4101-8229-bbc785108d3d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:50.370916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-99f130d1-3ea4-4dac-90e2-3e59793d0b57 req-39bfafbf-d1b6-4101-8229-bbc785108d3d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:50.386310 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-99f130d1-3ea4-4dac-90e2-3e59793d0b57 req-39bfafbf-d1b6-4101-8229-bbc785108d3d service nova] Creating event network-vif-deleted:0efd2717-e25d-490c-b4f1-61a62b3ee75b for instance b697ac2b-8135-4136-9629-b01a9afbb399 on nch2-master-1 Oct 15 21:17:50.395474 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-99f130d1-3ea4-4dac-90e2-3e59793d0b57 req-39bfafbf-d1b6-4101-8229-bbc785108d3d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050207 Oct 15 21:17:50.403983 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1374/4993] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:50.882538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:50.885408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:50.895337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:50.896088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:50.896545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:50.897356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:50.898108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:50.898848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:50.998417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lazy-loading 'services' on Instance uuid 3c0812e6-ee5e-470c-a73a-5eaaca23354c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:17:51.102947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:51.103658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:51.104139 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-84544a7e-71ee-41ab-9e5a-b9458e4cc7cd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments/970a9591-f727-42c1-abca-2a937aab458b" status: 202 len: 0 microversion: 2.1 time: 0.231411 Oct 15 21:17:51.105007 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3620/4994] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:17:50 2022] DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments/970a9591-f727-42c1-abca-2a937aab458b => generated 0 bytes in 233 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:17:51.117136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:51.117648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:51.118092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:51.118985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:51.119588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:51.120027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:51.122441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:51.127941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:51.136605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:51.138047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:51.138841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:51.139770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:51.140780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:51.141453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:51.190334 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc2cc86d-f1bc-4fc1-9033-88a676769fbf tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079466 Oct 15 21:17:51.191654 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3621/4995] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:17:51 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:17:51.924680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:51.942875 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7fad6515-e31e-40a4-8361-5ee0824236f9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1528 microversion: 2.1 time: 0.849294 Oct 15 21:17:51.944409 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1375/4996] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:51 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1528 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:52.204347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:52.211028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:52.231376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.232635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.234690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.239224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.240892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.243413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.305515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:52.331231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d7d2306-a559-4640-ac4b-83b2f252b97b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.131496 Oct 15 21:17:52.332315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3622/4997] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:17:52 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c/os-volume_attachments => generated 25 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:17:52.456022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:52.456549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:52.478920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.479443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.479900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.480637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.481169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.485218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.534822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:17:52.662322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b8c67e7f-749f-4e1d-a667-3e41f050087d req-7b4a77c1-9f6f-4858-89b3-f3144319b0a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe2893c-e662-47f4-bfeb-d2abf9e407a4", "name": "network-vif-deleted", "tag": "e0086a84-25a4-435d-aa79-fd167f4f989a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:52.691713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b8c67e7f-749f-4e1d-a667-3e41f050087d req-7b4a77c1-9f6f-4858-89b3-f3144319b0a1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.692843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b8c67e7f-749f-4e1d-a667-3e41f050087d req-7b4a77c1-9f6f-4858-89b3-f3144319b0a1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.694963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b8c67e7f-749f-4e1d-a667-3e41f050087d req-7b4a77c1-9f6f-4858-89b3-f3144319b0a1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.737401 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b8c67e7f-749f-4e1d-a667-3e41f050087d req-7b4a77c1-9f6f-4858-89b3-f3144319b0a1 service nova] Creating event network-vif-deleted:e0086a84-25a4-435d-aa79-fd167f4f989a for instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 on nch1-master-1 Oct 15 21:17:52.749465 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b8c67e7f-749f-4e1d-a667-3e41f050087d req-7b4a77c1-9f6f-4858-89b3-f3144319b0a1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102669 Oct 15 21:17:52.752738 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3623/4998] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:52.780398 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5c8f1e88-a18d-44c0-b62f-786130453974 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 204 len: 0 microversion: 2.1 time: 0.327883 Oct 15 21:17:52.783600 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1376/4999] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:17:52 2022] DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:17:52.794032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:52.795107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:52.814160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.817970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.818669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.819548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.820144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.820828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.970777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:52.971825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:52.988892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.989711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.992220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:52.993432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:52.995656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:52.996363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:53.084125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:53.084959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:53.085860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:53.086717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lazy-loading 'fault' on Instance uuid dfe2893c-e662-47f4-bfeb-d2abf9e407a4 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:17:53.188215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:53.227777 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df904099-91d4-4c8b-8f68-c3c680f7377c tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1315 microversion: 2.1 time: 0.262660 Oct 15 21:17:53.228815 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1377/5000] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:52 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1315 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:17:53.297528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:53.309128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6af3805a-466e-4cce-9f99-2ecc1b66732a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1682 microversion: 2.1 time: 0.520801 Oct 15 21:17:53.310027 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3624/5001] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:52 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1682 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:17:54.253965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:54.254974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:54.271414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:54.272082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:54.272547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:54.273317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:54.273783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:54.283920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:54.323435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:54.324918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:54.325811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:54.326802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lazy-loading 'fault' on Instance uuid dfe2893c-e662-47f4-bfeb-d2abf9e407a4 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:17:54.334748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:54.335973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:17:54.336424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:54.355544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:54.356587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:54.357318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:54.359043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:54.360422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:54.364214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:54.635290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58294fd300b840db9d21d4c8de89a144', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66100212a95b45afb4d16f48f0fca0e3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:54.647840 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e026006-ec00-4f40-aeaf-9a6216355ff9 tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 200 len: 1315 microversion: 2.1 time: 0.398766 Oct 15 21:17:54.648429 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1378/5002] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:54 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 1315 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:54.961566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:54.983239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a37ecafc-1c6a-4cf2-8fb6-3bf235f1bb6a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1682 microversion: 2.1 time: 0.653080 Oct 15 21:17:54.984275 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3625/5003] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:54 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1682 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:55.671680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:55.672305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] [instance: dfe2893c-e662-47f4-bfeb-d2abf9e407a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:55.681959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:55.682708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:55.683379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:55.684830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:55.686046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:55.686707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:55.702970 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] HTTP exception thrown: Instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 could not be found. Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Request method failure captured: Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 HTTP/1.1 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyNqnSymn72690h9nqqsl3UNZ0VGkvy5ePR0_na_i4P5ssaJB3nbQh38WcYbWW4EgDDsjKApL9CYeaeUq27YvGNvUMaqyuhJkRarg372JGuyZFCK4AgZcRfY3msMnSITfKLnp0Sd21au-jzrh6jUssJ8owpy3Syj4mplLFTQ-3TqGBk Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 66100212a95b45afb4d16f48f0fca0e3 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/66100212a95b45afb4d16f48f0fca0e3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 66100212a95b45afb4d16f48f0fca0e3 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1139617441 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerAddressesNegativeTestJSON-1139617441-project Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 58294fd300b840db9d21d4c8de89a144 Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1139617441-project Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 could not be found. Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'dfe2893c-e662-47f4-bfeb-d2abf9e407a4'} Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 could not be found. Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 could not be found. Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:55.706184 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 could not be found. Oct 15 21:17:55.711794 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:17:55.713675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] Returning 404 to user: Instance dfe2893c-e662-47f4-bfeb-d2abf9e407a4 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:17:55.714782 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d7c3c995-506a-4cdc-8dc7-e3dc1805b7fa tempest-ServerAddressesNegativeTestJSON-1139617441 tempest-ServerAddressesNegativeTestJSON-1139617441-project] 10.222.0.16 "GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4" status: 404 len: 111 microversion: 2.1 time: 0.047023 Oct 15 21:17:55.716944 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1379/5004] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:55 2022] GET /compute/v2.1/servers/dfe2893c-e662-47f4-bfeb-d2abf9e407a4 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:17:56.010555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:56.012071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:56.029521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:56.030303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:56.032168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:56.036155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:56.037584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:56.039393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:56.562066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:56.570829 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-920c57a8-dc30-48b4-a027-e0d65788dcf6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1682 microversion: 2.1 time: 0.564138 Oct 15 21:17:56.571416 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3626/5005] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:56 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1682 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:57.616054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:57.618072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:57.654651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:57.655494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:57.656114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:57.657093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:57.658346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:57.659299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:57.943266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:57.957116 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-26e0453f-e1a6-4283-a3c7-653a0bd2e73e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1614 microversion: 2.1 time: 0.361823 Oct 15 21:17:57.958427 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1380/5006] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:57 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1614 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:17:58.425901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4ad26597-cc5a-404b-82f1-f20fc48630cb req-898d5d37-3e0a-4d1d-b886-644fab20673a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c0812e6-ee5e-470c-a73a-5eaaca23354c", "name": "network-vif-deleted", "tag": "9b5226b2-acbc-4977-8451-6c4b8fb6bf30"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:17:58.448098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4ad26597-cc5a-404b-82f1-f20fc48630cb req-898d5d37-3e0a-4d1d-b886-644fab20673a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:58.448952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4ad26597-cc5a-404b-82f1-f20fc48630cb req-898d5d37-3e0a-4d1d-b886-644fab20673a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:58.449703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4ad26597-cc5a-404b-82f1-f20fc48630cb req-898d5d37-3e0a-4d1d-b886-644fab20673a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:58.481468 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4ad26597-cc5a-404b-82f1-f20fc48630cb req-898d5d37-3e0a-4d1d-b886-644fab20673a service nova] Creating event network-vif-deleted:9b5226b2-acbc-4977-8451-6c4b8fb6bf30 for instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c on nch1-master-1 Oct 15 21:17:58.491073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4ad26597-cc5a-404b-82f1-f20fc48630cb req-898d5d37-3e0a-4d1d-b886-644fab20673a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069468 Oct 15 21:17:58.503024 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3627/5007] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:17:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:17:58.982570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:17:58.984220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:17:59.014140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:59.015790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:59.016453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:59.018630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:59.019799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:59.022782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:59.172167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:17:59.180390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:17:59.180922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:17:59.181451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lazy-loading 'fault' on Instance uuid 3c0812e6-ee5e-470c-a73a-5eaaca23354c {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:17:59.430527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:17:59.440230 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4db92a04-78b0-4170-8b16-787e2c743cfb tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 200 len: 1331 microversion: 2.1 time: 0.463205 Oct 15 21:17:59.445683 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1381/5008] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:17:58 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 1331 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:00.477897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:00.478621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:00.493672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:00.494464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:00.495081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:00.495978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:00.496597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:00.497961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:00.516402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c HTTP/1.1 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3c0812e6-ee5e-470c-a73a-5eaaca23354c'} Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:18:00.519152 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:18:00.523025 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:18:00.524952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:18:00.526069 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a791a81-55bf-4bb5-8f8f-8938c1d234b6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 404 len: 111 microversion: 2.1 time: 0.062221 Oct 15 21:18:00.527886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3628/5009] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:00 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:18:00.536716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a9e4b6e-a87d-4354-91d6-21034e9b9697 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '.version_select at 0x7fe74ecff550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:00.580917 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a9e4b6e-a87d-4354-91d6-21034e9b9697 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-689819752" status: 202 len: 0 microversion: 2.1 time: 0.048423 Oct 15 21:18:00.582431 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1382/5010] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 21:18:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-689819752 => generated 0 bytes in 51 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:18:03.791702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75e46dfd-fea2-4d53-b061-e25482c3ca9d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: .version_select at 0x7fe7584100d0>, body: {"keypair": {"name": "tempest-keypair-1149358351", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCf8STTUg7iUQlPReB+bQmh+2kAqwcZGjb3//sHDDte6DkBE7INzyRdNj6uqD6GK+CMYBbre/TRIKEorUsW67noC7xB0vsPeFTLvPdlAgJQ/VjwbwU9T0xpDS3By3AagA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:18:03.823015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-75e46dfd-fea2-4d53-b061-e25482c3ca9d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:18:03.831726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-75e46dfd-fea2-4d53-b061-e25482c3ca9d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:18:03.894647 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75e46dfd-fea2-4d53-b061-e25482c3ca9d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.109808 Oct 15 21:18:03.895917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3629/5011] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:18:03 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:09.387980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-676237cc-8651-4f12-b69a-263d3a0fa888 tempest-ComputeTenantNetworksTest-330824532 tempest-ComputeTenantNetworksTest-330824532-project] Calling method '.version_select at 0x7fe74eebe0d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:09.435266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=19 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:09.900538 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-676237cc-8651-4f12-b69a-263d3a0fa888 tempest-ComputeTenantNetworksTest-330824532 tempest-ComputeTenantNetworksTest-330824532-project] 10.222.0.16 "GET /compute/v2.1/os-tenant-networks" status: 200 len: 308 microversion: 2.1 time: 0.715391 Oct 15 21:18:09.907255 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1383/5012] 10.222.0.16 () {58 vars in 1158 bytes} [Sat Oct 15 21:18:09 2022] GET /compute/v2.1/os-tenant-networks => generated 308 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:09.914898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8593af1-c087-43fa-82ad-6a443e6b9d75 tempest-ComputeTenantNetworksTest-330824532 tempest-ComputeTenantNetworksTest-330824532-project] Calling method '.version_select at 0x7fe758410ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:10.337887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8593af1-c087-43fa-82ad-6a443e6b9d75 tempest-ComputeTenantNetworksTest-330824532 tempest-ComputeTenantNetworksTest-330824532-project] 10.222.0.16 "GET /compute/v2.1/os-tenant-networks/4d212d7e-f988-4521-9286-ba45187a4b03" status: 200 len: 139 microversion: 2.1 time: 0.428976 Oct 15 21:18:10.339373 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3630/5013] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:18:09 2022] GET /compute/v2.1/os-tenant-networks/4d212d7e-f988-4521-9286-ba45187a4b03 => generated 139 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:10.575396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-834938816", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d51b163a-0bf6-4a5b-be5f-78f88001c0b8"}], "security_groups": [{"name": "tempest-securitygroup--523442725"}], "key_name": "tempest-keypair-1149358351", "user_data": "}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:18:10.620832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=3,nova_api:DELETE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:10.949910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] validate_networks() for [('d51b163a-0bf6-4a5b-be5f-78f88001c0b8', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:18:12.370300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:18:12.370912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:18:12.371395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:18:12.371924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:18:12.372486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:18:12.392396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:12.393131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:12.393587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:12.394353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:12.394818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:12.395253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:12.463392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:18:12.478874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:18:12.509847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:18:12.512229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:18:12.644947 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ae9671c-63b4-4728-a3fe-c8b97d9eb5c1 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.074199 Oct 15 21:18:12.647468 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1384/5014] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:18:10 2022] POST /compute/v2.1/servers => generated 401 bytes in 2077 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:18:12.657986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6dcffd9a-75cb-4df7-a237-d481cb1b1a92 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:12.660161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6dcffd9a-75cb-4df7-a237-d481cb1b1a92 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:12.663421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=29 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:12.667469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:12.849702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6dcffd9a-75cb-4df7-a237-d481cb1b1a92 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:12.859861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6dcffd9a-75cb-4df7-a237-d481cb1b1a92 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1241 microversion: 2.1 time: 0.206735 Oct 15 21:18:12.862632 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3631/5015] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:12 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1241 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:13.891352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:13.892000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:13.906181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:13.910348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:13.911109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:13.912242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:13.912964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:13.915071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:14.062217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:14.107210 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d4a75595-e6b6-411f-abd6-284fe125ff64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1293 microversion: 2.1 time: 0.187803 Oct 15 21:18:14.108184 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1385/5016] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:13 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1293 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:15.135360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:15.137922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:15.156920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:15.157538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:15.173462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:15.174255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:15.174741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:15.175181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:15.461475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:15.483222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-81757def-6f2e-4186-af50-f0a069a08245 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1299 microversion: 2.1 time: 0.354387 Oct 15 21:18:15.486930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3632/5017] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:15 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1299 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:16.512249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:16.512820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:16.537875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:16.540030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:16.540623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:16.542996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:16.547787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:16.548691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:17.857122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:17.870622 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7de4d515-eb05-4494-bcf2-7b0b75b351ae tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 1.364656 Oct 15 21:18:17.881262 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1386/5018] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:16 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 1376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:18.902185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:18.914620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:18.945094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:18.946969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:18.948193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:18.950291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:18.955434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:18.956379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:19.667352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:19.694086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3236b92c-78db-4dc0-8d1f-6e6b3a48ba37 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.797355 Oct 15 21:18:19.695148 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3633/5019] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:18 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 803 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:19.947819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e14389a0-407b-40d9-b0ae-dfebfe3f989e req-80c04df7-3437-4fd6-8e47-5fe3881866b2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da1775af-9897-4ef4-9e5a-5c7373ce3761", "tag": "5d17f129-635c-49c6-9c5e-63852eb0934f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:18:19.972358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e14389a0-407b-40d9-b0ae-dfebfe3f989e req-80c04df7-3437-4fd6-8e47-5fe3881866b2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:19.973115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e14389a0-407b-40d9-b0ae-dfebfe3f989e req-80c04df7-3437-4fd6-8e47-5fe3881866b2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:19.973761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e14389a0-407b-40d9-b0ae-dfebfe3f989e req-80c04df7-3437-4fd6-8e47-5fe3881866b2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:19.994680 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e14389a0-407b-40d9-b0ae-dfebfe3f989e req-80c04df7-3437-4fd6-8e47-5fe3881866b2 service nova] Creating event network-changed:5d17f129-635c-49c6-9c5e-63852eb0934f for instance da1775af-9897-4ef4-9e5a-5c7373ce3761 on nch2-master-1 Oct 15 21:18:20.007195 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e14389a0-407b-40d9-b0ae-dfebfe3f989e req-80c04df7-3437-4fd6-8e47-5fe3881866b2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063573 Oct 15 21:18:20.011736 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1387/5020] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:18:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:20.719815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:20.720517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:20.727303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:20.727790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:20.728233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:20.728943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:20.735987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:20.736460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:21.123666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:21.139683 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5db725a4-6813-42a6-8a8f-0f69c893daf9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.423519 Oct 15 21:18:21.142327 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3634/5021] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:20 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:22.164027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:22.164848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:22.173605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:22.174410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:22.175166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:22.176130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:22.176824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:22.177351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:22.423378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:22.490244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:22.502066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:22.504851 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e54daf1a-a9e9-4f97-ae49-43e2fce76dd6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.345157 Oct 15 21:18:22.505481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1388/5022] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:22 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:23.532957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:23.533586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:23.536412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:23.550864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:23.551370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:23.552440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:23.553382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:23.555643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:23.556451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:24.181061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:24.191686 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61c2c1f7-d8b4-44b5-bdf7-fcb20c076201 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.664134 Oct 15 21:18:24.193599 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3635/5023] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:23 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:25.220987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:25.222135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:25.235217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:25.236083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:25.236681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:25.237585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:25.238247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:25.238861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:25.634081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:25.646914 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b30283ea-d7f4-4fb9-961c-9442d4da8988 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.430855 Oct 15 21:18:25.648632 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1389/5024] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:25 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:26.673931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:26.674694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:26.696301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:26.700268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:26.706693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:26.710948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:26.711461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:26.716386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:27.043514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:27.063505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e040d0c6-ddc0-4adb-95c7-2ba8976b615f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.392091 Oct 15 21:18:27.067876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3636/5025] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:26 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:27.872150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b52665b5-ddcf-411e-8db8-fe4aec6ebf0e req-877576e5-e7fa-4ebd-85b2-babc23d87360 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1775af-9897-4ef4-9e5a-5c7373ce3761", "name": "network-vif-plugged", "status": "completed", "tag": "5d17f129-635c-49c6-9c5e-63852eb0934f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:18:27.890339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b52665b5-ddcf-411e-8db8-fe4aec6ebf0e req-877576e5-e7fa-4ebd-85b2-babc23d87360 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:27.890739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b52665b5-ddcf-411e-8db8-fe4aec6ebf0e req-877576e5-e7fa-4ebd-85b2-babc23d87360 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:27.891169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b52665b5-ddcf-411e-8db8-fe4aec6ebf0e req-877576e5-e7fa-4ebd-85b2-babc23d87360 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:27.913171 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b52665b5-ddcf-411e-8db8-fe4aec6ebf0e req-877576e5-e7fa-4ebd-85b2-babc23d87360 service nova] Creating event network-vif-plugged:5d17f129-635c-49c6-9c5e-63852eb0934f for instance da1775af-9897-4ef4-9e5a-5c7373ce3761 on nch2-master-1 Oct 15 21:18:27.919987 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b52665b5-ddcf-411e-8db8-fe4aec6ebf0e req-877576e5-e7fa-4ebd-85b2-babc23d87360 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051051 Oct 15 21:18:27.923149 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1390/5026] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:18:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:28.094304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:28.095692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:28.116298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:28.117175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:28.117849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:28.119046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:28.119701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:28.120503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:28.410238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:28.420696 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac8cc3ee-da75-4397-9804-2da9249a2c4a tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.332617 Oct 15 21:18:28.421462 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3637/5027] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:28 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:29.454987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:29.455744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:29.463717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:29.464152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:29.464681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:29.465766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:29.466238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:29.466670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:29.806818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:29.821256 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a991f25b-d8dc-4964-90a7-0e0f8700ee5b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1366 microversion: 2.1 time: 0.377658 Oct 15 21:18:29.822118 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1391/5028] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:29 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1366 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:18:30.848040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:30.849327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:30.867700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:30.867700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:30.867700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:30.869566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:30.870071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:30.870517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:31.287322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:18:31.302417 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27d204f0-7c5b-490b-97fc-11571b84fff4 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1556 microversion: 2.1 time: 0.459016 Oct 15 21:18:31.303229 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3638/5029] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:18:30 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1556 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:18:31.325628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:18:31.326120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:18:31.341068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:31.341438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:31.346040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:31.346542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:31.348032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:31.348479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:31.525467 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c476086-ed05-415b-b38c-205bc1a6b8c7 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.203506 Oct 15 21:18:31.526606 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1392/5030] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:18:31 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-interface => generated 285 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:33.897356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-74d5d77c-9ad1-4ae8-97fd-508b58c8b2fe req-f6818384-7f38-4811-a74e-556e43b6596a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da1775af-9897-4ef4-9e5a-5c7373ce3761", "tag": "5d17f129-635c-49c6-9c5e-63852eb0934f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:18:33.931464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-74d5d77c-9ad1-4ae8-97fd-508b58c8b2fe req-f6818384-7f38-4811-a74e-556e43b6596a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:18:33.932214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-74d5d77c-9ad1-4ae8-97fd-508b58c8b2fe req-f6818384-7f38-4811-a74e-556e43b6596a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:18:33.935143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-74d5d77c-9ad1-4ae8-97fd-508b58c8b2fe req-f6818384-7f38-4811-a74e-556e43b6596a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:18:33.957263 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-74d5d77c-9ad1-4ae8-97fd-508b58c8b2fe req-f6818384-7f38-4811-a74e-556e43b6596a service nova] Creating event network-changed:5d17f129-635c-49c6-9c5e-63852eb0934f for instance da1775af-9897-4ef4-9e5a-5c7373ce3761 on nch2-master-1 Oct 15 21:18:33.965432 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-74d5d77c-9ad1-4ae8-97fd-508b58c8b2fe req-f6818384-7f38-4811-a74e-556e43b6596a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080043 Oct 15 21:18:33.967122 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3639/5031] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:18:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:18:56.253088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:56.254725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=40 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:18:56.725537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-39436768-dce2-4009-a43b-2e0d4f46d610 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Action: 'create', calling method: .version_select at 0x7fe74ed283a0>, body: {"keypair": {"name": "tempest-TestSecurityGroupsBasicOps-1907142042", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNXnNMkPsD3v0Yp/IPh5dDQ+haB9wiC2aN/XbSV+9QNpMXmP7ZAfsfo2EnDGDbUQ4uJFk6vghvCcmttuFvvsTyQX3OMskpqUuDOnkmw7VniRJKNYKvbkkOOhqmbuhOAd9A=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:18:56.760685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-39436768-dce2-4009-a43b-2e0d4f46d610 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Getting quotas for project 2c132f2961594244a86c8a610462992e. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:18:56.770499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-39436768-dce2-4009-a43b-2e0d4f46d610 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Getting quotas for user 026771ec28444739afc13c47339af7b0 and project 2c132f2961594244a86c8a610462992e. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:18:56.818799 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-39436768-dce2-4009-a43b-2e0d4f46d610 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 405 microversion: 2.1 time: 0.568537 Oct 15 21:18:56.819678 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1393/5032] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:18:56 2022] POST /compute/v2.1/os-keypairs => generated 405 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:08.914922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-server-tempest-TestSecurityGroupsBasicOps-698525279-access_point-1084468335", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "99ad80c9-9b93-4749-a7dc-dd3d8c0d6125"}], "key_name": "tempest-TestSecurityGroupsBasicOps-1907142042", "security_groups": [{"name": "tempest-secgroup_access--1870307951"}, {"name": "tempest-secgroup_general--1574080266"}]}, "os:scheduler_hints": {"different_host": []}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:09.022493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=19 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:09.026256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=44 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:09.646155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] validate_networks() for [('99ad80c9-9b93-4749-a7dc-dd3d8c0d6125', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:19:11.221023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:19:11.221834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:19:11.221834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:19:11.222254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:19:11.223326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:19:11.236623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:11.237638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:11.238236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:11.239129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:11.239814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:11.240392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:11.276402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Getting quotas for project 2c132f2961594244a86c8a610462992e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:19:11.287292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Getting quotas for user 026771ec28444739afc13c47339af7b0 and project 2c132f2961594244a86c8a610462992e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:19:11.313263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:19:11.316293 ncdv-master-1 devstack@n-api.service[138688]: WARNING nova.utils [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Hostname tempest-server-tempest-TestSecurityGroupsBasicOps-698525279-access_point-1084468335 is longer than 63, truncate it to tempest-server-tempest-TestSecurityGroupsBasicOps-698525279-acc Oct 15 21:19:11.318354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:19:11.406782 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70521030-f688-4fb6-b097-6973b8cd4429 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 458 microversion: 2.1 time: 2.499285 Oct 15 21:19:11.408113 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3640/5033] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:19:08 2022] POST /compute/v2.1/servers => generated 458 bytes in 2501 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:19:11.418443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:11.420414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:11.427338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-990b9a3e-fe49-4812-97fb-83700f2dae4d tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:11.428175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-990b9a3e-fe49-4812-97fb-83700f2dae4d tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:11.584697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-990b9a3e-fe49-4812-97fb-83700f2dae4d tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:11.593067 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-990b9a3e-fe49-4812-97fb-83700f2dae4d tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1294 microversion: 2.1 time: 0.177769 Oct 15 21:19:11.597175 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1394/5034] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:11 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1294 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:12.636626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9bcaad1e-d5b5-45e2-9323-fc293804b424 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:12.638744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9bcaad1e-d5b5-45e2-9323-fc293804b424 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:12.927721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9bcaad1e-d5b5-45e2-9323-fc293804b424 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:12.939769 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9bcaad1e-d5b5-45e2-9323-fc293804b424 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1294 microversion: 2.1 time: 0.315060 Oct 15 21:19:12.945701 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3641/5035] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:12 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1294 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:13.971783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:13.972411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:13.984645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:13.987319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:13.987472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:13.988107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:13.988577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:13.989014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:14.230217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:14.241627 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4e04dfe1-c0dd-4fe3-9e01-4524877ce2f8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1364 microversion: 2.1 time: 0.272110 Oct 15 21:19:14.242810 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1395/5036] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:13 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1364 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:15.271279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:15.272265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:15.286980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:15.287655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:15.289206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:15.291654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:15.294417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:15.295511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:15.834140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:15.842990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdab0bf8-7362-488c-b200-d9e34bfa4681 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1352 microversion: 2.1 time: 0.576526 Oct 15 21:19:15.847130 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3642/5037] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:15 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1352 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:16.871474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:16.873843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:16.884496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:16.885322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:16.886288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:16.888303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:16.889005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:16.889847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:17.147676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:17.163241 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d6b83e6-3379-4a68-8eb3-fe499f76accf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.296328 Oct 15 21:19:17.163832 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1396/5038] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:16 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:17.864620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-480f936a-404a-4f58-b9f9-140bcde53140 req-cfb1c499-acd9-4ff3-9972-2dc08d0fe611 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77690280-0359-4a18-b207-11a49e9b491c", "tag": "6f9e24c9-cae4-40a2-a4ea-7377744bada5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:17.879964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-480f936a-404a-4f58-b9f9-140bcde53140 req-cfb1c499-acd9-4ff3-9972-2dc08d0fe611 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:17.880630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-480f936a-404a-4f58-b9f9-140bcde53140 req-cfb1c499-acd9-4ff3-9972-2dc08d0fe611 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:17.881479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-480f936a-404a-4f58-b9f9-140bcde53140 req-cfb1c499-acd9-4ff3-9972-2dc08d0fe611 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:17.894997 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-480f936a-404a-4f58-b9f9-140bcde53140 req-cfb1c499-acd9-4ff3-9972-2dc08d0fe611 service nova] Creating event network-changed:6f9e24c9-cae4-40a2-a4ea-7377744bada5 for instance 77690280-0359-4a18-b207-11a49e9b491c on nch1-master-1 Oct 15 21:19:17.902143 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-480f936a-404a-4f58-b9f9-140bcde53140 req-cfb1c499-acd9-4ff3-9972-2dc08d0fe611 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.329415 Oct 15 21:19:17.903766 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3643/5039] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:19:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:18.214781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:18.215415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:18.230414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:18.231679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:18.232165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:18.235600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:18.236087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:18.236530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:18.460908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:18.469416 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-15a37aa2-b08c-47d3-b80a-88cc01327986 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.282019 Oct 15 21:19:18.482668 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1397/5040] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:18 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:19.505223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:19.506437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:19.514760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:19.515545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:19.516256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:19.517379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:19.518226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:19.519180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:20.139619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:20.158481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98a05123-b653-48ac-9b8a-67d4dcd41778 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.667320 Oct 15 21:19:20.159337 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3644/5041] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:19 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:21.205717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:21.207521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:21.222166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:21.223139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:21.223940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:21.229325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:21.230051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:21.230708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:21.526608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:21.542933 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2410424b-fa17-47ee-a1c4-be7c30b2e2a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.356759 Oct 15 21:19:21.546418 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1398/5042] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:21 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:22.569769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:22.571240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:22.585793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:22.591319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:22.592496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:22.609001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:22.610335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:22.614566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:22.615530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:22.933492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:22.986308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:23.001253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b53bc1c-baec-403c-b8fd-735388f15805 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.438432 Oct 15 21:19:23.002790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3645/5043] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:22 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:24.036884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:24.040510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:24.048266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:24.053613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:24.054649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:24.055469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:24.056671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:24.058413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:24.059904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:24.667574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:24.677230 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de1de5f7-c448-4b20-84de-e2d48f963f56 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.652025 Oct 15 21:19:24.682726 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1399/5044] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:24 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:24.792631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b8a063c5-3887-4df9-9a78-f4dd07219dd6 req-3bd47404-3f2c-4d6d-a1fd-d0c1b4e23391 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77690280-0359-4a18-b207-11a49e9b491c", "name": "network-vif-plugged", "status": "completed", "tag": "6f9e24c9-cae4-40a2-a4ea-7377744bada5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:24.812791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b8a063c5-3887-4df9-9a78-f4dd07219dd6 req-3bd47404-3f2c-4d6d-a1fd-d0c1b4e23391 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:24.815985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b8a063c5-3887-4df9-9a78-f4dd07219dd6 req-3bd47404-3f2c-4d6d-a1fd-d0c1b4e23391 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:24.816845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b8a063c5-3887-4df9-9a78-f4dd07219dd6 req-3bd47404-3f2c-4d6d-a1fd-d0c1b4e23391 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:24.833787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b8a063c5-3887-4df9-9a78-f4dd07219dd6 req-3bd47404-3f2c-4d6d-a1fd-d0c1b4e23391 service nova] Creating event network-vif-plugged:6f9e24c9-cae4-40a2-a4ea-7377744bada5 for instance 77690280-0359-4a18-b207-11a49e9b491c on nch1-master-1 Oct 15 21:19:24.843987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b8a063c5-3887-4df9-9a78-f4dd07219dd6 req-3bd47404-3f2c-4d6d-a1fd-d0c1b4e23391 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.307516 Oct 15 21:19:24.844781 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3646/5045] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:19:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:25.705913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:25.706545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:25.713909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:25.714412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:25.714855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:25.723470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:25.724134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:25.725221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:26.259837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:26.274236 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec00e9f1-6667-4776-b87f-7e9c253f820c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1472 microversion: 2.1 time: 0.571025 Oct 15 21:19:26.292260 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1400/5046] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:25 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1472 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:27.302655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:27.303464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:27.315942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:27.319540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:27.321292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:27.323346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:27.325882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:27.326538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:27.630862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:27.641701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fafc0296-0b0c-4f35-9fd7-e8bab91cbad7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1645 microversion: 2.1 time: 0.346144 Oct 15 21:19:27.661183 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3647/5047] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:27 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1645 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:27.673880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:27.674564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:27.687289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:27.687785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:27.688226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:27.689086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:27.704482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:27.704976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:28.189006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:28.198093 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-40bfce29-754a-4980-be30-3bfa9647ba34 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1645 microversion: 2.1 time: 0.531561 Oct 15 21:19:28.198687 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1401/5048] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:27 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1645 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:28.229978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:28.230869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:28.240541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:28.243385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:28.244863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:28.245855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:28.247446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:28.249713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:29.196293 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d9ff1e49-f530-420a-b65f-07484e5086c7 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1796 microversion: 2.1 time: 0.970500 Oct 15 21:19:29.198334 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3648/5049] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:28 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1796 bytes in 973 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:30.394009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Calling method '.version_select at 0x7fe74ee26160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:30.399431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:30.400391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:30.401865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:30.426926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:30.428199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:30.428890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:30.470434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:30.471154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:30.471809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:30.482926 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cbb8fb9b-1a43-4765-8d11-4319be9d7fb8 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471 tempest-PortsAdminExtendedAttrsIpV6TestJSON-1006564471-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.301014 Oct 15 21:19:30.486127 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1402/5050] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:19:30 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:33.349743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-df7c9658-290b-4d99-82be-6eb05e8dba64 req-ea8491ab-8c43-4b6b-b851-92cf69ad6f68 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77690280-0359-4a18-b207-11a49e9b491c", "tag": "6f9e24c9-cae4-40a2-a4ea-7377744bada5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:33.376747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-df7c9658-290b-4d99-82be-6eb05e8dba64 req-ea8491ab-8c43-4b6b-b851-92cf69ad6f68 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:33.378028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-df7c9658-290b-4d99-82be-6eb05e8dba64 req-ea8491ab-8c43-4b6b-b851-92cf69ad6f68 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:33.378716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-df7c9658-290b-4d99-82be-6eb05e8dba64 req-ea8491ab-8c43-4b6b-b851-92cf69ad6f68 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:33.398635 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-df7c9658-290b-4d99-82be-6eb05e8dba64 req-ea8491ab-8c43-4b6b-b851-92cf69ad6f68 service nova] Creating event network-changed:6f9e24c9-cae4-40a2-a4ea-7377744bada5 for instance 77690280-0359-4a18-b207-11a49e9b491c on nch1-master-1 Oct 15 21:19:33.406350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-df7c9658-290b-4d99-82be-6eb05e8dba64 req-ea8491ab-8c43-4b6b-b851-92cf69ad6f68 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060108 Oct 15 21:19:33.407186 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3649/5051] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:19:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:19:37.019160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:37.019727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:37.026713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:37.027525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:37.028344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:37.029182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:37.029716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:37.030255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:37.058063 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ccc19fd1-1cda-4485-a6b4-7447e0f2351c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/ips" status: 200 len: 150 microversion: 2.1 time: 0.043588 Oct 15 21:19:37.065512 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1403/5052] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:19:37 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/ips => generated 150 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:38.591128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bf21d36e-16c9-45d6-84fa-9748ba710689"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:38.595141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:38.606880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:38.608349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:38.610213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:38.611243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:38.612002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:38.613242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:38.906283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-099cb6b7-bf9c-4e2e-92fb-8a75f14af175 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.322299 Oct 15 21:19:38.907290 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3650/5053] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:19:38 2022] POST /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments => generated 194 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:19:42.367835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bf21d36e-16c9-45d6-84fa-9748ba710689"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:42.377534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:42.381849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=25 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:42.386382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:42.387033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:42.387642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:42.388529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:42.389188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:42.389775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:42.405748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=53 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:42.495582 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Invalid volume: volume bf21d36e-16c9-45d6-84fa-9748ba710689 is already attached to instances: da1775af-9897-4ef4-9e5a-5c7373ce3761 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments HTTP/1.1 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 74 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: {"volumeAttachment": {"volumeId": "bf21d36e-16c9-45d6-84fa-9748ba710689"}} Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid volume: volume bf21d36e-16c9-45d6-84fa-9748ba710689 is already attached to instances: da1775af-9897-4ef4-9e5a-5c7373ce3761 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'server_id': 'da1775af-9897-4ef4-9e5a-5c7373ce3761', 'body': {'volumeAttachment': {'volumeId': 'bf21d36e-16c9-45d6-84fa-9748ba710689'}}} Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume bf21d36e-16c9-45d6-84fa-9748ba710689 is already attached to instances: da1775af-9897-4ef4-9e5a-5c7373ce3761 Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:19:42.497776 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume bf21d36e-16c9-45d6-84fa-9748ba710689 is already attached to instances: da1775af-9897-4ef4-9e5a-5c7373ce3761 Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume bf21d36e-16c9-45d6-84fa-9748ba710689 is already attached to instances: da1775af-9897-4ef4-9e5a-5c7373ce3761 Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 400 to user: Invalid volume: volume bf21d36e-16c9-45d6-84fa-9748ba710689 is already attached to instances: da1775af-9897-4ef4-9e5a-5c7373ce3761 {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a3b36cca-16ea-442e-b42c-771f0ddbe5a6 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.136439 Oct 15 21:19:42.503728 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1404/5054] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:19:42 2022] POST /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments => generated 174 bytes in 138 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:19:42.576345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:42.577010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:42.579393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:42.585344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:42.585904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:42.586425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:42.587403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:42.587923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:42.588552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:42.594572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:19:42.686574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lazy-loading 'services' on Instance uuid da1775af-9897-4ef4-9e5a-5c7373ce3761 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:19:42.781967 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66e9d580-f604-42a3-a17d-8efb2fb31e34 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments/bf21d36e-16c9-45d6-84fa-9748ba710689" status: 202 len: 0 microversion: 2.1 time: 0.206566 Oct 15 21:19:42.786074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3651/5055] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:19:42 2022] DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments/bf21d36e-16c9-45d6-84fa-9748ba710689 => generated 0 bytes in 213 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:19:42.795084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:42.797948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:42.806144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:42.806825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:42.807290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:42.808022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:42.808874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:42.809604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:42.859479 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b891f38b-1ed8-4bc0-b1e6-434aac3e9b64 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071253 Oct 15 21:19:42.864278 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1405/5056] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:19:42 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:43.874612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:43.878024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:43.886785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:43.887524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:43.888127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:43.889267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:43.890078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:43.890665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:43.924063 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ecc6775-6c74-4d8d-aae4-d80f30fc2417 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055782 Oct 15 21:19:43.926356 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3652/5057] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:19:43 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:19:44.295177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:44.295701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:44.304021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:44.305082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:44.306587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:44.307606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:44.308936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:44.310046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:44.367029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:19:44.502610 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1329f3ce-630c-4615-ab3c-a230bb44a40b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 204 len: 0 microversion: 2.1 time: 0.506270 Oct 15 21:19:44.505798 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1406/5058] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:19:43 2022] DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:19:44.514492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:44.515456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:44.522934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:44.523423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:44.524041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:44.524944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:44.525590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:44.526187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:44.813559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:44.821856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97989756-07c2-45dc-b1f3-4cf2f6e5653c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1681 microversion: 2.1 time: 0.311777 Oct 15 21:19:44.822438 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3653/5059] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:44 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1681 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:45.859695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:45.861184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:45.870027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:45.871608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:45.872832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:45.876946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:45.878105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:45.878993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:46.079668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:46.093074 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98f0b6ff-728c-4a79-bc00-25e75f7caa3b tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1681 microversion: 2.1 time: 0.248578 Oct 15 21:19:46.107190 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1407/5060] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:45 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1681 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:47.118936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:47.120731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:47.131456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:47.132554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:47.133259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:47.134148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:47.134756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:47.135332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:47.380420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-9617234c-d9e9-4ed3-9c61-e99c539b4b07 req-763ba6e6-cd08-438f-bd5f-e2be58a6e1b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1775af-9897-4ef4-9e5a-5c7373ce3761", "name": "network-vif-deleted", "tag": "5d17f129-635c-49c6-9c5e-63852eb0934f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:47.416301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9617234c-d9e9-4ed3-9c61-e99c539b4b07 req-763ba6e6-cd08-438f-bd5f-e2be58a6e1b4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:47.419191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9617234c-d9e9-4ed3-9c61-e99c539b4b07 req-763ba6e6-cd08-438f-bd5f-e2be58a6e1b4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:47.419648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-9617234c-d9e9-4ed3-9c61-e99c539b4b07 req-763ba6e6-cd08-438f-bd5f-e2be58a6e1b4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:47.462658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:47.471169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7cf2a8b5-2c1f-45fc-92c8-566433eb7098 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 200 len: 1614 microversion: 2.1 time: 0.356310 Oct 15 21:19:47.477390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3654/5061] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:47 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 1614 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:47.497395 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-9617234c-d9e9-4ed3-9c61-e99c539b4b07 req-763ba6e6-cd08-438f-bd5f-e2be58a6e1b4 service nova] Creating event network-vif-deleted:5d17f129-635c-49c6-9c5e-63852eb0934f for instance da1775af-9897-4ef4-9e5a-5c7373ce3761 on nch2-master-1 Oct 15 21:19:47.503295 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-9617234c-d9e9-4ed3-9c61-e99c539b4b07 req-763ba6e6-cd08-438f-bd5f-e2be58a6e1b4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.128812 Oct 15 21:19:47.504495 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1408/5062] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:19:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:48.504865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:48.506490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:48.521386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:48.522154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:48.522956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:48.524777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:48.525672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:48.528482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:48.552571 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 HTTP/1.1 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'da1775af-9897-4ef4-9e5a-5c7373ce3761'} Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:19:48.556083 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:19:48.560160 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:19:48.562883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:19:48.569481 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c3de5aa7-e245-4ce6-aeb0-0fdcf1327d47 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 404 len: 111 microversion: 2.1 time: 0.073069 Oct 15 21:19:48.570675 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3655/5063] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:48 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:19:48.580281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ba922226-049a-426e-9b9e-ccfb2568d09c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '.version_select at 0x7fe74ed28ee0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:48.640672 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ba922226-049a-426e-9b9e-ccfb2568d09c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1149358351" status: 202 len: 0 microversion: 2.1 time: 0.064959 Oct 15 21:19:48.646996 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1409/5064] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:19:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1149358351 => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:19:49.883094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fbae9334-5522-43ce-bf87-ebafccda5c29 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: .version_select at 0x7fe74ebfa5e0>, body: {"keypair": {"name": "tempest-keypair-1334686893", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLYoFozIs3TTiWFBHMQvGocZ07qjFXETgUMd5SEn8On23uHJOKazO2oGn16JzcEwN87uwwSJWrlZpjsMlfIfnm0Ry3OrOWdAIAhH9AoDXrCtDZUgmqI0VzoT1aQ9HrX2Bw=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:49.922094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-fbae9334-5522-43ce-bf87-ebafccda5c29 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:19:49.933087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-fbae9334-5522-43ce-bf87-ebafccda5c29 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:19:49.972111 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fbae9334-5522-43ce-bf87-ebafccda5c29 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.097044 Oct 15 21:19:49.973510 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3656/5065] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:19:49 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:53.140782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-465104048", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d51b163a-0bf6-4a5b-be5f-78f88001c0b8"}], "security_groups": [{"name": "tempest-securitygroup--484285772"}], "key_name": "tempest-keypair-1334686893", "user_data": "}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:53.311902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] validate_networks() for [('d51b163a-0bf6-4a5b-be5f-78f88001c0b8', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:19:54.103343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:19:54.103774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:19:54.104591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:19:54.105228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:19:54.105925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:19:54.117320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:54.118263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:54.118877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:54.119772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:54.120918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:54.121527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:54.163558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:19:54.171037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Getting quotas for user 379c929d93b54b9a9751defeef161f57 and project 87333497286e48ef95cb933c074efa70. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:19:54.188463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:19:54.213340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:19:54.312892 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2e737c3f-8151-491a-9122-d86c983a73fd tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.176857 Oct 15 21:19:54.313809 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1410/5066] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:19:53 2022] POST /compute/v2.1/servers => generated 401 bytes in 1178 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:19:54.328311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-272908f6-9888-4535-98d2-82910c644e70 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:54.330010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-272908f6-9888-4535-98d2-82910c644e70 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:54.435775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-272908f6-9888-4535-98d2-82910c644e70 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:54.443592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-272908f6-9888-4535-98d2-82910c644e70 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1241 microversion: 2.1 time: 0.122235 Oct 15 21:19:54.448569 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3657/5067] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:54 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1241 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:55.469402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:55.470606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:55.485933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:55.491487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:55.492293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:55.493315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:55.494005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:55.494658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:55.613398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:55.632088 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d7f1102-e35d-4258-bb30-5a13dcf970b3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1293 microversion: 2.1 time: 0.168003 Oct 15 21:19:55.632721 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1411/5068] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:55 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1293 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:19:56.657842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:56.658635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:56.668020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:56.669062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:56.669779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:56.670528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:56.670992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:56.671427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:56.785473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:56.798737 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-88247434-9e53-4170-a42d-b0502b37a329 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1299 microversion: 2.1 time: 0.144736 Oct 15 21:19:56.799512 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3658/5069] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:56 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1299 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:57.823881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:57.824489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:57.837093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:57.837592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:57.842764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:57.845914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:57.846415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:57.846865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:58.214868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:58.242417 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cc735b29-bcd2-4fc0-ac22-7c6f1f60c5d2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1366 microversion: 2.1 time: 0.422464 Oct 15 21:19:58.243028 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1412/5070] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:57 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1366 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:19:58.623161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4713f433-8803-41c7-ba38-858f74f578ec req-580ba8c9-dd60-4790-b1db-8be4381478cf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62", "tag": "f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:19:58.655316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4713f433-8803-41c7-ba38-858f74f578ec req-580ba8c9-dd60-4790-b1db-8be4381478cf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:58.659406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4713f433-8803-41c7-ba38-858f74f578ec req-580ba8c9-dd60-4790-b1db-8be4381478cf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:58.660238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4713f433-8803-41c7-ba38-858f74f578ec req-580ba8c9-dd60-4790-b1db-8be4381478cf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:58.692614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4713f433-8803-41c7-ba38-858f74f578ec req-580ba8c9-dd60-4790-b1db-8be4381478cf service nova] Creating event network-changed:f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f for instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 on nch2-master-1 Oct 15 21:19:58.704617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4713f433-8803-41c7-ba38-858f74f578ec req-580ba8c9-dd60-4790-b1db-8be4381478cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086896 Oct 15 21:19:58.706142 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3659/5071] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:19:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:19:59.268779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:19:59.269624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:19:59.282406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:59.282914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:59.283582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:59.284323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:19:59.284847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:19:59.285655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:19:59.511889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:19:59.520243 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0a7b2a2d-12d7-4ca2-8532-f29d5a8529f9 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1366 microversion: 2.1 time: 0.255836 Oct 15 21:19:59.527359 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1413/5072] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:19:59 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1366 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:00.558060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:00.559527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:00.567984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:00.569109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:00.569878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:00.570845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:00.571540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:00.572219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:00.879441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:00.896196 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-50bdfadd-0f28-41eb-9d5f-3cfd92672b9c tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1366 microversion: 2.1 time: 0.347729 Oct 15 21:20:00.898120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3660/5073] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:00 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1366 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:01.921493 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:01.922117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:01.937851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:01.938356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:01.940953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:01.941728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:01.942192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:01.944623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:02.335248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:02.351996 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ac9980f3-8ec5-4863-ad6b-61d4579c9c69 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1366 microversion: 2.1 time: 0.430959 Oct 15 21:20:02.357288 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1414/5074] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:01 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1366 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:03.391682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:03.397653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:03.408788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:03.410142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:03.410763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:03.411650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:03.412295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:03.413078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:03.754409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:03.762622 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86d276a9-e0ad-4fa0-8d99-2ed2ea93579f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1366 microversion: 2.1 time: 0.386282 Oct 15 21:20:03.763222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3661/5075] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:03 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1366 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:04.096284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d60ebf90-0af1-44e3-8bdc-0721b361100b req-efb76d46-7830-4d16-96e8-4fbe84ce8ea2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62", "name": "network-vif-plugged", "status": "completed", "tag": "f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:04.109740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d60ebf90-0af1-44e3-8bdc-0721b361100b req-efb76d46-7830-4d16-96e8-4fbe84ce8ea2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:04.110223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d60ebf90-0af1-44e3-8bdc-0721b361100b req-efb76d46-7830-4d16-96e8-4fbe84ce8ea2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:04.110963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d60ebf90-0af1-44e3-8bdc-0721b361100b req-efb76d46-7830-4d16-96e8-4fbe84ce8ea2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:04.130599 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d60ebf90-0af1-44e3-8bdc-0721b361100b req-efb76d46-7830-4d16-96e8-4fbe84ce8ea2 service nova] Creating event network-vif-plugged:f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f for instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 on nch2-master-1 Oct 15 21:20:04.140171 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d60ebf90-0af1-44e3-8bdc-0721b361100b req-efb76d46-7830-4d16-96e8-4fbe84ce8ea2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048446 Oct 15 21:20:04.141118 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1415/5076] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:04.788496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:04.790018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:04.799374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:04.804230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:04.809734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:04.810647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:04.814374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:04.815109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:04.815797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:05.638588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:05.653928 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf32fcf1-8b9b-456f-b37e-00d45bfaf233 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1366 microversion: 2.1 time: 0.869771 Oct 15 21:20:05.671808 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3662/5077] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:04 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1366 bytes in 888 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:06.694756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:06.696164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:06.722766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:06.725122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:06.732077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:06.732938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:06.733828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:06.735206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:06.736441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:06.738186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:07.026835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:07.042039 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-90a04dee-be74-4658-a1c5-8e5a01cee3c3 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1555 microversion: 2.1 time: 0.366428 Oct 15 21:20:07.046486 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1416/5078] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:06 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1555 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:07.080734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:07.082505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:07.094879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:07.097218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:07.098220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:07.099800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:07.101258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:07.101918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:07.250781 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08cfb151-82eb-46e9-ae62-c1134256b090 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.180064 Oct 15 21:20:07.251498 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3663/5079] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:20:07 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-interface => generated 284 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:09.303324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-591f19d9-d9f0-4be7-90cd-56d0f0cef1d4 req-4d6284e0-429e-4eb0-a260-dd4f68759315 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62", "tag": "f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:09.318092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-591f19d9-d9f0-4be7-90cd-56d0f0cef1d4 req-4d6284e0-429e-4eb0-a260-dd4f68759315 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:09.318689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-591f19d9-d9f0-4be7-90cd-56d0f0cef1d4 req-4d6284e0-429e-4eb0-a260-dd4f68759315 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:09.319230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-591f19d9-d9f0-4be7-90cd-56d0f0cef1d4 req-4d6284e0-429e-4eb0-a260-dd4f68759315 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:09.332787 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-591f19d9-d9f0-4be7-90cd-56d0f0cef1d4 req-4d6284e0-429e-4eb0-a260-dd4f68759315 service nova] Creating event network-changed:f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f for instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 on nch2-master-1 Oct 15 21:20:09.341524 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-591f19d9-d9f0-4be7-90cd-56d0f0cef1d4 req-4d6284e0-429e-4eb0-a260-dd4f68759315 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041494 Oct 15 21:20:09.342797 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1417/5080] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:13.217642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-server-smoke-211553801", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "a7d429d0-6e58-4b73-90b9-8dd123682aaf"}], "key_name": "tempest-TestSecurityGroupsBasicOps-1907142042", "security_groups": []}, "os:scheduler_hints": {"different_host": ["77690280-0359-4a18-b207-11a49e9b491c"]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:13.232869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:13.316532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] validate_networks() for [('a7d429d0-6e58-4b73-90b9-8dd123682aaf', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:20:14.519395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:20:14.519999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:20:14.520584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:20:14.522543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:20:14.523221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:20:14.557133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:14.558014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:14.558689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:14.559692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:14.560399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:14.561104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:14.581961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=59,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:14.596268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Getting quotas for project 2c132f2961594244a86c8a610462992e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:20:14.602915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Getting quotas for user 026771ec28444739afc13c47339af7b0 and project 2c132f2961594244a86c8a610462992e. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:20:14.619890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:20:14.622782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:20:14.683694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c90f7bd6-982b-47b4-9c52-30a140882bdf tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 361 microversion: 2.1 time: 1.471367 Oct 15 21:20:14.685340 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3664/5081] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:20:13 2022] POST /compute/v2.1/servers => generated 361 bytes in 1473 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:20:14.697915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-785d9fb1-0a4e-419d-880b-25aa5d563ba7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:14.699370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-785d9fb1-0a4e-419d-880b-25aa5d563ba7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:14.704303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:14.808451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-785d9fb1-0a4e-419d-880b-25aa5d563ba7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:14.818090 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-785d9fb1-0a4e-419d-880b-25aa5d563ba7 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1241 microversion: 2.1 time: 0.124141 Oct 15 21:20:14.818693 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1418/5082] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:14 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1241 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:15.849072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:15.849846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:15.858834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:15.859504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:15.860164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:15.861290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:15.862069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:15.863303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:16.058546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:16.073340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3be2d050-cf7e-4f04-af0f-9abf75f67e74 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1293 microversion: 2.1 time: 0.229904 Oct 15 21:20:16.089959 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3665/5083] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:15 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1293 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:17.102872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:17.104107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:17.121905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:17.123134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:17.123848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:17.125205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:17.125921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:17.126577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:17.139240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:17.348595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:17.361933 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-88f3fcbb-0f4a-4403-995e-58625ccad407 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.263762 Oct 15 21:20:17.363408 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1419/5084] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:17 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:18.389475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:18.390668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:18.401401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:18.403104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:18.403558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:18.406022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:18.407633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:18.411190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:18.660176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:18.677878 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b27f79cd-8987-4de6-8deb-1221bc09aaa5 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.293806 Oct 15 21:20:18.678652 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3666/5085] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:18 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:19.316041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-69f1774e-c913-444b-aeb2-c792d721ed41 req-8bf28573-6697-4d2e-a52a-6f27f3e127e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "32aba2f6-7fe2-4777-9ca1-2d205ff62174", "tag": "52c53874-2277-4203-aa1c-7920efe02c2d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:19.334400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-69f1774e-c913-444b-aeb2-c792d721ed41 req-8bf28573-6697-4d2e-a52a-6f27f3e127e9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:19.335669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-69f1774e-c913-444b-aeb2-c792d721ed41 req-8bf28573-6697-4d2e-a52a-6f27f3e127e9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:19.336570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-69f1774e-c913-444b-aeb2-c792d721ed41 req-8bf28573-6697-4d2e-a52a-6f27f3e127e9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:19.350831 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-69f1774e-c913-444b-aeb2-c792d721ed41 req-8bf28573-6697-4d2e-a52a-6f27f3e127e9 service nova] Creating event network-changed:52c53874-2277-4203-aa1c-7920efe02c2d for instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 on nch2-master-1 Oct 15 21:20:19.357249 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-69f1774e-c913-444b-aeb2-c792d721ed41 req-8bf28573-6697-4d2e-a52a-6f27f3e127e9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044568 Oct 15 21:20:19.358100 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1420/5086] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:19.703537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:19.704158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:19.711935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:19.713396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:19.713833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:19.714557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:19.715011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:19.715418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:20.171321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:20.181019 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d02af952-6579-415d-b266-e170ece8b6bc tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.482089 Oct 15 21:20:20.182910 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3667/5087] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:19 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:21.210069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:21.210795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:21.222812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:21.223351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:21.223800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:21.225947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:21.226433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:21.229529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:21.392518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:21.414182 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8fe51918-b63f-410d-a40a-a3dee4f74194 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.212014 Oct 15 21:20:21.415789 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1421/5088] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:21 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:22.440638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:22.441886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:22.449424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:22.450147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:22.450589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:22.451303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:22.451761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:22.452422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:22.576494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:22.586617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a7ae91b3-0f2d-42d1-9186-c65f950798d8 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.152275 Oct 15 21:20:22.597118 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3668/5089] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:22 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:23.615645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:23.616893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:23.624975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:23.625997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:23.626441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:23.627910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:23.628411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:23.628840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:23.726741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:23.737436 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc685ecb-75ea-4dc1-91cb-1aaf02bc3f91 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.125853 Oct 15 21:20:23.738764 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1422/5090] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:23 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:24.762457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:24.763060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:24.765830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:24.766631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:24.775988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:24.776490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:24.776949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:24.778119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:24.778613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:24.779203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:24.970562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:24.980035 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b33ac7cd-f5d3-4980-9b57-e37935c93c85 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.222204 Oct 15 21:20:24.980951 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3669/5091] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:24 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:25.770927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4edafbb2-3f8d-403c-bd9d-f66e2cd65b2b req-bd35c8d6-2969-49b8-b45f-3645a2e18e02 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "32aba2f6-7fe2-4777-9ca1-2d205ff62174", "name": "network-vif-plugged", "status": "completed", "tag": "52c53874-2277-4203-aa1c-7920efe02c2d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:25.781933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:25.797611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4edafbb2-3f8d-403c-bd9d-f66e2cd65b2b req-bd35c8d6-2969-49b8-b45f-3645a2e18e02 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:25.798400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4edafbb2-3f8d-403c-bd9d-f66e2cd65b2b req-bd35c8d6-2969-49b8-b45f-3645a2e18e02 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:25.799220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4edafbb2-3f8d-403c-bd9d-f66e2cd65b2b req-bd35c8d6-2969-49b8-b45f-3645a2e18e02 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:25.813594 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4edafbb2-3f8d-403c-bd9d-f66e2cd65b2b req-bd35c8d6-2969-49b8-b45f-3645a2e18e02 service nova] Creating event network-vif-plugged:52c53874-2277-4203-aa1c-7920efe02c2d for instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 on nch2-master-1 Oct 15 21:20:25.828263 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4edafbb2-3f8d-403c-bd9d-f66e2cd65b2b req-bd35c8d6-2969-49b8-b45f-3645a2e18e02 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060940 Oct 15 21:20:25.831238 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1423/5092] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:20:26.006495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:26.007832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:26.016782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:26.017658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:26.018105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:26.018816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:26.019419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:26.019850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:26.128736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:26.138935 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-858b3175-b1eb-46f3-969e-c3611f86cd93 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.139410 Oct 15 21:20:26.139902 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3670/5093] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:25 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:27.163970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:27.164685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:27.172725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:27.173561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:27.174545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:27.175647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:27.176293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:27.177118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:27.288918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:27.299157 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17d673da-9d70-4598-9be9-b9e7870dff23 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1299 microversion: 2.1 time: 0.138721 Oct 15 21:20:27.299998 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1424/5094] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:27 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1299 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:28.326688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:28.333951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:28.346564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:28.347276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:28.347825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:28.348842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:28.349694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:28.350240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:28.877366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:28.890710 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4923dca4-f850-4cd8-9ea9-46dbe8172672 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1472 microversion: 2.1 time: 0.570142 Oct 15 21:20:28.903667 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3671/5095] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:28 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1472 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:28.918344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:28.919047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:28.929560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:28.930235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:28.931766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:28.932867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:28.933531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:28.933966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.060001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:29.075272 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-665a9cda-8984-4a47-a7b3-d687b48d29b0 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1472 microversion: 2.1 time: 0.162781 Oct 15 21:20:29.088471 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1425/5096] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:28 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1472 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:29.107069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:29.108194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:29.119363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.122039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.122963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.123981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.124746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.125446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.240611 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-01e4b714-491c-421a-b529-44c883313fdf tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1623 microversion: 2.1 time: 0.138031 Oct 15 21:20:29.241236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3672/5097] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:29 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1623 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:29.265105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:29.265734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:29.274030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.275109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.275559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.276317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.276954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.277426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.509926 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bf67b046-8991-4dcc-b98b-5d2df1e4bed3 tempest-TestSecurityGroupsBasicOps-828488019 tempest-TestSecurityGroupsBasicOps-828488019-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1915 microversion: 2.1 time: 0.249228 Oct 15 21:20:29.511079 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1426/5098] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:29 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1915 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:29.607488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:29.608596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:29.615535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.616366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.616966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.617961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.619241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.619846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.649006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:20:29.767039 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1243b685-dce7-4b1f-9b58-132428a7e608 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "DELETE /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 204 len: 0 microversion: 2.1 time: 0.163898 Oct 15 21:20:29.768498 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3673/5099] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:20:29 2022] DELETE /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:20:29.778821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:29.780208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:29.788821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.789481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.790520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.791471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:29.792047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:29.793690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:29.939919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:29.950329 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b4ba08ed-db89-4785-8e94-54c3fa3d7617 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1478 microversion: 2.1 time: 0.177487 Oct 15 21:20:29.951293 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1427/5100] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:29 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1478 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:30.975988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:30.976709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:30.984815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:30.985640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:30.990802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:30.991871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:30.994447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:30.994911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:31.112921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:31.122386 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-adec338a-6560-4102-8a71-91281ab787a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1478 microversion: 2.1 time: 0.151373 Oct 15 21:20:31.123035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3674/5101] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:30 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1478 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:31.891116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ec7cc228-3cc7-4e1a-a178-e519cd12c679 req-9b2fac58-677b-46ba-81dd-a7b70707bb32 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "32aba2f6-7fe2-4777-9ca1-2d205ff62174", "name": "network-vif-deleted", "tag": "52c53874-2277-4203-aa1c-7920efe02c2d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:31.911263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ec7cc228-3cc7-4e1a-a178-e519cd12c679 req-9b2fac58-677b-46ba-81dd-a7b70707bb32 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:31.912135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ec7cc228-3cc7-4e1a-a178-e519cd12c679 req-9b2fac58-677b-46ba-81dd-a7b70707bb32 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:31.912774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ec7cc228-3cc7-4e1a-a178-e519cd12c679 req-9b2fac58-677b-46ba-81dd-a7b70707bb32 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:31.938211 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ec7cc228-3cc7-4e1a-a178-e519cd12c679 req-9b2fac58-677b-46ba-81dd-a7b70707bb32 service nova] Creating event network-vif-deleted:52c53874-2277-4203-aa1c-7920efe02c2d for instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 on nch2-master-1 Oct 15 21:20:31.951131 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ec7cc228-3cc7-4e1a-a178-e519cd12c679 req-9b2fac58-677b-46ba-81dd-a7b70707bb32 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064283 Oct 15 21:20:31.952400 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1428/5102] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:32.146707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:32.148720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:32.162297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:32.163605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:32.164228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:32.167365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:32.169167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:32.169827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:32.305904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:32.306393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:32.307015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:32.308192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lazy-loading 'fault' on Instance uuid 32aba2f6-7fe2-4777-9ca1-2d205ff62174 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:20:32.375355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:32.386644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ebd9e4d-9c35-4ecf-a034-4a05e2512e17 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 200 len: 1331 microversion: 2.1 time: 0.243928 Oct 15 21:20:32.387609 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3675/5103] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:32 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 1331 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:33.415448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:33.416888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 32aba2f6-7fe2-4777-9ca1-2d205ff62174] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:33.424188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:33.424714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:33.425303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:33.426740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:33.427449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:33.427975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:33.451561 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] HTTP exception thrown: Instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 could not be found. Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Request method failure captured: Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 HTTP/1.1 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyO-dLh3F3-HjB3YQZFG6aSByOxar41fQT_LttAkvw6c33JhfYN2-haBiK43qhwJY9W-UEjD3Jb9es1fW6N9UzIzZFzw0N4W7E-YwI9mhXXAFKmz7MazfEtOe-e1536ZWB9Zk4LlUjE8tvflSNvzhmO5B7HXXTYIIsACHrngKBarFkY Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 2c132f2961594244a86c8a610462992e Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestSecurityGroupsBasicOps-698525279 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2c132f2961594244a86c8a610462992e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2c132f2961594244a86c8a610462992e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2c132f2961594244a86c8a610462992e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2c132f2961594244a86c8a610462992e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestSecurityGroupsBasicOps-698525279 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 2c132f2961594244a86c8a610462992e Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestSecurityGroupsBasicOps-698525279 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestSecurityGroupsBasicOps-698525279-project Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 026771ec28444739afc13c47339af7b0 Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestSecurityGroupsBasicOps-698525279-project Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 could not be found. Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '32aba2f6-7fe2-4777-9ca1-2d205ff62174'} Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 could not be found. Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 could not be found. Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:20:33.453765 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 could not be found. Oct 15 21:20:33.460624 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:33.463585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Returning 404 to user: Instance 32aba2f6-7fe2-4777-9ca1-2d205ff62174 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:20:33.464662 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8370b5eb-f8ac-4078-b860-d2c301955561 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174" status: 404 len: 111 microversion: 2.1 time: 0.054676 Oct 15 21:20:33.465837 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1429/5104] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:33 2022] GET /compute/v2.1/servers/32aba2f6-7fe2-4777-9ca1-2d205ff62174 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:20:40.567626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:40.568139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:40.580322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-49432ddd-f7fa-453f-a493-2c33726084d3 req-6728678d-e073-4755-8b54-fb0878d6e25d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77690280-0359-4a18-b207-11a49e9b491c", "tag": "6f9e24c9-cae4-40a2-a4ea-7377744bada5"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:40.593874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:40.594372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:40.594815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:40.595528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:40.596012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:40.596409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:40.603897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-49432ddd-f7fa-453f-a493-2c33726084d3 req-6728678d-e073-4755-8b54-fb0878d6e25d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:40.604379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-49432ddd-f7fa-453f-a493-2c33726084d3 req-6728678d-e073-4755-8b54-fb0878d6e25d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:40.604827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-49432ddd-f7fa-453f-a493-2c33726084d3 req-6728678d-e073-4755-8b54-fb0878d6e25d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:40.632308 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-49432ddd-f7fa-453f-a493-2c33726084d3 req-6728678d-e073-4755-8b54-fb0878d6e25d service nova] Creating event network-changed:6f9e24c9-cae4-40a2-a4ea-7377744bada5 for instance 77690280-0359-4a18-b207-11a49e9b491c on nch1-master-1 Oct 15 21:20:40.641372 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-49432ddd-f7fa-453f-a493-2c33726084d3 req-6728678d-e073-4755-8b54-fb0878d6e25d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064615 Oct 15 21:20:40.647089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1430/5105] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:40.658307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:20:40.785618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6224114a-3ae7-41f7-b839-229f92f4c78c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "DELETE /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 204 len: 0 microversion: 2.1 time: 0.221524 Oct 15 21:20:40.792013 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3676/5106] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:20:40 2022] DELETE /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:20:40.798262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:40.799027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:40.808251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:40.808790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:40.809262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:40.809971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:40.820782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:40.821662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:41.591406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:41.602924 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-950477f1-5eb0-424f-873e-13af6a58a6a6 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1770 microversion: 2.1 time: 0.808477 Oct 15 21:20:41.605395 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1431/5107] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:40 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1770 bytes in 813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:42.631354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:42.633859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:42.643059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:42.644124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:42.644724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:42.645632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:42.654214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:42.655022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:43.074941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:43.078254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:43.093871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e5cb2f4-a243-40be-b67f-403fa80cb91a tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1770 microversion: 2.1 time: 0.468381 Oct 15 21:20:43.111390 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3677/5108] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:42 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1770 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:43.931423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-872be888-7ddc-45e5-bcad-db8d041b8ab6 req-b0d65088-e6cf-4448-83b5-301e710be722 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77690280-0359-4a18-b207-11a49e9b491c", "name": "network-vif-deleted", "tag": "6f9e24c9-cae4-40a2-a4ea-7377744bada5"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:43.950167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-872be888-7ddc-45e5-bcad-db8d041b8ab6 req-b0d65088-e6cf-4448-83b5-301e710be722 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:43.951943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-872be888-7ddc-45e5-bcad-db8d041b8ab6 req-b0d65088-e6cf-4448-83b5-301e710be722 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:43.952593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-872be888-7ddc-45e5-bcad-db8d041b8ab6 req-b0d65088-e6cf-4448-83b5-301e710be722 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:43.968737 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-872be888-7ddc-45e5-bcad-db8d041b8ab6 req-b0d65088-e6cf-4448-83b5-301e710be722 service nova] Creating event network-vif-deleted:6f9e24c9-cae4-40a2-a4ea-7377744bada5 for instance 77690280-0359-4a18-b207-11a49e9b491c on nch1-master-1 Oct 15 21:20:43.973576 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-872be888-7ddc-45e5-bcad-db8d041b8ab6 req-b0d65088-e6cf-4448-83b5-301e710be722 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047626 Oct 15 21:20:43.974395 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1432/5109] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:20:44.125029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:44.126376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:44.135286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:44.139073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:44.139807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:44.140532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:44.141808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:44.142479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:44.143855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:44.215796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:44.216321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:44.216780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:44.218379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lazy-loading 'fault' on Instance uuid 77690280-0359-4a18-b207-11a49e9b491c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:20:44.270758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '026771ec28444739afc13c47339af7b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c132f2961594244a86c8a610462992e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:44.279908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48729e5b-1a28-4e05-b9d4-2f2f968dc4a3 tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 200 len: 1384 microversion: 2.1 time: 0.160946 Oct 15 21:20:44.280785 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3678/5110] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:44 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 1384 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:45.305984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:45.307433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] [instance: 77690280-0359-4a18-b207-11a49e9b491c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:45.309693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=32 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:45.318716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:45.319666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:45.321163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:45.322171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:45.322805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:45.323411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:45.345548 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] HTTP exception thrown: Instance 77690280-0359-4a18-b207-11a49e9b491c could not be found. Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Request method failure captured: Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c HTTP/1.1 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyO-dLh3F3-HjB3YQZFG6aSByOxar41fQT_LttAkvw6c33JhfYN2-haBiK43qhwJY9W-UEjD3Jb9es1fW6N9UzIzZFzw0N4W7E-YwI9mhXXAFKmz7MazfEtOe-e1536ZWB9Zk4LlUjE8tvflSNvzhmO5B7HXXTYIIsACHrngKBarFkY Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 2c132f2961594244a86c8a610462992e Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestSecurityGroupsBasicOps-698525279 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2c132f2961594244a86c8a610462992e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2c132f2961594244a86c8a610462992e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2c132f2961594244a86c8a610462992e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2c132f2961594244a86c8a610462992e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestSecurityGroupsBasicOps-698525279 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 2c132f2961594244a86c8a610462992e Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestSecurityGroupsBasicOps-698525279 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestSecurityGroupsBasicOps-698525279-project Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 026771ec28444739afc13c47339af7b0 Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestSecurityGroupsBasicOps-698525279-project Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 77690280-0359-4a18-b207-11a49e9b491c could not be found. Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '77690280-0359-4a18-b207-11a49e9b491c'} Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77690280-0359-4a18-b207-11a49e9b491c could not be found. Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77690280-0359-4a18-b207-11a49e9b491c could not be found. Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:20:45.347230 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77690280-0359-4a18-b207-11a49e9b491c could not be found. Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Returning 404 to user: Instance 77690280-0359-4a18-b207-11a49e9b491c could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c0f01466-f14c-4420-84d7-98ab3031d94c tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c" status: 404 len: 111 microversion: 2.1 time: 0.049195 Oct 15 21:20:45.353084 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1433/5111] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:45 2022] GET /compute/v2.1/servers/77690280-0359-4a18-b207-11a49e9b491c => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:20:50.171972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5abab9c7-9812-4981-8fc9-ef0a81e3919e tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] Calling method '.version_select at 0x7fe74f12d670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:50.221305 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5abab9c7-9812-4981-8fc9-ef0a81e3919e tempest-TestSecurityGroupsBasicOps-698525279 tempest-TestSecurityGroupsBasicOps-698525279-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestSecurityGroupsBasicOps-1907142042" status: 202 len: 0 microversion: 2.1 time: 0.052875 Oct 15 21:20:50.221305 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3679/5112] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Oct 15 21:20:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestSecurityGroupsBasicOps-1907142042 => generated 0 bytes in 55 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:20:55.821417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:55.822274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=61 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:55.941992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-819245888", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "92ae3027-fac4-4d64-995d-e9c70ac0a4fb"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:55.997581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] validate_networks() for [('92ae3027-fac4-4d64-995d-e9c70ac0a4fb', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:20:56.467827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:20:56.468260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:20:56.468739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:20:56.469219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:20:56.469640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:20:56.479957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:56.480449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:56.481154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:56.481913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:56.482365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:56.482826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:56.501131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Getting quotas for project 3877567fc65349baadbfa1af9e8f5d15. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:20:56.506502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Getting quotas for user e95c827f3b5644fba8d7f14ea009f597 and project 3877567fc65349baadbfa1af9e8f5d15. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:20:56.520311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:20:56.522693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:20:56.574195 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35cdf896-3fb9-4a93-b44b-268e9c3c3d53 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.759377 Oct 15 21:20:56.574847 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1434/5113] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:20:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 761 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:20:56.586006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72e826ff-93c0-4f10-b98b-5ded085e3aa0 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:56.586715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72e826ff-93c0-4f10-b98b-5ded085e3aa0 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:56.593895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=9 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:20:56.683812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72e826ff-93c0-4f10-b98b-5ded085e3aa0 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:56.693269 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72e826ff-93c0-4f10-b98b-5ded085e3aa0 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1213 microversion: 2.1 time: 0.110923 Oct 15 21:20:56.693922 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3680/5114] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:56 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1213 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:57.718983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:57.720201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:57.744978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:57.745719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:57.746621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:57.747540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:57.748232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:57.749332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:57.833676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:57.843206 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-16ce77f8-eb5c-4ad1-a9b3-e31bc772df91 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1273 microversion: 2.1 time: 0.128649 Oct 15 21:20:57.844149 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1435/5115] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:57 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1273 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:20:58.871012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:20:58.871912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:20:58.878860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:58.879423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:58.879930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:58.880746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:58.881342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:58.882872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:59.131597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:20:59.141635 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e093172-600d-4e04-8ac8-a5c859d7c1b6 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1313 microversion: 2.1 time: 0.274874 Oct 15 21:20:59.145076 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3681/5116] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:20:58 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1313 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:20:59.330058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-cc3c28d0-e75b-4c98-8937-3c7fe12adceb req-044fa1ee-fa92-46af-b14b-8dd181e82852 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1afb212c-6b08-4d56-9749-83389e16f5c6", "tag": "264829d2-bd48-4514-9bc6-7a3dc7b2b5c8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:20:59.345692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cc3c28d0-e75b-4c98-8937-3c7fe12adceb req-044fa1ee-fa92-46af-b14b-8dd181e82852 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:20:59.346994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cc3c28d0-e75b-4c98-8937-3c7fe12adceb req-044fa1ee-fa92-46af-b14b-8dd181e82852 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:20:59.347805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cc3c28d0-e75b-4c98-8937-3c7fe12adceb req-044fa1ee-fa92-46af-b14b-8dd181e82852 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:20:59.364485 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-cc3c28d0-e75b-4c98-8937-3c7fe12adceb req-044fa1ee-fa92-46af-b14b-8dd181e82852 service nova] Creating event network-changed:264829d2-bd48-4514-9bc6-7a3dc7b2b5c8 for instance 1afb212c-6b08-4d56-9749-83389e16f5c6 on nch1-master-1 Oct 15 21:20:59.370980 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-cc3c28d0-e75b-4c98-8937-3c7fe12adceb req-044fa1ee-fa92-46af-b14b-8dd181e82852 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045673 Oct 15 21:20:59.371842 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1436/5117] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:20:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:00.165965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:00.166584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:00.173483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:00.173967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:00.174407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:00.175232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:00.175567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:00.175998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:00.317442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:00.326082 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eeb327dd-0a05-49e3-bff4-78529a47ae70 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1313 microversion: 2.1 time: 0.164763 Oct 15 21:21:00.326666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3682/5118] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:00 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:01.352543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:01.353298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:01.361811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:01.368051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:01.368510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:01.369297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:01.370209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:01.370648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:01.519576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:01.527957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d256f4a-ff49-43a9-a9d2-4bc6eb666df8 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1313 microversion: 2.1 time: 0.181705 Oct 15 21:21:01.528539 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1437/5119] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:01 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1313 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:02.552116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:02.552948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:02.561436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:02.562181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:02.562852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:02.563875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:02.564549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:02.565317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:02.728689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:02.738000 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-745a7aed-aceb-464d-8c27-d4ea98c8eb44 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1313 microversion: 2.1 time: 0.190146 Oct 15 21:21:02.738927 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3683/5120] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:02 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1313 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:03.762426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:03.763034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:03.770017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:03.770506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:03.771138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:03.772026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:03.773669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:03.774505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:03.957798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:03.966934 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2650a2d7-6185-48d3-b774-3f55f23ada1d tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1313 microversion: 2.1 time: 0.208508 Oct 15 21:21:03.967530 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1438/5121] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:03 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1313 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:04.634665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-55701ca9-4815-4103-8e1c-aa2078cc8d41 req-1f5fd1a0-1841-4821-a92d-5dffc15ca9bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1afb212c-6b08-4d56-9749-83389e16f5c6", "name": "network-vif-plugged", "status": "completed", "tag": "264829d2-bd48-4514-9bc6-7a3dc7b2b5c8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:04.645680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-55701ca9-4815-4103-8e1c-aa2078cc8d41 req-1f5fd1a0-1841-4821-a92d-5dffc15ca9bc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:04.646003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-55701ca9-4815-4103-8e1c-aa2078cc8d41 req-1f5fd1a0-1841-4821-a92d-5dffc15ca9bc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:04.646519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-55701ca9-4815-4103-8e1c-aa2078cc8d41 req-1f5fd1a0-1841-4821-a92d-5dffc15ca9bc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:04.660237 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-55701ca9-4815-4103-8e1c-aa2078cc8d41 req-1f5fd1a0-1841-4821-a92d-5dffc15ca9bc service nova] Creating event network-vif-plugged:264829d2-bd48-4514-9bc6-7a3dc7b2b5c8 for instance 1afb212c-6b08-4d56-9749-83389e16f5c6 on nch1-master-1 Oct 15 21:21:04.666952 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-55701ca9-4815-4103-8e1c-aa2078cc8d41 req-1f5fd1a0-1841-4821-a92d-5dffc15ca9bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037086 Oct 15 21:21:04.668034 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3684/5122] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:04.991924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:04.992502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:04.999812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:05.000264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:05.000705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:05.001457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:05.001911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:05.010046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:05.170132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:05.179813 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e743b038-3676-4d7f-8486-703d6af7a421 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1313 microversion: 2.1 time: 0.192114 Oct 15 21:21:05.180934 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1439/5123] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:04 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1313 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:06.206314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:06.206683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:06.213492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:06.214081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:06.214592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:06.215362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:06.216010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:06.216443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:06.355559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:06.363695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58067d5a-9664-44b3-9cc3-f869841b08eb tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1497 microversion: 2.1 time: 0.164225 Oct 15 21:21:06.364890 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3685/5124] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:06 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:07.295259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1189128794", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "62c8d754-ac32-428c-93b1-903dbb264238"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:07.305385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:07.306174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:07.351880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] validate_networks() for [('62c8d754-ac32-428c-93b1-903dbb264238', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:21:07.729867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:21:07.730466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:21:07.731423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:21:07.732165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:21:07.735680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:21:07.742715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:07.743212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:07.743731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:07.744476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:07.744940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:07.745409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:07.763269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Getting quotas for project 8ed96672d3d344a9b00edd3377df0ad3. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:21:07.768945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Getting quotas for user afa9c93b9e3d41d69f92909fd98a34a2 and project 8ed96672d3d344a9b00edd3377df0ad3. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:21:07.782364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:21:07.785002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:21:07.838156 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0afa07a6-4354-4762-ae70-0f5a66253ad3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.550306 Oct 15 21:21:07.839863 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1440/5125] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:21:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 552 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:21:07.851684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5ffa94a-4f5b-4a48-8cb1-f380545417e6 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:07.852821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c5ffa94a-4f5b-4a48-8cb1-f380545417e6 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:07.857944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:07.940547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5ffa94a-4f5b-4a48-8cb1-f380545417e6 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:07.948574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c5ffa94a-4f5b-4a48-8cb1-f380545417e6 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1214 microversion: 2.1 time: 0.101933 Oct 15 21:21:07.949329 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3686/5126] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:07 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1214 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:08.740548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:08.741198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:08.749744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:08.750247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:08.750726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:08.751439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:08.751940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:08.752378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:08.779665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:21:08.858513 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e9c23172-a4cb-423a-9486-9949cbe7669c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 204 len: 0 microversion: 2.1 time: 0.121558 Oct 15 21:21:08.859207 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1441/5127] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:08 2022] DELETE /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:21:08.868898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:08.869570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:08.877297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:08.877798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:08.878420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:08.879161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:08.879611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:08.880084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:08.972661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:08.973411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:08.980124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:08.980628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:08.981354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:08.982069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:08.982523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:08.982954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:09.017057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:09.025874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee0d3d40-fac1-43a4-a41d-807c57f5790a tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1503 microversion: 2.1 time: 0.161661 Oct 15 21:21:09.026806 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3687/5128] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:08 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1503 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:09.047903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:09.056304 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d8e1fd01-9ee6-4e8e-abf6-b2be6f961450 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1266 microversion: 2.1 time: 0.087871 Oct 15 21:21:09.056858 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1442/5129] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:08 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1266 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:10.051529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:10.052216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:10.060961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:10.061483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:10.061928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:10.062636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:10.063096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:10.063526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:10.080798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:10.082183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:10.089641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:10.090547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:10.090754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:10.091649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:10.092182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:10.092688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:10.472336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:10.478972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:10.480915 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1533f054-de18-418e-bf2b-8bd6b4a8a61c tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1503 microversion: 2.1 time: 0.433360 Oct 15 21:21:10.481790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3688/5130] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:10 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1503 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:10.487496 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6715854f-f2e0-49e9-a5de-d7e2a6092170 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.411192 Oct 15 21:21:10.488426 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1443/5131] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:10 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:11.135464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c570e674-6c24-4fd9-b567-07b49af531f0 req-79a9188c-1e5f-468a-9da0-a8ae76092f55 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1afb212c-6b08-4d56-9749-83389e16f5c6", "name": "network-vif-deleted", "tag": "264829d2-bd48-4514-9bc6-7a3dc7b2b5c8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:11.148048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c570e674-6c24-4fd9-b567-07b49af531f0 req-79a9188c-1e5f-468a-9da0-a8ae76092f55 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:11.149247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c570e674-6c24-4fd9-b567-07b49af531f0 req-79a9188c-1e5f-468a-9da0-a8ae76092f55 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:11.149874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c570e674-6c24-4fd9-b567-07b49af531f0 req-79a9188c-1e5f-468a-9da0-a8ae76092f55 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:11.163381 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c570e674-6c24-4fd9-b567-07b49af531f0 req-79a9188c-1e5f-468a-9da0-a8ae76092f55 service nova] Creating event network-vif-deleted:264829d2-bd48-4514-9bc6-7a3dc7b2b5c8 for instance 1afb212c-6b08-4d56-9749-83389e16f5c6 on nch1-master-1 Oct 15 21:21:11.168799 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c570e674-6c24-4fd9-b567-07b49af531f0 req-79a9188c-1e5f-468a-9da0-a8ae76092f55 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038963 Oct 15 21:21:11.169520 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3689/5132] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:11.505317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:11.505947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:11.512645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:11.513693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:11.517764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:11.519150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:11.519599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:11.520903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:11.521292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:11.521859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:11.522133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:11.522584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:11.523005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:11.523745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:11.526233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:11.526908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:11.577318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:11.578117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:11.579110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:11.579583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lazy-loading 'fault' on Instance uuid 1afb212c-6b08-4d56-9749-83389e16f5c6 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:21:11.705584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e95c827f3b5644fba8d7f14ea009f597', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3877567fc65349baadbfa1af9e8f5d15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:11.714153 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-daaa0e05-813c-4295-9f83-acb2fc6afa77 tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 200 len: 1303 microversion: 2.1 time: 0.212736 Oct 15 21:21:11.714735 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1444/5133] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:11 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 1303 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:11.877586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:11.886584 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83ef4d10-0076-4ae7-bc15-9c76b73a58d8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.378545 Oct 15 21:21:11.887167 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3690/5134] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:11 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:12.652442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:12.653007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:12.660018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:12.660558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:12.661018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:12.661739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:12.662197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:12.662646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:12.685984 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-334c443b-1956-42d7-90c8-d0f2ab846186 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/ips" status: 200 len: 148 microversion: 2.1 time: 0.037502 Oct 15 21:21:12.686554 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1445/5135] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:21:12 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/ips => generated 148 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:12.737495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:12.738198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] [instance: 1afb212c-6b08-4d56-9749-83389e16f5c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:12.745993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:12.746488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:12.746931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:12.747642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:12.748105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:12.748538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:12.765793 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] HTTP exception thrown: Instance 1afb212c-6b08-4d56-9749-83389e16f5c6 could not be found. Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Request method failure captured: Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 HTTP/1.1 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyQ2j5RV6uVnoL4EN8lsl6IGN1_fbS0Ggk8AhdKsy9UJtSN0BfS2MBf-yB8IANk8JNtb-OJWKfVp6SeUYdOV4m9x26VQzcTHNA6NIZFMkaf7HtZ-MI3v2bB24AkIu1egRrgFDpYqhaysQPZdeW0GQ3MZP3iYlbo_1PA8Y71MinLycFM Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 3877567fc65349baadbfa1af9e8f5d15 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesActionsTest-1498580122 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3877567fc65349baadbfa1af9e8f5d15"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3877567fc65349baadbfa1af9e8f5d15"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3877567fc65349baadbfa1af9e8f5d15"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3877567fc65349baadbfa1af9e8f5d15"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesActionsTest-1498580122 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 3877567fc65349baadbfa1af9e8f5d15 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesActionsTest-1498580122 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesActionsTest-1498580122-project Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e95c827f3b5644fba8d7f14ea009f597 Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesActionsTest-1498580122-project Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1afb212c-6b08-4d56-9749-83389e16f5c6 could not be found. Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1afb212c-6b08-4d56-9749-83389e16f5c6'} Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1afb212c-6b08-4d56-9749-83389e16f5c6 could not be found. Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1afb212c-6b08-4d56-9749-83389e16f5c6 could not be found. Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:12.767466 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1afb212c-6b08-4d56-9749-83389e16f5c6 could not be found. Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] Returning 404 to user: Instance 1afb212c-6b08-4d56-9749-83389e16f5c6 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8517b19-dadb-413a-a7f8-d13b68db53ff tempest-VolumesActionsTest-1498580122 tempest-VolumesActionsTest-1498580122-project] 10.222.0.16 "GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6" status: 404 len: 111 microversion: 2.1 time: 0.035243 Oct 15 21:21:12.772467 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3691/5136] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:12 2022] GET /compute/v2.1/servers/1afb212c-6b08-4d56-9749-83389e16f5c6 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:12.919642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:12.920344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:12.930063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:12.930569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:12.931013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:12.942053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:12.942821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:12.943489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:13.090694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:13.099257 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-73a8e04b-8faf-4d61-b5c4-905c69aeddda tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.184651 Oct 15 21:21:13.100672 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1446/5137] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:12 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:13.192460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cef4f156-01df-4bbd-86ac-4f953353b0e6 req-518fd1a9-fcb9-436a-9234-8885dd26e352 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9279f444-8681-4fb4-81e2-0852cd7619d8", "tag": "5c145264-2db8-462b-a1b7-89d54037f08d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:13.206856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cef4f156-01df-4bbd-86ac-4f953353b0e6 req-518fd1a9-fcb9-436a-9234-8885dd26e352 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:13.207357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cef4f156-01df-4bbd-86ac-4f953353b0e6 req-518fd1a9-fcb9-436a-9234-8885dd26e352 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:13.207947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cef4f156-01df-4bbd-86ac-4f953353b0e6 req-518fd1a9-fcb9-436a-9234-8885dd26e352 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:13.221536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cef4f156-01df-4bbd-86ac-4f953353b0e6 req-518fd1a9-fcb9-436a-9234-8885dd26e352 service nova] Creating event network-changed:5c145264-2db8-462b-a1b7-89d54037f08d for instance 9279f444-8681-4fb4-81e2-0852cd7619d8 on nch2-master-1 Oct 15 21:21:13.228515 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cef4f156-01df-4bbd-86ac-4f953353b0e6 req-518fd1a9-fcb9-436a-9234-8885dd26e352 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039511 Oct 15 21:21:13.229370 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3692/5138] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:21:14.037542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7cb69d54-ca4b-4771-921f-5ae51fb0b55d"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:14.040092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:14.047012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:14.047505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:14.047944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:14.048646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:14.049131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:14.049633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:14.125911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:14.126614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:14.134073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:14.134563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:14.135002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:14.135703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:14.136315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:14.136570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:14.419402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1670880e-7ae0-476e-b16c-f762015e1bad tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "POST /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.385577 Oct 15 21:21:14.429861 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1447/5139] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:21:14 2022] POST /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments => generated 194 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:14.446665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:14.456406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43f8afcd-499e-47ce-bcfa-8477b0737b2e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.335408 Oct 15 21:21:14.459140 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3693/5140] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:14 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:15.484960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:15.487163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:15.495552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:15.496050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:15.496505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:15.497244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:15.497698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:15.498123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:15.638428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:15.647240 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f071aa0e-67f3-46b9-a682-be7c23c97975 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.167252 Oct 15 21:21:15.648208 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1448/5141] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:15 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:16.673099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:16.673732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:16.676693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=39,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:16.683335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:16.683830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:16.684266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:16.684968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:16.685466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:16.685893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:16.822075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:16.829841 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7f1c5e6e-1f19-4a44-9059-2a1a9008ccfc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.160776 Oct 15 21:21:16.830401 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3694/5142] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:16 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:17.853260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:17.853874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:17.857112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:17.857935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:17.862970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:17.863451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:17.863883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:17.864577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:17.865250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:17.865680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:17.884776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:17.896468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:17.898108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:17.906432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:17.906933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:17.907389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:17.908109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:17.908575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:17.909012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:17.955889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:18.012051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lazy-loading 'services' on Instance uuid 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:21:18.023815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:18.034875 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9fed77aa-861c-4d1b-8119-f8a69fe16e4c tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.186020 Oct 15 21:21:18.049322 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1449/5143] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:17 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:18.095155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb3a3dce-29f9-453e-9249-c32800b26299 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments/7cb69d54-ca4b-4771-921f-5ae51fb0b55d" status: 202 len: 0 microversion: 2.1 time: 0.202227 Oct 15 21:21:18.096352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3695/5144] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:21:17 2022] DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments/7cb69d54-ca4b-4771-921f-5ae51fb0b55d => generated 0 bytes in 205 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:21:18.097967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-72d7d982-1531-4116-b1e0-8140c5bb2c6b req-04e562b5-d347-4759-a49c-32dc215a5d42 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9279f444-8681-4fb4-81e2-0852cd7619d8", "name": "network-vif-plugged", "status": "completed", "tag": "5c145264-2db8-462b-a1b7-89d54037f08d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:18.108875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72d7d982-1531-4116-b1e0-8140c5bb2c6b req-04e562b5-d347-4759-a49c-32dc215a5d42 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:18.109388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72d7d982-1531-4116-b1e0-8140c5bb2c6b req-04e562b5-d347-4759-a49c-32dc215a5d42 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:18.109825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72d7d982-1531-4116-b1e0-8140c5bb2c6b req-04e562b5-d347-4759-a49c-32dc215a5d42 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:18.116306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:18.119733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:18.122806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-72d7d982-1531-4116-b1e0-8140c5bb2c6b req-04e562b5-d347-4759-a49c-32dc215a5d42 service nova] Creating event network-vif-plugged:5c145264-2db8-462b-a1b7-89d54037f08d for instance 9279f444-8681-4fb4-81e2-0852cd7619d8 on nch2-master-1 Oct 15 21:21:18.127476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:18.128191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:18.128820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:18.129496 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-72d7d982-1531-4116-b1e0-8140c5bb2c6b req-04e562b5-d347-4759-a49c-32dc215a5d42 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035120 Oct 15 21:21:18.129997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:18.130498 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1450/5145] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:18.141154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:18.141679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:18.213252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77dcadfe-7be6-46f6-a133-a2e2be63e393 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.102794 Oct 15 21:21:18.218521 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3696/5146] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:21:18 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments => generated 197 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:19.060404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:19.061208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:19.067384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.067867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.068307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.069548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.070009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.070432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.207752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:19.216592 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d3501c7-60c5-4736-bcac-e3c78be7402e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.161355 Oct 15 21:21:19.217183 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1451/5147] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:19 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:19.226049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:19.228246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:19.235951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.236433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.236874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.237619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.238099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.238527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.268439 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d5ac13f-8e97-4f46-b777-4b8f25eddaca tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046525 Oct 15 21:21:19.268992 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3697/5148] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:21:19 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:21:19.332432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:19.332938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:19.339599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.340135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.340573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.341304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.341761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.342265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.369882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:21:19.447898 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7cd0cd92-dba6-490d-b2c8-76239794e035 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 204 len: 0 microversion: 2.1 time: 0.118815 Oct 15 21:21:19.448501 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1452/5149] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:19 2022] DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:21:19.457788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:19.459168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:19.465793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.466277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.466708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.467402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:19.467857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:19.468310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:19.610764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:19.619782 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ece46f6-df7d-4278-9cfe-20f040bc8216 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1679 microversion: 2.1 time: 0.166389 Oct 15 21:21:19.620357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3698/5150] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:19 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1679 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:20.240689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:20.241422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:20.248242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:20.248721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:20.249255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:20.249948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:20.250398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:20.250819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:20.390747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:20.399402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-953289e8-f2d7-438a-a093-ecb9f321573f tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1314 microversion: 2.1 time: 0.162996 Oct 15 21:21:20.400052 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1453/5151] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:20 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:20.643211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:20.643833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:20.651534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:20.652020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:20.652458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:20.653187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:20.653647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:20.654076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:20.800649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:20.809385 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71400c2f-29ea-4515-ab06-f978513b101f tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1679 microversion: 2.1 time: 0.170025 Oct 15 21:21:20.809959 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3699/5152] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:20 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1679 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:21.422708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:21.423329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:21.430202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:21.430691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:21.431130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:21.431865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:21.432324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:21.432754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:21.695907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:21.703700 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e08720bc-f0fd-4ee8-a098-d186b4a98059 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1498 microversion: 2.1 time: 0.284970 Oct 15 21:21:21.704280 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1454/5153] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:21 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1498 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:21.832079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:21.832685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:21.840240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:21.840742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:21.841302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:21.842059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:21.842566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:21.843007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:21.863818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:21.925289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:21.934043 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2e6f5c0e-7332-4898-a103-560cd3b9746e tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1612 microversion: 2.1 time: 0.105949 Oct 15 21:21:21.934798 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3700/5154] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:21 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1612 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:22.108451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:22.108962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:22.115490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.115982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.116419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:22.117148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.117603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.118204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:22.146082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:21:22.217265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f1b26584-9529-403d-b529-abceffd5f192 req-dcc8147f-cae6-4e75-a4c6-ace16b2a85a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62", "name": "network-vif-deleted", "tag": "f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:22.236043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f1b26584-9529-403d-b529-abceffd5f192 req-dcc8147f-cae6-4e75-a4c6-ace16b2a85a6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.236999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f1b26584-9529-403d-b529-abceffd5f192 req-dcc8147f-cae6-4e75-a4c6-ace16b2a85a6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.237712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f1b26584-9529-403d-b529-abceffd5f192 req-dcc8147f-cae6-4e75-a4c6-ace16b2a85a6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:22.261618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f1b26584-9529-403d-b529-abceffd5f192 req-dcc8147f-cae6-4e75-a4c6-ace16b2a85a6 service nova] Creating event network-vif-deleted:f0bf7e37-3c5f-48b8-890b-f7dac2b10e1f for instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 on nch2-master-1 Oct 15 21:21:22.269695 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f1b26584-9529-403d-b529-abceffd5f192 req-dcc8147f-cae6-4e75-a4c6-ace16b2a85a6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056066 Oct 15 21:21:22.274233 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b708309a-548a-4276-967c-52391f4eebfe tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 204 len: 0 microversion: 2.1 time: 0.169852 Oct 15 21:21:22.275680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3701/5155] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:22.275976 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1455/5156] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:22 2022] DELETE /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:21:22.285151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:22.286110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:22.295303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.296107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.299619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:22.300696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.301586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.302491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:22.513666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:22.525556 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52aa205d-3936-45f1-8704-95a495939858 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1504 microversion: 2.1 time: 0.244866 Oct 15 21:21:22.533051 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3702/5157] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:22 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1504 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:22.958755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:22.959997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:22.968051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.968544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.968992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:22.969733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:22.970199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:22.970633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:23.002981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:23.003478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:23.003922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:23.004387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lazy-loading 'fault' on Instance uuid 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:21:23.056885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '379c929d93b54b9a9751defeef161f57', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '87333497286e48ef95cb933c074efa70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:23.065317 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d17285c0-caee-4c12-9167-d033e125121d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 200 len: 1331 microversion: 2.1 time: 0.110838 Oct 15 21:21:23.065894 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1456/5158] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:22 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 1331 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:23.549394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:23.550021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:23.559017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:23.560128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:23.560598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:23.561344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:23.561809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:23.562243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:23.709679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:23.718115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7785e003-2288-49d5-82ab-79a3edcea918 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1504 microversion: 2.1 time: 0.172841 Oct 15 21:21:23.718777 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3703/5159] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:23 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1504 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:24.088380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:24.089002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:24.096464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:24.096957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:24.097580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:24.098510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:24.098937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:24.099387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:24.117457 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 HTTP/1.1 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62'} Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:24.119121 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c6249ab9-4977-489f-8ba7-1db8c6eaee60 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 404 len: 111 microversion: 2.1 time: 0.036424 Oct 15 21:21:24.124071 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1457/5160] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:24 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:24.130119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d399a6ff-b420-4831-bd14-3cf35fac4e90 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '.version_select at 0x7fe74ed5dca0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:24.151341 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d399a6ff-b420-4831-bd14-3cf35fac4e90 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1334686893" status: 202 len: 0 microversion: 2.1 time: 0.025003 Oct 15 21:21:24.151940 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3704/5161] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:21:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1334686893 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:21:24.354025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-17cb95f3-6390-4ec7-b1fd-17c4ba7a5edd req-bd2f0b40-7df3-4454-b2c5-aca07818a537 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9279f444-8681-4fb4-81e2-0852cd7619d8", "name": "network-vif-deleted", "tag": "5c145264-2db8-462b-a1b7-89d54037f08d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:24.366762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=6,nova_api:UPDATE=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:24.371070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-17cb95f3-6390-4ec7-b1fd-17c4ba7a5edd req-bd2f0b40-7df3-4454-b2c5-aca07818a537 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:24.371846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-17cb95f3-6390-4ec7-b1fd-17c4ba7a5edd req-bd2f0b40-7df3-4454-b2c5-aca07818a537 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:24.372455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-17cb95f3-6390-4ec7-b1fd-17c4ba7a5edd req-bd2f0b40-7df3-4454-b2c5-aca07818a537 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:24.392965 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-17cb95f3-6390-4ec7-b1fd-17c4ba7a5edd req-bd2f0b40-7df3-4454-b2c5-aca07818a537 service nova] Creating event network-vif-deleted:5c145264-2db8-462b-a1b7-89d54037f08d for instance 9279f444-8681-4fb4-81e2-0852cd7619d8 on nch2-master-1 Oct 15 21:21:24.411285 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-17cb95f3-6390-4ec7-b1fd-17c4ba7a5edd req-bd2f0b40-7df3-4454-b2c5-aca07818a537 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061620 Oct 15 21:21:24.412423 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1458/5162] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:24.743019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:24.744153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:24.753469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:24.753998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:24.754660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:24.755550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:24.756521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:24.756932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:24.792958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:24.793483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:24.793923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:24.794374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lazy-loading 'fault' on Instance uuid 9279f444-8681-4fb4-81e2-0852cd7619d8 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:21:24.848065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:24.860335 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13c7afac-8d85-43be-9b35-279dfa8ee88b tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 200 len: 1304 microversion: 2.1 time: 0.120916 Oct 15 21:21:24.861217 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3705/5163] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:24 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 1304 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:25.886516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:25.887219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 9279f444-8681-4fb4-81e2-0852cd7619d8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:25.893529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:25.894012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:25.894446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:25.895140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:25.895592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:25.897004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:25.912962 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] HTTP exception thrown: Instance 9279f444-8681-4fb4-81e2-0852cd7619d8 could not be found. Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Request method failure captured: Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 HTTP/1.1 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyRBSu1CwdPIGate8f0MyQYqx505G3PHX4Ar-iKwrWx-yyFpMw5jkamCi5NHL8tD16-Qq32HPlup6GVMthu87PPmu4ubCyMbjBVgvCHMo3ezbiyVlejfMBeAdMTfX1001pG8AHGu_YedEy-kTfYA6ZLFE75JUes_nC3cwjhfDaqPsIk Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 8ed96672d3d344a9b00edd3377df0ad3 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesActionsTest-1107724203 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesActionsTest-1107724203 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 8ed96672d3d344a9b00edd3377df0ad3 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesActionsTest-1107724203 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesActionsTest-1107724203-project Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: afa9c93b9e3d41d69f92909fd98a34a2 Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesActionsTest-1107724203-project Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 9279f444-8681-4fb4-81e2-0852cd7619d8 could not be found. Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '9279f444-8681-4fb4-81e2-0852cd7619d8'} Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9279f444-8681-4fb4-81e2-0852cd7619d8 could not be found. Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9279f444-8681-4fb4-81e2-0852cd7619d8 could not be found. Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:25.914766 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9279f444-8681-4fb4-81e2-0852cd7619d8 could not be found. Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Returning 404 to user: Instance 9279f444-8681-4fb4-81e2-0852cd7619d8 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9618aac2-9472-40fa-b8ab-aed88fdce0fb tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8" status: 404 len: 111 microversion: 2.1 time: 0.036183 Oct 15 21:21:25.930134 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1459/5164] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:25 2022] GET /compute/v2.1/servers/9279f444-8681-4fb4-81e2-0852cd7619d8 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:25.976449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1056936804", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "62c8d754-ac32-428c-93b1-903dbb264238"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:26.033445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] validate_networks() for [('62c8d754-ac32-428c-93b1-903dbb264238', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:21:26.337019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:26.337543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:26.344981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:26.345542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:26.346132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:26.347084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:26.347551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:26.347983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:26.362717 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 HTTP/1.1 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62'} Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:26.364288 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5b9376d3-92b9-4666-ba1d-3b8c436c6e3d tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 404 len: 111 microversion: 2.1 time: 0.033188 Oct 15 21:21:26.369664 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1460/5165] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:26 2022] DELETE /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:21:26.374808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:26.375421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:26.382095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:26.382590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:26.383030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:26.383731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:26.384187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:26.384621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:26.390460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:21:26.390884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:21:26.391355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:21:26.391766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:21:26.392178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:21:26.398761 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 HTTP/1.1 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62'} Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:26.400476 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6c1ef6c9-1b58-49ae-9ffe-69b7cc9c80e8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62" status: 404 len: 111 microversion: 2.1 time: 0.030764 Oct 15 21:21:26.405402 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1461/5166] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:26 2022] GET /compute/v2.1/servers/9d1e5f7c-758a-4b3b-bf00-9aa7f7b24a62 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:21:26.406459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:26.406459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:26.406685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:26.407574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:26.407903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:26.408451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:26.426677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Getting quotas for project 8ed96672d3d344a9b00edd3377df0ad3. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:21:26.432981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Getting quotas for user afa9c93b9e3d41d69f92909fd98a34a2 and project 8ed96672d3d344a9b00edd3377df0ad3. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:21:26.447019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:21:26.449410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:21:26.501999 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-452d03b9-087a-4306-86c9-34b9ff1b807d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.529349 Oct 15 21:21:26.502661 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3706/5167] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:21:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 531 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:21:26.515336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd706f40-6ed6-4f9e-b298-d1ca614707cc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:26.516129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd706f40-6ed6-4f9e-b298-d1ca614707cc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:26.592157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dd706f40-6ed6-4f9e-b298-d1ca614707cc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:26.601715 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dd706f40-6ed6-4f9e-b298-d1ca614707cc tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1214 microversion: 2.1 time: 0.091278 Oct 15 21:21:26.609644 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1462/5168] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:26 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1214 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:27.567500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:27.568056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:27.575823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.576257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.576707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.577485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.578121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.578677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.592963 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 HTTP/1.1 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'da1775af-9897-4ef4-9e5a-5c7373ce3761'} Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:27.594734 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3f3f6af-bfd5-4cb4-89a0-30789603dfb2 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 404 len: 111 microversion: 2.1 time: 0.033590 Oct 15 21:21:27.603395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3707/5169] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:27 2022] DELETE /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:27.605374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:27.606413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: da1775af-9897-4ef4-9e5a-5c7373ce3761] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:27.614446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.616442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.616442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.616763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.618068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.618068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.624694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:27.625359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:27.633907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.634413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.634865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.635592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.636065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.636504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.650187 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 HTTP/1.1 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'da1775af-9897-4ef4-9e5a-5c7373ce3761'} Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.653563 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.658717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance da1775af-9897-4ef4-9e5a-5c7373ce3761 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:27.659777 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7c8a342b-f841-465b-aa2c-f7cd3a4211ed tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761" status: 404 len: 111 microversion: 2.1 time: 0.058649 Oct 15 21:21:27.660858 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1463/5170] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:27 2022] GET /compute/v2.1/servers/da1775af-9897-4ef4-9e5a-5c7373ce3761 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:27.668726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:27.669256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:27.675624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.676106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.676548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.677302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.678592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.679193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.694650 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 HTTP/1.1 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '265795dc-9437-48ae-b7bd-626db1aaa6a4'} Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:27.696258 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:27.700484 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-633e9229-c8c9-4b8b-ac56-b0b912b701a8 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 404 len: 111 microversion: 2.1 time: 0.032814 Oct 15 21:21:27.701610 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1464/5171] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:27 2022] DELETE /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:21:27.706319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:27.706942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 265795dc-9437-48ae-b7bd-626db1aaa6a4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:27.713806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.714302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.714747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.715455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:27.715914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:27.716346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:27.726808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:27.731990 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 HTTP/1.1 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '265795dc-9437-48ae-b7bd-626db1aaa6a4'} Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:27.734259 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:27.738594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 265795dc-9437-48ae-b7bd-626db1aaa6a4 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:27.739134 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fda982b-9787-4904-91b8-7d390f339645 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1274 microversion: 2.1 time: 0.113971 Oct 15 21:21:27.739230 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fd246969-7931-4ef2-a708-ea92754b00ef tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4" status: 404 len: 111 microversion: 2.1 time: 0.032483 Oct 15 21:21:27.739327 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3708/5172] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:27 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1274 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:27.739410 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1465/5173] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:27 2022] GET /compute/v2.1/servers/265795dc-9437-48ae-b7bd-626db1aaa6a4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:28.764660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:28.765449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:28.773439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:28.774247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:28.774778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:28.775615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:28.776173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:28.777188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:28.892667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:28.893308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:28.901849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:28.902348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:28.902790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:28.903506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:28.903973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:28.904395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:28.919988 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c HTTP/1.1 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3c0812e6-ee5e-470c-a73a-5eaaca23354c'} Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:28.921705 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8be724e5-6525-448a-baf4-b27eb6b55390 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 404 len: 111 microversion: 2.1 time: 0.034496 Oct 15 21:21:28.926651 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3709/5174] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:28 2022] DELETE /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:21:28.932296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:28.932916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] [instance: 3c0812e6-ee5e-470c-a73a-5eaaca23354c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:28.939424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:28.939922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:28.940367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:28.941149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:28.941614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:28.942053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:28.957492 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] HTTP exception thrown: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Request method failure captured: Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c HTTP/1.1 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyLDZ7GGxR4nJKJCLlzlY28YcMs4lxqwyY6ckf338aCJHrYX2LGl4fkUkzQiA7XGLkS0OBXbcXphybf8PHoX2WUT1lAguc-pTaLGt7JXtZEeCMmKD_nOUssvzpP7u0WcyzZU82xKxWJKbLvPAjEoinHl3zhu3kTSYdZG3E4bbcKkKT0 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_87333497286e48ef95cb933c074efa70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/87333497286e48ef95cb933c074efa70"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/87333497286e48ef95cb933c074efa70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 87333497286e48ef95cb933c074efa70 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1409355566 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 379c929d93b54b9a9751defeef161f57 Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-AttachVolumeNegativeTest-1409355566-project Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '3c0812e6-ee5e-470c-a73a-5eaaca23354c'} Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:28.959992 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] Returning 404 to user: Instance 3c0812e6-ee5e-470c-a73a-5eaaca23354c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cd03914a-c479-4e58-aa2a-7d6788d07f84 tempest-AttachVolumeNegativeTest-1409355566 tempest-AttachVolumeNegativeTest-1409355566-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c" status: 404 len: 111 microversion: 2.1 time: 0.032970 Oct 15 21:21:28.968655 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3710/5175] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:28 2022] GET /compute/v2.1/servers/3c0812e6-ee5e-470c-a73a-5eaaca23354c => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:21:29.402673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:29.411097 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c87dc922-1f56-4c98-865e-c540f74afb90 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1314 microversion: 2.1 time: 0.651185 Oct 15 21:21:29.412280 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1466/5176] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:28 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1314 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:29.777655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c20aaa54-4a26-46de-945f-de5adfda6345 req-3ed05f42-15cc-4b55-a70f-9255e1f9331d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62009155-05e7-4f9f-9dd9-1a4c608d3a49", "tag": "ca574f8d-723d-47c2-866d-a5839dccdd8d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:29.798561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c20aaa54-4a26-46de-945f-de5adfda6345 req-3ed05f42-15cc-4b55-a70f-9255e1f9331d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:29.799441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c20aaa54-4a26-46de-945f-de5adfda6345 req-3ed05f42-15cc-4b55-a70f-9255e1f9331d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:29.800118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c20aaa54-4a26-46de-945f-de5adfda6345 req-3ed05f42-15cc-4b55-a70f-9255e1f9331d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:29.813209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c20aaa54-4a26-46de-945f-de5adfda6345 req-3ed05f42-15cc-4b55-a70f-9255e1f9331d service nova] Creating event network-changed:ca574f8d-723d-47c2-866d-a5839dccdd8d for instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 on nch1-master-1 Oct 15 21:21:29.819921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c20aaa54-4a26-46de-945f-de5adfda6345 req-3ed05f42-15cc-4b55-a70f-9255e1f9331d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049258 Oct 15 21:21:29.820615 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3711/5177] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:30.436394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:30.438607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:30.448111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:30.449009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:30.449692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:30.450636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:30.451316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:30.452241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:30.628221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:30.636619 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-863b6cf8-3cab-4f8a-99c1-5a0c235b10b3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1314 microversion: 2.1 time: 0.205361 Oct 15 21:21:30.637234 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1467/5178] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:30 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1314 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:31.662986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:31.663679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:31.677641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:31.678286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:31.678732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:31.679452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:31.679916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:31.680348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:31.909638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:31.918905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ec63058-57ce-4832-ad06-8b58283904f3 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1314 microversion: 2.1 time: 0.262632 Oct 15 21:21:31.919561 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3712/5179] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:31 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:32.942233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:32.943791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:32.950822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:32.951323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:32.951928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:32.952654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:32.953149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:32.953591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:33.123573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:33.133277 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d8581bd-307a-400b-8bf8-2d7560ecb46e tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1314 microversion: 2.1 time: 0.194904 Oct 15 21:21:33.142158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3713/5180] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:32 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1314 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:33.213681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-51cd74bb-26af-4266-bae1-ff214268076e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: .version_select at 0x7fe74ee84dc0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1405964179", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNwUx6GB/7BQbL07QDnwDW18c6RdYjluVeepVHTG0GvlY7GtGuYlUEtwpS9sdYaCCjXIpdEaaBV8Wa+DjBEzNu8P1TFY5w0ScuxOZw8IWOH1XGMgM19Bht05aERH9MtgBw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:33.237946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-51cd74bb-26af-4266-bae1-ff214268076e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:21:33.243818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-51cd74bb-26af-4266-bae1-ff214268076e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:21:33.268847 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-51cd74bb-26af-4266-bae1-ff214268076e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.389870 Oct 15 21:21:33.269483 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1468/5181] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:21:32 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:34.081584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf6be80d-a065-47f5-b8b7-f47874734d7b tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Action: 'create', calling method: .version_select at 0x7fe75846e310>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1735148924"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:34.158018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:34.159029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:34.167438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:34.168191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:34.169000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:34.169953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:34.170414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:34.170913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:34.346376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:34.356939 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78fe11fd-4cbc-4058-b673-fe48ebb565c8 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1314 microversion: 2.1 time: 0.202814 Oct 15 21:21:34.358188 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1469/5182] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:34 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1314 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:34.484550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf6be80d-a065-47f5-b8b7-f47874734d7b tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 338 microversion: 2.1 time: 0.530332 Oct 15 21:21:34.485394 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3714/5183] 10.222.0.16 () {60 vars in 1155 bytes} [Sat Oct 15 21:21:33 2022] POST /compute/v2.1/os-volumes => generated 338 bytes in 532 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 15 21:21:34.501269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e1c2b32-1c38-43ca-a44f-18f1f61a61ac tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74ee26a60>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:34.572907 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e1c2b32-1c38-43ca-a44f-18f1f61a61ac tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 200 len: 338 microversion: 2.1 time: 0.075733 Oct 15 21:21:34.573565 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1470/5184] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:21:34 2022] GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 338 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:21:34.813525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-65ff9b3a-5937-48da-95fb-7c11bbeb6ae0 req-e7ad05d6-16f2-4999-bc0c-3c6684090e12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62009155-05e7-4f9f-9dd9-1a4c608d3a49", "name": "network-vif-plugged", "status": "completed", "tag": "ca574f8d-723d-47c2-866d-a5839dccdd8d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:34.826817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-65ff9b3a-5937-48da-95fb-7c11bbeb6ae0 req-e7ad05d6-16f2-4999-bc0c-3c6684090e12 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:34.827430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-65ff9b3a-5937-48da-95fb-7c11bbeb6ae0 req-e7ad05d6-16f2-4999-bc0c-3c6684090e12 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:34.828105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-65ff9b3a-5937-48da-95fb-7c11bbeb6ae0 req-e7ad05d6-16f2-4999-bc0c-3c6684090e12 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:34.839506 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-65ff9b3a-5937-48da-95fb-7c11bbeb6ae0 req-e7ad05d6-16f2-4999-bc0c-3c6684090e12 service nova] Creating event network-vif-plugged:ca574f8d-723d-47c2-866d-a5839dccdd8d for instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 on nch1-master-1 Oct 15 21:21:34.845073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-65ff9b3a-5937-48da-95fb-7c11bbeb6ae0 req-e7ad05d6-16f2-4999-bc0c-3c6684090e12 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035085 Oct 15 21:21:34.845765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3715/5185] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:35.382037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:35.382681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:35.389702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:35.390272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:35.390793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:35.391496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:35.391968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:35.392400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:35.411354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:35.588900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed930d05-3995-4966-81aa-fe52f86002fe tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74ebfad30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:35.639310 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed930d05-3995-4966-81aa-fe52f86002fe tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 200 len: 339 microversion: 2.1 time: 0.056027 Oct 15 21:21:35.639886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3716/5186] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:21:35 2022] GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 339 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:21:35.653479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0de622dc-682a-476b-8304-f8fd1db0234a tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Action: 'create', calling method: .version_select at 0x7fe74f14d670>, body: {"snapshot": {"volume_id": "f735bb22-d1be-4672-94be-2747d3c2d941", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-2131153161"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:35.672110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:35.680645 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64ada787-782c-48b2-9058-bd43e3f03563 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1314 microversion: 2.1 time: 0.303567 Oct 15 21:21:35.681558 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1471/5187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:35 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1314 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:35.809253 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0de622dc-682a-476b-8304-f8fd1db0234a tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.159419 Oct 15 21:21:35.809850 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3717/5188] 10.222.0.16 () {60 vars in 1162 bytes} [Sat Oct 15 21:21:35 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:21:35.821421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-698c4d57-d19f-4db4-9e83-6da92d41c6ae tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74ee26160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:35.876081 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-698c4d57-d19f-4db4-9e83-6da92d41c6ae tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f" status: 200 len: 287 microversion: 2.1 time: 0.058751 Oct 15 21:21:35.877202 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1472/5189] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:21:35 2022] GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f => generated 287 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:35.888465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2bd054ce-ec11-4a82-885e-ab44a7971b48 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74eba0dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:35.923650 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2bd054ce-ec11-4a82-885e-ab44a7971b48 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.039058 Oct 15 21:21:35.924497 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3718/5190] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Oct 15 21:21:35 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:35.937350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e8e82810-e447-49cf-a990-ae1b933e96a6 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74f018550>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:35.971601 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e8e82810-e447-49cf-a990-ae1b933e96a6 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f" status: 200 len: 287 microversion: 2.1 time: 0.038659 Oct 15 21:21:35.972343 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1473/5191] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:21:35 2022] GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:21:36.700748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:36.702396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:36.708832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:36.709901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:36.719835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:36.720632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:36.721304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:36.722436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:36.723086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:36.723683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:36.952821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:36.961167 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c08b1587-914a-4c33-b29c-956708d64e9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1499 microversion: 2.1 time: 0.261205 Oct 15 21:21:36.961765 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3719/5192] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:36 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1499 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:36.994217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-653a5fd1-a163-41c4-84af-21f040561ba5 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74ed43310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:37.041120 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-653a5fd1-a163-41c4-84af-21f040561ba5 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f" status: 200 len: 288 microversion: 2.1 time: 0.055959 Oct 15 21:21:37.041728 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1474/5193] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:21:36 2022] GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f => generated 288 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:37.054146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa89f29b-4ca0-43c7-abcd-21f1f6931a94 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74febad30>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:37.131045 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fa89f29b-4ca0-43c7-abcd-21f1f6931a94 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "DELETE /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f" status: 202 len: 0 microversion: 2.1 time: 0.081164 Oct 15 21:21:37.131804 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3720/5194] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:21:37 2022] DELETE /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:21:37.140352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-39a238cc-2918-41f9-b47d-017cd2a1429e tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74ee260d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:37.178658 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-39a238cc-2918-41f9-b47d-017cd2a1429e tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f" status: 200 len: 287 microversion: 2.1 time: 0.042193 Oct 15 21:21:37.179290 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1475/5195] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:21:37 2022] GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f => generated 287 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:37.617760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:37.618906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:37.626757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:37.627220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:37.627839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:37.629228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:37.629658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:37.630360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:37.673624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:21:37.785322 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3733e09-a86c-41a3-b201-d2f6b3becb9d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "DELETE /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 204 len: 0 microversion: 2.1 time: 0.170619 Oct 15 21:21:37.796722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:37.797438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:37.800970 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3721/5196] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:21:37 2022] DELETE /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:21:37.804961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:37.805632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:37.806348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:37.807364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:37.808035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:37.808668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:37.959728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:37.968235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c1db9db3-9932-409c-8415-ca5ce626172d tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1505 microversion: 2.1 time: 0.176201 Oct 15 21:21:37.968956 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1476/5197] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:37 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1505 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:38.196316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ac7bdb1-c89e-4b59-afff-f70929291fbb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74eba0dc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:38.228908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-2ac7bdb1-c89e-4b59-afff-f70929291fbb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] HTTP exception thrown: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ac7bdb1-c89e-4b59-afff-f70929291fbb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Request method failure captured: Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f HTTP/1.1 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRdOuO7o39xddF-OPtJpXyhStqXVGw1VnK7BypC5SGFiqwOk31U3m8oSI_tGi94O76pimLJBc_4Di_XnJES0YQfQrdKR_yKBnXYBXUUrxEYQi9TiojMQPm_bCRwqwMaPjCZHAgAnwNxaxuaMxZqodN_ghzVvrdKTgDLcEA-OLLivew Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: edfa2630c11d49ff8d925e71abaebf9b Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: exception: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '62d57316-6ef3-41b7-8b50-9ae1cd87630f'} Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. (HTTP 404) (Request-ID: req-4a942776-fa86-4955-b928-3610ccabf791) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:38.242219 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. Oct 15 21:21:38.247554 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:38.257876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2ac7bdb1-c89e-4b59-afff-f70929291fbb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Returning 404 to user: Snapshot 62d57316-6ef3-41b7-8b50-9ae1cd87630f could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:38.257876 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2ac7bdb1-c89e-4b59-afff-f70929291fbb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f" status: 404 len: 111 microversion: 2.1 time: 0.063288 Oct 15 21:21:38.257876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3722/5198] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 21:21:38 2022] GET /compute/v2.1/os-snapshots/62d57316-6ef3-41b7-8b50-9ae1cd87630f => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:38.263487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09f95126-41fe-4128-89ea-d2dbed3238eb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74f018dc0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:38.350882 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-09f95126-41fe-4128-89ea-d2dbed3238eb tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 202 len: 0 microversion: 2.1 time: 0.092098 Oct 15 21:21:38.353271 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1477/5199] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:21:38 2022] DELETE /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 0 bytes in 95 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:21:38.365803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-388afd2b-d353-4a3f-a9e4-4a63c7ce8991 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74f1b0ee0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:38.437407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-388afd2b-d353-4a3f-a9e4-4a63c7ce8991 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 200 len: 338 microversion: 2.1 time: 0.077881 Oct 15 21:21:38.438966 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3723/5200] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:21:38 2022] GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 338 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:38.994674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:38.995703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:39.006395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:39.007522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:39.008512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:39.010123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:39.011536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:39.012172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:39.323813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:39.335687 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ac4724d9-5219-45cc-bac4-98674dc0bcf5 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1505 microversion: 2.1 time: 0.345195 Oct 15 21:21:39.351933 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1478/5201] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:38 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1505 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:39.454194 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d47616-5d46-49ef-8b75-429f8cb364c0 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74f12d0d0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:39.497636 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-46d47616-5d46-49ef-8b75-429f8cb364c0 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] HTTP exception thrown: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d47616-5d46-49ef-8b75-429f8cb364c0 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Request method failure captured: Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 HTTP/1.1 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRdOuO7o39xddF-OPtJpXyhStqXVGw1VnK7BypC5SGFiqwOk31U3m8oSI_tGi94O76pimLJBc_4Di_XnJES0YQfQrdKR_yKBnXYBXUUrxEYQi9TiojMQPm_bCRwqwMaPjCZHAgAnwNxaxuaMxZqodN_ghzVvrdKTgDLcEA-OLLivew Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: edfa2630c11d49ff8d925e71abaebf9b Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: exception: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f735bb22-d1be-4672-94be-2747d3c2d941'} Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. (HTTP 404) (Request-ID: req-ac221c07-a2c3-44d8-a32a-282a45930502) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:39.499574 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d47616-5d46-49ef-8b75-429f8cb364c0 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Returning 404 to user: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46d47616-5d46-49ef-8b75-429f8cb364c0 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 404 len: 109 microversion: 2.1 time: 0.051761 Oct 15 21:21:39.503811 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3724/5202] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:21:39 2022] GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 109 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:39.511123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad109b74-6a50-43af-a040-1b202c6686f7 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe74ef3b430>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:39.553537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-ad109b74-6a50-43af-a040-1b202c6686f7 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] HTTP exception thrown: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad109b74-6a50-43af-a040-1b202c6686f7 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Request method failure captured: Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 HTTP/1.1 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyRdOuO7o39xddF-OPtJpXyhStqXVGw1VnK7BypC5SGFiqwOk31U3m8oSI_tGi94O76pimLJBc_4Di_XnJES0YQfQrdKR_yKBnXYBXUUrxEYQi9TiojMQPm_bCRwqwMaPjCZHAgAnwNxaxuaMxZqodN_ghzVvrdKTgDLcEA-OLLivew Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: edfa2630c11d49ff8d925e71abaebf9b Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: exception: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'f735bb22-d1be-4672-94be-2747d3c2d941'} Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. (HTTP 404) (Request-ID: req-8aaac201-76e7-41a2-bac1-2d7b7b1527c4) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:39.555542 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad109b74-6a50-43af-a040-1b202c6686f7 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Returning 404 to user: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:39.560447 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ad109b74-6a50-43af-a040-1b202c6686f7 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 404 len: 109 microversion: 2.1 time: 0.052723 Oct 15 21:21:39.563647 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1479/5203] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:21:39 2022] DELETE /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 109 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:39.569705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d112b9fe-5686-4731-be80-54f465a7ce28 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Calling method '.version_select at 0x7fe758410670>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:39.618945 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-d112b9fe-5686-4731-be80-54f465a7ce28 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] HTTP exception thrown: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d112b9fe-5686-4731-be80-54f465a7ce28 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Request method failure captured: Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 HTTP/1.1 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRdOuO7o39xddF-OPtJpXyhStqXVGw1VnK7BypC5SGFiqwOk31U3m8oSI_tGi94O76pimLJBc_4Di_XnJES0YQfQrdKR_yKBnXYBXUUrxEYQi9TiojMQPm_bCRwqwMaPjCZHAgAnwNxaxuaMxZqodN_ghzVvrdKTgDLcEA-OLLivew Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/01aeeb44218e4747b21d01776e8909f6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 01aeeb44218e4747b21d01776e8909f6 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1066764473 Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: edfa2630c11d49ff8d925e71abaebf9b Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1066764473-project Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: exception: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'f735bb22-d1be-4672-94be-2747d3c2d941'} Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. (HTTP 404) (Request-ID: req-6fcad63b-9486-4c3d-90ac-040c485f757c) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:39.621311 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. Oct 15 21:21:39.625874 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:39.629405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d112b9fe-5686-4731-be80-54f465a7ce28 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] Returning 404 to user: Volume f735bb22-d1be-4672-94be-2747d3c2d941 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:39.630530 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d112b9fe-5686-4731-be80-54f465a7ce28 tempest-VolumesSnapshotsTestJSON-1066764473 tempest-VolumesSnapshotsTestJSON-1066764473-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941" status: 404 len: 109 microversion: 2.1 time: 0.065461 Oct 15 21:21:39.631569 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3725/5204] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:21:39 2022] GET /compute/v2.1/os-volumes/f735bb22-d1be-4672-94be-2747d3c2d941 => generated 109 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:39.817295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4ef639f2-7ae0-4e3f-8285-efbd810cbda6 req-a6354676-4eef-41ec-bed5-08377823e12f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62009155-05e7-4f9f-9dd9-1a4c608d3a49", "name": "network-vif-deleted", "tag": "ca574f8d-723d-47c2-866d-a5839dccdd8d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:39.853678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4ef639f2-7ae0-4e3f-8285-efbd810cbda6 req-a6354676-4eef-41ec-bed5-08377823e12f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:39.854170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4ef639f2-7ae0-4e3f-8285-efbd810cbda6 req-a6354676-4eef-41ec-bed5-08377823e12f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:39.858072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4ef639f2-7ae0-4e3f-8285-efbd810cbda6 req-a6354676-4eef-41ec-bed5-08377823e12f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:39.873138 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4ef639f2-7ae0-4e3f-8285-efbd810cbda6 req-a6354676-4eef-41ec-bed5-08377823e12f service nova] Creating event network-vif-deleted:ca574f8d-723d-47c2-866d-a5839dccdd8d for instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 on nch1-master-1 Oct 15 21:21:39.879551 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4ef639f2-7ae0-4e3f-8285-efbd810cbda6 req-a6354676-4eef-41ec-bed5-08377823e12f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066552 Oct 15 21:21:39.881546 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1480/5205] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:40.360307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:40.361143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:40.371366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:40.372113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:40.372932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:40.373875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:40.374501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:40.375105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:40.430231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:40.430666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:40.431118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:40.431578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lazy-loading 'fault' on Instance uuid 62009155-05e7-4f9f-9dd9-1a4c608d3a49 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:21:40.519406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'afa9c93b9e3d41d69f92909fd98a34a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ed96672d3d344a9b00edd3377df0ad3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:40.537245 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9d09824-2666-43d7-ad4a-5957819d81b7 tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 200 len: 1304 microversion: 2.1 time: 0.179215 Oct 15 21:21:40.537860 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3726/5206] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:40 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 1304 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:40.620763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-205154105", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "56dc5c95-574b-4ac9-90ff-8197970674b0"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1405964179", "security_groups": [{"name": "tempest-secgroup-smoke-1592018175"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:40.851126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] validate_networks() for [('56dc5c95-574b-4ac9-90ff-8197970674b0', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:21:41.430875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:21:41.431467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:21:41.432315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:21:41.432806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:21:41.433459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:21:41.450177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:41.458752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:41.459886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:41.461567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:41.462248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:41.462988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:41.487537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:21:41.496057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:21:41.516038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:21:41.519445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:21:41.564516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:41.566612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] [instance: 62009155-05e7-4f9f-9dd9-1a4c608d3a49] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:41.576930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:41.578060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:41.579841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:41.586251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:41.586921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:41.589525 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65241c2b-8cf5-4801-8346-a84213d8b6ed tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.973255 Oct 15 21:21:41.590891 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1481/5207] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:21:40 2022] POST /compute/v2.1/servers => generated 406 bytes in 976 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:21:41.599995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:41.603703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-43ea2a25-9253-4817-ac53-b68f2b43ac7a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:41.604331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-43ea2a25-9253-4817-ac53-b68f2b43ac7a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:41.637211 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] HTTP exception thrown: Instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 could not be found. Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Request method failure captured: Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 HTTP/1.1 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRBSu1CwdPIGate8f0MyQYqx505G3PHX4Ar-iKwrWx-yyFpMw5jkamCi5NHL8tD16-Qq32HPlup6GVMthu87PPmu4ubCyMbjBVgvCHMo3ezbiyVlejfMBeAdMTfX1001pG8AHGu_YedEy-kTfYA6ZLFE75JUes_nC3cwjhfDaqPsIk Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8ed96672d3d344a9b00edd3377df0ad3 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesActionsTest-1107724203 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8ed96672d3d344a9b00edd3377df0ad3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesActionsTest-1107724203 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8ed96672d3d344a9b00edd3377df0ad3 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesActionsTest-1107724203 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesActionsTest-1107724203-project Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: afa9c93b9e3d41d69f92909fd98a34a2 Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesActionsTest-1107724203-project Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 could not be found. Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '62009155-05e7-4f9f-9dd9-1a4c608d3a49'} Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 could not be found. Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 could not be found. Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:41.641605 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 could not be found. Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] Returning 404 to user: Instance 62009155-05e7-4f9f-9dd9-1a4c608d3a49 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b580fe45-e9ee-4c24-945e-1cbedd7230ab tempest-VolumesActionsTest-1107724203 tempest-VolumesActionsTest-1107724203-project] 10.222.0.16 "GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49" status: 404 len: 111 microversion: 2.1 time: 0.085956 Oct 15 21:21:41.646917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3727/5208] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:41 2022] GET /compute/v2.1/servers/62009155-05e7-4f9f-9dd9-1a4c608d3a49 => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:41.678558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-43ea2a25-9253-4817-ac53-b68f2b43ac7a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:41.688181 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-43ea2a25-9253-4817-ac53-b68f2b43ac7a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1266 microversion: 2.1 time: 0.089632 Oct 15 21:21:41.689140 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1482/5209] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:41 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1266 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:21:42.062922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-086dcc4f-9034-4b48-89c8-846931848600 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Action: 'create', calling method: .version_select at 0x7fe74ebc0820>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-501513086", "metadata": {"Type": "work"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-086dcc4f-9034-4b48-89c8-846931848600 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Request method failure captured: Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 121 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRl80xBFmKLzABNo3OUgErIFAhR0XDIeAz1tXfs7UboJJVtqbb6Ui_zjgI6wAxx_JfgwiSDSFal4s3gsY61LJs_vR8vrIAot-r1WQMYddzCFCGuj0ocqRNvvsxMhSxVtelcQOBVhszzpgV30bRhShbhxCsMTzK85drxTazx_MBzNio Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a4b05035bc3042d18ed6205634710a82 Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-501513086", "metadata": {"Type": "work"}}} Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-501513086', 'metadata': {'Type': 'work'}}}} Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '#$%' Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:21:42.068467 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-086dcc4f-9034-4b48-89c8-846931848600 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-086dcc4f-9034-4b48-89c8-846931848600 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.210969 Oct 15 21:21:42.073912 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3728/5210] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:21:41 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 212 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:21:42.079883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4cf67bf3-1b1e-439e-a26e-ea913b96ebe8 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Action: 'create', calling method: .version_select at 0x7fe74ecff550>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-811677061", "metadata": {"Type": "work"}}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4cf67bf3-1b1e-439e-a26e-ea913b96ebe8 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Request method failure captured: Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 119 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyRl80xBFmKLzABNo3OUgErIFAhR0XDIeAz1tXfs7UboJJVtqbb6Ui_zjgI6wAxx_JfgwiSDSFal4s3gsY61LJs_vR8vrIAot-r1WQMYddzCFCGuj0ocqRNvvsxMhSxVtelcQOBVhszzpgV30bRhShbhxCsMTzK85drxTazx_MBzNio Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a4b05035bc3042d18ed6205634710a82 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-811677061", "metadata": {"Type": "work"}}} Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-811677061', 'metadata': {'Type': 'work'}}}} Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi '0' Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:21:42.084820 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4cf67bf3-1b1e-439e-a26e-ea913b96ebe8 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4cf67bf3-1b1e-439e-a26e-ea913b96ebe8 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010746 Oct 15 21:21:42.090042 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1483/5211] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:21:42 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:21:42.096591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43ab16fc-dea5-431c-89b7-ff0c643def40 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Action: 'create', calling method: .version_select at 0x7fe758344dc0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1156239929", "metadata": {"Type": "work"}}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43ab16fc-dea5-431c-89b7-ff0c643def40 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Request method failure captured: Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 119 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRl80xBFmKLzABNo3OUgErIFAhR0XDIeAz1tXfs7UboJJVtqbb6Ui_zjgI6wAxx_JfgwiSDSFal4s3gsY61LJs_vR8vrIAot-r1WQMYddzCFCGuj0ocqRNvvsxMhSxVtelcQOBVhszzpgV30bRhShbhxCsMTzK85drxTazx_MBzNio Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a4b05035bc3042d18ed6205634710a82 Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1156239929", "metadata": {"Type": "work"}}} Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1156239929', 'metadata': {'Type': 'work'}}}} Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '' Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:21:42.104417 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43ab16fc-dea5-431c-89b7-ff0c643def40 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43ab16fc-dea5-431c-89b7-ff0c643def40 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.014012 Oct 15 21:21:42.109474 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3729/5212] 10.222.0.16 () {60 vars in 1156 bytes} [Sat Oct 15 21:21:42 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:21:42.118019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08bf6a79-d132-4baf-baf3-1c3c695880c0 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Calling method '.version_select at 0x7fe74f01ac10>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:42.155653 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-08bf6a79-d132-4baf-baf3-1c3c695880c0 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] HTTP exception thrown: Volume tempest-invalid-686277286 could not be found. Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08bf6a79-d132-4baf-baf3-1c3c695880c0 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Request method failure captured: Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-686277286 HTTP/1.1 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyRl80xBFmKLzABNo3OUgErIFAhR0XDIeAz1tXfs7UboJJVtqbb6Ui_zjgI6wAxx_JfgwiSDSFal4s3gsY61LJs_vR8vrIAot-r1WQMYddzCFCGuj0ocqRNvvsxMhSxVtelcQOBVhszzpgV30bRhShbhxCsMTzK85drxTazx_MBzNio Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a4b05035bc3042d18ed6205634710a82 Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: exception: Volume tempest-invalid-686277286 could not be found. Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'tempest-invalid-686277286'} Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-686277286 could not be found. (HTTP 404) (Request-ID: req-3294f849-29f4-4385-9dd0-9302ea6f7fc6) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:42.158070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-686277286 could not be found. Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-686277286 could not be found. Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-686277286 could not be found. Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08bf6a79-d132-4baf-baf3-1c3c695880c0 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Returning 404 to user: Volume tempest-invalid-686277286 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-08bf6a79-d132-4baf-baf3-1c3c695880c0 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/tempest-invalid-686277286" status: 404 len: 98 microversion: 2.1 time: 0.046805 Oct 15 21:21:42.163564 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1484/5213] 10.222.0.16 () {58 vars in 1215 bytes} [Sat Oct 15 21:21:42 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-686277286 => generated 98 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 15 21:21:42.171770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f42b2baf-824b-4d17-93b2-898f07071351 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004004 Oct 15 21:21:42.172889 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3730/5214] 10.222.0.16 () {58 vars in 1140 bytes} [Sat Oct 15 21:21:42 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 15 21:21:42.182792 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9506b049-c664-4f4b-8dde-278d33323043 tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005457 Oct 15 21:21:42.183465 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1485/5215] 10.222.0.16 () {58 vars in 1137 bytes} [Sat Oct 15 21:21:42 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Oct 15 21:21:42.195905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9ab6f0b-70bf-410d-8488-77f9ae7eaccd tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Calling method '.version_select at 0x7fe74f9eb820>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:42.277380 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c9ab6f0b-70bf-410d-8488-77f9ae7eaccd tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] HTTP exception thrown: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9ab6f0b-70bf-410d-8488-77f9ae7eaccd tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Request method failure captured: Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/os-volumes/cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 HTTP/1.1 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRl80xBFmKLzABNo3OUgErIFAhR0XDIeAz1tXfs7UboJJVtqbb6Ui_zjgI6wAxx_JfgwiSDSFal4s3gsY61LJs_vR8vrIAot-r1WQMYddzCFCGuj0ocqRNvvsxMhSxVtelcQOBVhszzpgV30bRhShbhxCsMTzK85drxTazx_MBzNio Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a4b05035bc3042d18ed6205634710a82 Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: method: Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: exception: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9'} Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. (HTTP 404) (Request-ID: req-65f17d10-d793-4245-b9d0-43093b42005c) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:42.279755 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. Oct 15 21:21:42.284486 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.293161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9ab6f0b-70bf-410d-8488-77f9ae7eaccd tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Returning 404 to user: Volume cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:42.296169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9ab6f0b-70bf-410d-8488-77f9ae7eaccd tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9" status: 404 len: 109 microversion: 2.1 time: 0.107640 Oct 15 21:21:42.296862 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3731/5216] 10.222.0.16 () {58 vars in 1248 bytes} [Sat Oct 15 21:21:42 2022] DELETE /compute/v2.1/os-volumes/cc333cd4-ddbd-4ad6-90f8-6c02cae7cba9 => generated 109 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:42.307014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9df1f756-73be-4513-920c-71093880448a tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Calling method '.version_select at 0x7fe74f017280>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:42.348136 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-9df1f756-73be-4513-920c-71093880448a tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] HTTP exception thrown: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9df1f756-73be-4513-920c-71093880448a tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Request method failure captured: Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/os-volumes/537ed3a5-5f06-4251-ac2c-96daf81b87f7 HTTP/1.1 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyRl80xBFmKLzABNo3OUgErIFAhR0XDIeAz1tXfs7UboJJVtqbb6Ui_zjgI6wAxx_JfgwiSDSFal4s3gsY61LJs_vR8vrIAot-r1WQMYddzCFCGuj0ocqRNvvsxMhSxVtelcQOBVhszzpgV30bRhShbhxCsMTzK85drxTazx_MBzNio Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f48c35cf7e049e58cd9413b58a9b1ed"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 8f48c35cf7e049e58cd9413b58a9b1ed Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesNegativeTest-1388798415 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a4b05035bc3042d18ed6205634710a82 Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesNegativeTest-1388798415-project Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: method: Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: exception: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '537ed3a5-5f06-4251-ac2c-96daf81b87f7'} Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. (HTTP 404) (Request-ID: req-45268913-44fa-4172-bbe4-6a26deac90b6) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 15 21:21:42.350931 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9df1f756-73be-4513-920c-71093880448a tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] Returning 404 to user: Volume 537ed3a5-5f06-4251-ac2c-96daf81b87f7 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9df1f756-73be-4513-920c-71093880448a tempest-VolumesNegativeTest-1388798415 tempest-VolumesNegativeTest-1388798415-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/537ed3a5-5f06-4251-ac2c-96daf81b87f7" status: 404 len: 109 microversion: 2.1 time: 0.049647 Oct 15 21:21:42.356403 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1486/5217] 10.222.0.16 () {58 vars in 1245 bytes} [Sat Oct 15 21:21:42 2022] GET /compute/v2.1/os-volumes/537ed3a5-5f06-4251-ac2c-96daf81b87f7 => generated 109 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:21:42.716931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:42.718502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:42.742093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:42.743697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:42.744363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:42.746699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:42.747182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:42.748155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:42.856398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:42.866723 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-86337593-a859-4e65-bcaf-197edb99f5c4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1318 microversion: 2.1 time: 0.157590 Oct 15 21:21:42.895920 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3732/5218] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:42 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1318 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:43.893057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:43.894345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:43.906556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:43.907062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:43.907508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:43.908244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:43.913479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:43.914574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:44.135851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:44.148719 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-33f06398-0220-4c4c-8328-145a63aec674 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1324 microversion: 2.1 time: 0.261808 Oct 15 21:21:44.149742 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1487/5219] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:43 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1324 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:45.175428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:45.176024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:45.188606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:45.189380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:45.189833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:45.190554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:45.191029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:45.191453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:45.507019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:45.516719 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cdcbb406-9d3c-47d7-b682-db764955dcc4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1392 microversion: 2.1 time: 0.345109 Oct 15 21:21:45.530367 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3733/5220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:45 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1392 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:45.587929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5ab2d078-9711-40b1-9c70-c93dca66f0d4 req-c26c9e3f-2dbb-4647-873e-ec3093601dc8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:45.599728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5ab2d078-9711-40b1-9c70-c93dca66f0d4 req-c26c9e3f-2dbb-4647-873e-ec3093601dc8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:45.600536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5ab2d078-9711-40b1-9c70-c93dca66f0d4 req-c26c9e3f-2dbb-4647-873e-ec3093601dc8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:45.601244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5ab2d078-9711-40b1-9c70-c93dca66f0d4 req-c26c9e3f-2dbb-4647-873e-ec3093601dc8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:45.614452 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5ab2d078-9711-40b1-9c70-c93dca66f0d4 req-c26c9e3f-2dbb-4647-873e-ec3093601dc8 service nova] Creating event network-changed:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch1-master-1 Oct 15 21:21:45.620977 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5ab2d078-9711-40b1-9c70-c93dca66f0d4 req-c26c9e3f-2dbb-4647-873e-ec3093601dc8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037044 Oct 15 21:21:45.622296 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1488/5221] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:46.549380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:46.551813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:46.561318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:46.562090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:46.562728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:46.563616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:46.564246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:46.564867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:46.570318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:46.752545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:46.764780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-388e7bdd-a2d9-40a7-8b3f-737eea2be5e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1392 microversion: 2.1 time: 0.222254 Oct 15 21:21:46.765880 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3734/5222] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:46 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1392 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:47.790473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:47.791989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:47.804532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:47.805467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:47.806162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:47.807301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:47.808060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:47.808721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:48.043736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:48.054367 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5309a624-e707-4b73-a444-8458c10b0b03 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1392 microversion: 2.1 time: 0.268588 Oct 15 21:21:48.076265 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1489/5223] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:47 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1392 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:49.083986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:49.085977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:49.096165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=77,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:49.100809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:49.101835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:49.102249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:49.104523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:49.105009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:49.106679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:49.341887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:49.356144 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-921b8ec8-bfac-4dc9-970b-e227bcde8f5b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1392 microversion: 2.1 time: 0.276237 Oct 15 21:21:49.356753 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3735/5224] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:49 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1392 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:50.379903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:50.381413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:50.390027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:50.390523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:50.390969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:50.391688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:50.392332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:50.392969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:50.548662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:50.571226 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e486f90d-eaf6-4b7c-85e9-d805e56c3c9f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1392 microversion: 2.1 time: 0.193476 Oct 15 21:21:50.573937 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1490/5225] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:50 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1392 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:51.041180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4fb71237-dc8a-463a-8479-79d72ded7929 req-8a2546dd-0cf8-49b4-a0fc-b6357b4b7329 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "name": "network-vif-plugged", "status": "completed", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:51.065921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4fb71237-dc8a-463a-8479-79d72ded7929 req-8a2546dd-0cf8-49b4-a0fc-b6357b4b7329 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:51.066796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4fb71237-dc8a-463a-8479-79d72ded7929 req-8a2546dd-0cf8-49b4-a0fc-b6357b4b7329 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:51.067596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4fb71237-dc8a-463a-8479-79d72ded7929 req-8a2546dd-0cf8-49b4-a0fc-b6357b4b7329 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:51.118512 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4fb71237-dc8a-463a-8479-79d72ded7929 req-8a2546dd-0cf8-49b4-a0fc-b6357b4b7329 service nova] Creating event network-vif-plugged:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch1-master-1 Oct 15 21:21:51.125774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4fb71237-dc8a-463a-8479-79d72ded7929 req-8a2546dd-0cf8-49b4-a0fc-b6357b4b7329 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089832 Oct 15 21:21:51.127231 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3736/5226] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:21:51.601284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:51.601917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:51.604996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:51.618543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:51.619114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:51.619517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:51.620250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:51.620720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:51.621188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:51.689525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=14 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:51.843013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:51.864893 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fcf34f5f-d665-4799-99ba-6a0df922cc48 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1392 microversion: 2.1 time: 0.264125 Oct 15 21:21:51.865618 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1491/5227] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:51 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1392 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:52.890095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:52.891673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:52.899399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:52.899892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:52.900331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:52.905361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:52.906049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:52.906661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:53.095276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:53.108099 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69fb7087-9840-462c-bc84-94a250d76a20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1564 microversion: 2.1 time: 0.223134 Oct 15 21:21:53.121671 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3737/5228] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:52 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1564 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:53.132174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:53.133631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:53.140927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:53.141880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:53.142541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:53.143455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:53.144121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:53.146010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:53.357517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:53.370432 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-55f782a3-249f-4670-9521-7fbde409424a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1564 microversion: 2.1 time: 0.240165 Oct 15 21:21:53.370432 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1492/5229] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:53 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1564 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:21:56.434475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:21:56.435112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:21:56.440667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-05f744ef-d9c4-43f9-bc6b-16f80ab65363 req-f7538304-cf70-4098-b59a-1ee8af775e23 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:21:56.448401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:21:56.452663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:56.456193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:56.456954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-05f744ef-d9c4-43f9-bc6b-16f80ab65363 req-f7538304-cf70-4098-b59a-1ee8af775e23 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:56.458748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-05f744ef-d9c4-43f9-bc6b-16f80ab65363 req-f7538304-cf70-4098-b59a-1ee8af775e23 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:56.459430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-05f744ef-d9c4-43f9-bc6b-16f80ab65363 req-f7538304-cf70-4098-b59a-1ee8af775e23 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:56.460081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:56.462211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:21:56.462938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:21:56.464322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:21:56.476323 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-05f744ef-d9c4-43f9-bc6b-16f80ab65363 req-f7538304-cf70-4098-b59a-1ee8af775e23 service nova] Creating event network-changed:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch1-master-1 Oct 15 21:21:56.484052 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-05f744ef-d9c4-43f9-bc6b-16f80ab65363 req-f7538304-cf70-4098-b59a-1ee8af775e23 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047457 Oct 15 21:21:56.488233 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1493/5230] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:21:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:21:56.764121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:21:56.775268 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-865377b3-9914-41aa-94fd-09b6bec79df5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1564 microversion: 2.1 time: 0.346718 Oct 15 21:21:56.776158 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3738/5231] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:21:56 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1564 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:16.328391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=52 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:16.334542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:16.336905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:16.343892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:16.344388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:16.344833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:16.345569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:16.346034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:16.346508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:16.412501 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d252aa06-0e09-4436-a333-e0451b6cbc3f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action" status: 200 len: 24560 microversion: 2.1 time: 0.086083 Oct 15 21:22:16.412766 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1494/5232] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:22:16 2022] POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action => generated 24560 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 21:22:27.436416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:27.437089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:27.439320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:27.440600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=24 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:27.448099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:27.448970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:27.450270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:27.450735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:27.451345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:27.451896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:27.901321 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-66f3e291-6cdf-4e09-9fb5-916aa8777a38 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1834 microversion: 2.1 time: 0.468799 Oct 15 21:22:27.902212 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3739/5233] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:27 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1834 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:27.926111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:27.926828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:27.929786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:27.930463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:22:27.936446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:27.936936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:27.937428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:27.938143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:27.938607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:27.939044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:28.119592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:28.129234 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4623f135-3412-4ae8-a9bd-a22fc28181db tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1683 microversion: 2.1 time: 0.207604 Oct 15 21:22:28.130402 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1495/5234] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:27 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1683 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:28.259659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:28.262258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:28.273529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:28.274315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:28.274934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:28.276663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:28.277437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:28.281324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:28.356463 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fe11885-3a90-42bd-9d24-c72d12fffefc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action" status: 200 len: 32635 microversion: 2.1 time: 0.100550 Oct 15 21:22:28.357628 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3740/5235] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:22:28 2022] POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action => generated 32635 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:22:28.529593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:28.530195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:28.536825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:28.537354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:28.537797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:28.538516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:28.538977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:28.539410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:28.585546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '4641b6af6bf940439be4f38054c6f6fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4405aa20ee8a4987878be13acb9b0029', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:28.585797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] flavor_id is None. Assuming migration. {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4196}} Oct 15 21:22:28.586782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:22:28.967472 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-09e007f1-3043-41b4-833f-85537b2cd82e tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action" status: 202 len: 0 microversion: 2.1 time: 0.441924 Oct 15 21:22:28.969711 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1496/5236] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:22:28 2022] POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action => generated 0 bytes in 445 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:22:28.979164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:28.980662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:28.990407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:28.990995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:28.991354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:28.992061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:28.992712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:28.993181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:29.254447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:29.265116 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-56362468-746d-453c-a91b-6a13fd9b3b9d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1692 microversion: 2.1 time: 0.290874 Oct 15 21:22:29.267473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3741/5237] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:28 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1692 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:30.300347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:30.301168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:30.308542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:30.309286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:30.309996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:30.318070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:30.318944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:30.320354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:30.671736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:30.682847 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e78aad86-db77-4351-bec1-ddc4b6bdf022 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1692 microversion: 2.1 time: 0.393336 Oct 15 21:22:30.683906 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1497/5238] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:30 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1692 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:31.707427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:31.708054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:31.716649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:31.717467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:31.717902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:31.718614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:31.719082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:31.720066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:31.943940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:31.953363 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1221b7f8-07f8-4c8e-b526-82ae5c1fa41c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1692 microversion: 2.1 time: 0.250301 Oct 15 21:22:31.953988 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3742/5239] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:31 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1692 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:32.977557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:32.978457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:32.985702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:32.986192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:32.987188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:32.988085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:32.988546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:32.988969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:33.188363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:33.197918 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9401b421-f0a5-41cd-8022-e5f9ed678779 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.224711 Oct 15 21:22:33.198503 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1498/5240] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:32 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:34.222185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:34.222821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:34.230830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:34.231543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:34.232113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:34.232912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:34.233529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:34.234051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:34.411109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:34.421213 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b449e83-dbc1-4e3a-9c9d-0277dc614ce6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.203027 Oct 15 21:22:34.421796 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3743/5241] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:34 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:22:35.446077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:35.446810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:35.454630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:35.455244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:35.455896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:35.456709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:35.457374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:35.457807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:35.659517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:35.671420 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bd83903a-ab54-4a3a-a35b-ee5cb2b442aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.228071 Oct 15 21:22:35.686246 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1499/5242] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:35 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:36.696826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:36.697620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:36.705748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:36.706590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:36.707095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:36.707870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:36.708396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:36.708891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:36.860053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:36.869431 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2d151c3b-5f5b-4535-9efa-12815e865f20 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.176838 Oct 15 21:22:36.870824 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3744/5243] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:36 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:37.893840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:37.894462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:37.901395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:37.901855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:37.902295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:37.903857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:37.904718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:37.905181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:38.058070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:38.075828 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-74f3bf90-1500-4f38-afd8-1ba672c02426 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.186538 Oct 15 21:22:38.076716 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1500/5244] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:37 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:39.100325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:39.100852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:39.108184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:39.108676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:39.109143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:39.109856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:39.110319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:39.110754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:39.363039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:39.371770 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a6fb231-89bd-49c8-ba41-157a6e183415 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.275873 Oct 15 21:22:39.372364 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3745/5245] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:39 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:40.396567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:40.397255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:40.404850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:40.405366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:40.406016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:40.406996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:40.407454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:40.407955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:40.632766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:40.642803 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e06acca-bc72-4cc9-975a-ebd326f0b40e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.249874 Oct 15 21:22:40.653533 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1501/5246] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:40 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:41.667115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:41.667774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:41.675644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:41.676141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:41.676583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:41.677328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:41.677791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:41.678252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:41.893573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:41.901320 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82d64b2e-2cc2-4725-8844-f36485dcf66c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.238151 Oct 15 21:22:41.901899 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3746/5247] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:41 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:42.924908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:42.925564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:42.932804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:42.933380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:42.934022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:42.934953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:42.935592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:42.936208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:43.172618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:43.182288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-43d558d7-b6cd-4213-93c7-67989e77c117 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1697 microversion: 2.1 time: 0.261365 Oct 15 21:22:43.197258 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1502/5248] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:42 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1697 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:44.175069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "name": "network-vif-unplugged", "status": "completed", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:44.188079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:44.188699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:44.190215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:44.211599 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] Creating event network-vif-unplugged:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch2-master-1 Oct 15 21:22:44.213255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:44.213889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:44.218952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:22:44.221331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:44.222483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:44.222947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:44.223671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:44.224228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:44.224654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:44.235032 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e2ca8ae2-4958-4437-9fd3-15ba2628db31 req-0087d0c6-7757-44e2-bf69-660542f510ac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.065249 Oct 15 21:22:44.236829 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3747/5249] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:22:44 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:22:44.507913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:44.518344 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7191da50-59b0-42c0-82f0-5b0a188ff54a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1696 microversion: 2.1 time: 0.310811 Oct 15 21:22:44.519196 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1503/5250] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:44 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1696 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:45.553529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:45.555074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:45.569644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:45.570144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:45.570588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:45.571522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:45.572081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:45.572712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:45.742695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:45.750727 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c99fb3e-84c5-46f2-be8e-7f8c0e965c15 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1694 microversion: 2.1 time: 0.205133 Oct 15 21:22:45.751331 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3748/5251] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:45 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1694 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:46.263759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:46.277261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:46.278255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:46.278696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:46.292310 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] Creating event network-changed:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch2-master-1 Oct 15 21:22:46.299650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:22:46.311725 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8007a95f-0ff6-4d73-aa8a-9d3fdf437af8 req-d9c51aa2-87e9-4469-86e3-8e9a313a6d62 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052030 Oct 15 21:22:46.312338 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1504/5252] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:22:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:22:46.776628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:46.777348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:46.785253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:46.785815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:46.786267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:46.786988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:46.787454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:46.787991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:47.501080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:47.510133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3bcd129f-c02a-44cd-bf8f-622823d9b844 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1694 microversion: 2.1 time: 0.739612 Oct 15 21:22:47.510873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3749/5253] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:46 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1694 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:48.537811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:48.538580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:48.556708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:48.557256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:48.557715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:48.558438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:48.558908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:48.559343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:48.858252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:48.876922 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9e663fe2-45e6-4bed-be05-9d41df91bc2d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1691 microversion: 2.1 time: 0.345644 Oct 15 21:22:48.879145 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1505/5254] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:48 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1691 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:48.905378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:48.910644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:48.923062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:48.924208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:48.925807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:48.929588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:48.930819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:48.931443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:49.016350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-775c0e33-02c9-4b5e-9c70-a0599787b1fa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action" status: 204 len: 0 microversion: 2.1 time: 0.117344 Oct 15 21:22:49.017254 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3750/5255] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:22:48 2022] POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:22:49.025842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:49.026633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:49.033458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:49.034288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:49.034932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:49.035649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:49.036174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:49.036617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:49.386188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:49.394681 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-18d510cd-1d14-463d-89a7-ef0d5ddaf9bb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1691 microversion: 2.1 time: 0.373156 Oct 15 21:22:49.395492 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1506/5256] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:49 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1691 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:22:50.402524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-661d3950-d14a-4a87-a5a1-81b885da57fe req-6ca54518-ee11-469b-bc78-a9f1a3035bc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "name": "network-vif-plugged", "status": "completed", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:22:50.419609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:50.420345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:50.425149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-661d3950-d14a-4a87-a5a1-81b885da57fe req-6ca54518-ee11-469b-bc78-a9f1a3035bc5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:50.427495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-661d3950-d14a-4a87-a5a1-81b885da57fe req-6ca54518-ee11-469b-bc78-a9f1a3035bc5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:50.428003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-661d3950-d14a-4a87-a5a1-81b885da57fe req-6ca54518-ee11-469b-bc78-a9f1a3035bc5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:50.433948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:50.434578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:50.435087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:50.435878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:50.436407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:50.436928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:50.456679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-661d3950-d14a-4a87-a5a1-81b885da57fe req-6ca54518-ee11-469b-bc78-a9f1a3035bc5 service nova] Creating event network-vif-plugged:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch2-master-1 Oct 15 21:22:50.467336 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-661d3950-d14a-4a87-a5a1-81b885da57fe req-6ca54518-ee11-469b-bc78-a9f1a3035bc5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070206 Oct 15 21:22:50.469647 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3751/5257] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:22:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:22:50.633135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:50.645980 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2061946d-7253-4a44-9cae-419b16ccd8fb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1691 microversion: 2.1 time: 0.229905 Oct 15 21:22:50.646962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1507/5258] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:50 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1691 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:22:51.673323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:22:51.674104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:22:51.681743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:51.682217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:51.682670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:51.683384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:22:51.684190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:22:51.684357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:22:51.838806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:22:51.847981 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5009f66a-1bdf-429e-9c56-5b3c0eb43ebd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1683 microversion: 2.1 time: 0.179874 Oct 15 21:22:51.848581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3752/5259] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:22:51 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1683 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:23:20.070833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:23:20.072873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:20.075129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:20.075802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:20.082125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:20.082617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:20.083778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:20.084570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:20.085043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:20.085482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:20.147777 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-19b53306-95e6-4483-8e5b-d6631ccf7890 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action" status: 200 len: 24575 microversion: 2.1 time: 0.081566 Oct 15 21:23:20.149109 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1508/5260] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:23:20 2022] POST /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6/action => generated 24575 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:23:43.242609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:43.243341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:43.246311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=28 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:43.247252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:43.253576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:43.254069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:43.254518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:43.255225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:43.255694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:43.256125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:43.568752 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ad6093f-8032-4e24-b150-22d6e4edeac1 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1834 microversion: 2.1 time: 0.330422 Oct 15 21:23:43.569584 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3753/5261] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:23:43 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1834 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:23:44.921809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:44.922974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:44.925977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:44.927198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:44.935178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:44.935863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:44.936491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:44.937437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:44.938064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:44.938664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:44.969864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:23:45.086045 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a7411372-7303-447b-a9f1-da78f41f6008 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 204 len: 0 microversion: 2.1 time: 0.168395 Oct 15 21:23:45.086716 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1509/5262] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:23:44 2022] DELETE /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:23:45.097553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:45.098745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:45.106582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:45.107385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:45.108006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:45.109203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:45.109844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:45.110452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:45.267617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0f82d45b-61f9-47e2-9188-225d4b14b0af req-cfc54f21-df91-43b8-8441-55be82578511 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:23:45.279575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:23:45.283264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f82d45b-61f9-47e2-9188-225d4b14b0af req-cfc54f21-df91-43b8-8441-55be82578511 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:45.284037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f82d45b-61f9-47e2-9188-225d4b14b0af req-cfc54f21-df91-43b8-8441-55be82578511 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:45.284694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0f82d45b-61f9-47e2-9188-225d4b14b0af req-cfc54f21-df91-43b8-8441-55be82578511 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:45.289981 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-72f9e97e-0121-40a7-9971-1e24c2c30fac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1689 microversion: 2.1 time: 0.198000 Oct 15 21:23:45.290570 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1510/5263] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:23:45 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1689 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:23:45.306975 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0f82d45b-61f9-47e2-9188-225d4b14b0af req-cfc54f21-df91-43b8-8441-55be82578511 service nova] Creating event network-changed:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch2-master-1 Oct 15 21:23:45.315467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0f82d45b-61f9-47e2-9188-225d4b14b0af req-cfc54f21-df91-43b8-8441-55be82578511 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.392626 Oct 15 21:23:45.316716 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3754/5264] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:23:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:23:46.314488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:46.315765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:46.330847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:46.331840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:46.332519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:46.333448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:46.334085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:46.334557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:46.475021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:23:46.484733 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f976a9c7-230f-462b-b1df-1d75b0bdd732 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1689 microversion: 2.1 time: 0.174341 Oct 15 21:23:46.485667 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1511/5265] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:23:46 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1689 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:23:47.425680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fc00a4b3-1775-4aad-ab4e-f422387c79b1 req-292f7fa6-98d9-4f04-a8d9-7569ab492da4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6", "name": "network-vif-deleted", "tag": "087b3777-18ac-46cf-a50f-5b9131358262"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:23:47.435644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc00a4b3-1775-4aad-ab4e-f422387c79b1 req-292f7fa6-98d9-4f04-a8d9-7569ab492da4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:47.436136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc00a4b3-1775-4aad-ab4e-f422387c79b1 req-292f7fa6-98d9-4f04-a8d9-7569ab492da4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:47.436580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fc00a4b3-1775-4aad-ab4e-f422387c79b1 req-292f7fa6-98d9-4f04-a8d9-7569ab492da4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:47.448992 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fc00a4b3-1775-4aad-ab4e-f422387c79b1 req-292f7fa6-98d9-4f04-a8d9-7569ab492da4 service nova] Creating event network-vif-deleted:087b3777-18ac-46cf-a50f-5b9131358262 for instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 on nch2-master-1 Oct 15 21:23:47.453710 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fc00a4b3-1775-4aad-ab4e-f422387c79b1 req-292f7fa6-98d9-4f04-a8d9-7569ab492da4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033072 Oct 15 21:23:47.454535 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3755/5266] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:23:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:23:47.511706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:47.512320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:47.520110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:47.520602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:47.521058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:47.521762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:47.523790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:47.524402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:47.557317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:47.557826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:47.558271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:47.558733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lazy-loading 'fault' on Instance uuid d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:23:47.601089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:23:47.609537 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-be6572d3-0804-45bb-910b-c25064a93814 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 200 len: 1356 microversion: 2.1 time: 0.103089 Oct 15 21:23:47.610106 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1512/5267] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:23:47 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 1356 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:23:48.634691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:48.635441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:48.641666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:48.642156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:48.642599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:48.643301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:48.643766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:48.644204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:48.658184 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] HTTP exception thrown: Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 could not be found. Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Request method failure captured: Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 HTTP/1.1 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRbA0cLyHQrk49g1yDTx2mawYGe7wmuR6AIZfAxBzfijpxhir3KV-4WCV7R2Uxj7n0-Vd4Epp7EBbcjQo861nJwrUnuYBBc0VHGRvHVNbDWK0zr0_BWwaxNQHYbUerlTCJoIXcNVviBEtMXGQni285yOrYoEZhsCaxrPEmh_p7oayg Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a0d6261b46824acf958ef749425ae421 Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 could not be found. Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd45e6a01-1e9e-4082-8d3d-6eb3ac71dac6'} Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 could not be found. Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 could not be found. Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:23:48.659794 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 could not be found. Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Returning 404 to user: Instance d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8738052d-2ce1-4a7b-8530-049249e2e3cf tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6" status: 404 len: 111 microversion: 2.1 time: 0.032030 Oct 15 21:23:48.666328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3756/5268] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:23:48 2022] GET /compute/v2.1/servers/d45e6a01-1e9e-4082-8d3d-6eb3ac71dac6 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:23:52.087842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e87146fe-2905-4c3d-a02c-daf2783cb6a2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '.version_select at 0x7fe74ec92670>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:52.120702 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e87146fe-2905-4c3d-a02c-daf2783cb6a2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1405964179" status: 202 len: 0 microversion: 2.1 time: 0.037421 Oct 15 21:23:52.121327 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1513/5269] 10.222.0.16 () {58 vars in 1284 bytes} [Sat Oct 15 21:23:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1405964179 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:23:52.151903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-424b2783-d1d0-4916-89f0-e5a9b4926bd6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: .version_select at 0x7fe7583524c0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-62898679", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBdBUbtu6u3YAwdRb19vB1gLrgfnBMhxpg7CxrO9w2Z9ldHHnP0rGyxV4dtO+lw4bqC9leMk5Jx62mrzx0bVyhICs+5CNi+Oim9Pwo/06smxV00qmdBsFNekxoF9FokXHA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:23:52.170468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-424b2783-d1d0-4916-89f0-e5a9b4926bd6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:23:52.175790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-424b2783-d1d0-4916-89f0-e5a9b4926bd6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:23:52.216833 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-424b2783-d1d0-4916-89f0-e5a9b4926bd6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 405 microversion: 2.1 time: 0.069630 Oct 15 21:23:52.217487 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3757/5270] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:23:52 2022] POST /compute/v2.1/os-keypairs => generated 405 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:23:58.475852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=34,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:58.482923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-273666172", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "57395d03-f494-4b63-afd3-bbb370b1fee4"}], "key_name": "tempest-TestNetworkAdvancedServerOps-62898679", "security_groups": [{"name": "tempest-secgroup-smoke-442668048"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:23:58.654080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] validate_networks() for [('57395d03-f494-4b63-afd3-bbb370b1fee4', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:23:59.274679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:23:59.275678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:23:59.276420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:23:59.277399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:23:59.278067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:23:59.291134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:59.291723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:59.292240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:59.293071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:23:59.293781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:23:59.294291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:23:59.313185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:23:59.320469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:23:59.334201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:23:59.336645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:23:59.394373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5a67e424-b145-4498-8b57-f1da64304a0a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.922980 Oct 15 21:23:59.403291 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1514/5271] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:23:58 2022] POST /compute/v2.1/servers => generated 405 bytes in 933 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:23:59.407604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd711f14-7443-4972-94e8-502993ba0ee8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:23:59.408297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd711f14-7443-4972-94e8-502993ba0ee8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:23:59.411538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:23:59.487579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fd711f14-7443-4972-94e8-502993ba0ee8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:23:59.495101 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd711f14-7443-4972-94e8-502993ba0ee8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1264 microversion: 2.1 time: 0.092354 Oct 15 21:23:59.495680 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3758/5272] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:23:59 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1264 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:00.527205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:00.528423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:00.538157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:00.539534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:00.540935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:00.541711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:00.542184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:00.542955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:00.684755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:00.712188 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cc9cfa2a-5605-48bd-b8e2-8ef6c436f1d4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1324 microversion: 2.1 time: 0.176105 Oct 15 21:24:00.712188 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1515/5273] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:00 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1324 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:01.732759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:01.733487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:01.741781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:01.742368 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:01.743073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:01.744010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:01.745305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:01.745921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:01.979735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:01.990050 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9f19639c-2597-4fb3-b396-1b61fb9aa0b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1389 microversion: 2.1 time: 0.264498 Oct 15 21:24:01.991570 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3759/5274] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:01 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1389 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:02.377268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-1139d4a1-a2f3-4dd3-9d1a-54468154d673 req-ac3375cb-e86e-4b53-a0fb-c9ef29464f59 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:02.403305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1139d4a1-a2f3-4dd3-9d1a-54468154d673 req-ac3375cb-e86e-4b53-a0fb-c9ef29464f59 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:02.403782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1139d4a1-a2f3-4dd3-9d1a-54468154d673 req-ac3375cb-e86e-4b53-a0fb-c9ef29464f59 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:02.404333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1139d4a1-a2f3-4dd3-9d1a-54468154d673 req-ac3375cb-e86e-4b53-a0fb-c9ef29464f59 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:02.416226 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-1139d4a1-a2f3-4dd3-9d1a-54468154d673 req-ac3375cb-e86e-4b53-a0fb-c9ef29464f59 service nova] Creating event network-changed:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:24:02.421547 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-1139d4a1-a2f3-4dd3-9d1a-54468154d673 req-ac3375cb-e86e-4b53-a0fb-c9ef29464f59 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048618 Oct 15 21:24:02.422487 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1516/5275] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:24:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:24:03.014526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:03.015141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:03.022703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:03.023141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:03.023581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:03.024282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:03.024742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:03.025201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:03.165433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:03.173557 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f6cf507-7551-4877-8097-c84ab6382a79 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1389 microversion: 2.1 time: 0.162918 Oct 15 21:24:03.174218 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3760/5276] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:03 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1389 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:04.197897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:04.199355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:04.207781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:04.208306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:04.208759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:04.209538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:04.210013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:04.210450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:04.366810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:04.376473 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cb9c1beb-d0ea-4871-8590-f37d2b183c8f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1389 microversion: 2.1 time: 0.183114 Oct 15 21:24:04.377266 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1517/5277] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:04 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1389 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:05.406602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:05.408585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:05.417437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:05.418026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:05.418545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:05.419502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:05.420034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:05.420875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:05.577514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:05.586142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-20af4e6b-3d9b-4142-8c1a-31730015a9e9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1389 microversion: 2.1 time: 0.188863 Oct 15 21:24:05.586734 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3761/5278] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:05 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1389 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:06.610051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:06.610694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:06.618034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:06.618530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:06.618975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:06.619689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:06.620153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:06.620590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:06.767947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:06.776025 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-08c2d782-9c90-464b-a36e-3c4a10f18a59 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1389 microversion: 2.1 time: 0.170739 Oct 15 21:24:06.776702 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1518/5279] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:06 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1389 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:07.406383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-aa97a85c-7ef1-40ec-91a6-b42218a3dc05 req-88272151-6768-4953-b91c-a72b8ede03ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "name": "network-vif-plugged", "status": "completed", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:07.418349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aa97a85c-7ef1-40ec-91a6-b42218a3dc05 req-88272151-6768-4953-b91c-a72b8ede03ce service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:07.419275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aa97a85c-7ef1-40ec-91a6-b42218a3dc05 req-88272151-6768-4953-b91c-a72b8ede03ce service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:07.419812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aa97a85c-7ef1-40ec-91a6-b42218a3dc05 req-88272151-6768-4953-b91c-a72b8ede03ce service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:07.436655 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-aa97a85c-7ef1-40ec-91a6-b42218a3dc05 req-88272151-6768-4953-b91c-a72b8ede03ce service nova] Creating event network-vif-plugged:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:24:07.441794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-aa97a85c-7ef1-40ec-91a6-b42218a3dc05 req-88272151-6768-4953-b91c-a72b8ede03ce service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039289 Oct 15 21:24:07.442700 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3762/5280] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:24:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:24:07.799298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:07.799895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:07.807913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:07.808401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:07.808850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:07.809558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:07.810016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:07.810438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:07.948535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:07.956197 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-98dd4b2e-ab06-43ba-9d3f-a5ca5eb68b22 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1389 microversion: 2.1 time: 0.160854 Oct 15 21:24:07.956775 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1519/5281] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:07 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1389 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:08.982095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:08.982820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:08.989252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:08.989740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:08.990825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:08.991538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:08.992004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:08.992432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:09.129120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:09.137693 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f035145d-c388-45fa-8e8f-1356a6310b01 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1561 microversion: 2.1 time: 0.160953 Oct 15 21:24:09.138267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3763/5282] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:08 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:09.160683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:09.161333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:09.168621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:09.169151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:09.169598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:09.170305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:09.170768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:09.171202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:09.304743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:09.308135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:09.316113 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dbea3e52-c73f-46dd-aff6-22650ea00fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1561 microversion: 2.1 time: 0.159236 Oct 15 21:24:09.316679 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1520/5283] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:09 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:11.491199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:11.492590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:11.496011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:11.497504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-008aa49d-2bb6-4104-9f83-941e2be18be6 req-5d39b401-432d-429a-bf0b-977cbc4ede47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:11.504369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:11.507057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:11.507600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:11.508191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:11.509827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:11.510589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:11.511129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:11.511924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-008aa49d-2bb6-4104-9f83-941e2be18be6 req-5d39b401-432d-429a-bf0b-977cbc4ede47 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:11.512550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-008aa49d-2bb6-4104-9f83-941e2be18be6 req-5d39b401-432d-429a-bf0b-977cbc4ede47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:11.513168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-008aa49d-2bb6-4104-9f83-941e2be18be6 req-5d39b401-432d-429a-bf0b-977cbc4ede47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:11.526915 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-008aa49d-2bb6-4104-9f83-941e2be18be6 req-5d39b401-432d-429a-bf0b-977cbc4ede47 service nova] Creating event network-changed:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:24:11.532620 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-008aa49d-2bb6-4104-9f83-941e2be18be6 req-5d39b401-432d-429a-bf0b-977cbc4ede47 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039008 Oct 15 21:24:11.533682 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1521/5284] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:24:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:24:11.788580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:11.799135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0238e585-5e65-4b0f-9896-3f227c20a043 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1561 microversion: 2.1 time: 0.312103 Oct 15 21:24:11.813679 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3764/5285] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:11 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1561 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:33.068787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:33.070190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=36 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:33.079322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:33.082236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:33.091374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:33.091875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:33.092488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:33.093231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:33.093730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:33.094159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:33.160394 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-86a49187-812c-4ca6-91a4-5c0e74bafedb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action" status: 200 len: 24571 microversion: 2.1 time: 0.094794 Oct 15 21:24:33.161128 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1522/5286] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:24:33 2022] POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action => generated 24571 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:24:39.454022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1562050566", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c44a3d28-5fb1-4d4f-a5d6-b74bdb233a9e"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:39.462946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:39.463662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:39.512434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] validate_networks() for [('c44a3d28-5fb1-4d4f-a5d6-b74bdb233a9e', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:24:39.866573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:24:39.867077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:24:39.867622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:24:39.868051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:24:39.868479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:24:39.877774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:39.878261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:39.878707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:39.879984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:39.880643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:39.881126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:39.898845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Getting quotas for project 73d7ccfabbb44b5d8b1aa1253b2048e1. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:24:39.904171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Getting quotas for user d159c11f06c4482ebf4e600305bfe0ee and project 73d7ccfabbb44b5d8b1aa1253b2048e1. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:24:39.916699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:24:39.919045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:24:39.973806 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [e8f8a3fb-35ec-40da-86d5-324eb17c0caa] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:24:40.986043 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [e8f8a3fb-35ec-40da-86d5-324eb17c0caa] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 37426. Oct 15 21:24:40.990203 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47f82e81-91b6-44d6-a445-078216725d61 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.540508 Oct 15 21:24:40.990922 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3765/5287] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:24:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 1542 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:24:41.002512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6da873b7-00e3-457d-9914-49d4cd228a6a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:41.003213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6da873b7-00e3-457d-9914-49d4cd228a6a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:41.066256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6da873b7-00e3-457d-9914-49d4cd228a6a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:41.074987 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6da873b7-00e3-457d-9914-49d4cd228a6a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1219 microversion: 2.1 time: 0.076414 Oct 15 21:24:41.075676 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1523/5288] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:40 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1219 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:42.101064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:42.102605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:42.111458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:42.112855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:42.113522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:42.114656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:42.116192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:42.116844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:42.190852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:42.199629 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87f28619-f212-48f8-b724-79a7f1abdb1c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1279 microversion: 2.1 time: 0.104621 Oct 15 21:24:42.200497 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3766/5289] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:42 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1279 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:43.224654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:43.225466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:43.229125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:43.235203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:43.235693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:43.236129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:43.236833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:43.237320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:43.237745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:43.460536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:43.469078 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7e25d3d2-6c69-4e01-86e8-ed926bcab5ba tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.249231 Oct 15 21:24:43.469659 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1524/5290] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:43 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:43.749354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d49b8a20-4217-4bb3-9e3d-054d4692df2d req-140aa0cd-b988-471d-ac6d-8709d8d32486 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8d41dd49-2e79-410b-82cd-1c59be07093e", "tag": "4c8af1c9-bda4-4f36-9a8a-272f53f19356"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:43.759853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d49b8a20-4217-4bb3-9e3d-054d4692df2d req-140aa0cd-b988-471d-ac6d-8709d8d32486 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:43.761553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d49b8a20-4217-4bb3-9e3d-054d4692df2d req-140aa0cd-b988-471d-ac6d-8709d8d32486 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:43.761997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d49b8a20-4217-4bb3-9e3d-054d4692df2d req-140aa0cd-b988-471d-ac6d-8709d8d32486 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:43.773821 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d49b8a20-4217-4bb3-9e3d-054d4692df2d req-140aa0cd-b988-471d-ac6d-8709d8d32486 service nova] Creating event network-changed:4c8af1c9-bda4-4f36-9a8a-272f53f19356 for instance 8d41dd49-2e79-410b-82cd-1c59be07093e on nch2-master-1 Oct 15 21:24:43.779093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d49b8a20-4217-4bb3-9e3d-054d4692df2d req-140aa0cd-b988-471d-ac6d-8709d8d32486 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.172761 Oct 15 21:24:43.779695 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3767/5291] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:24:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:24:44.185246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:44.185863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:44.193284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.193829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.194269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.194973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.195428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.195856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.329158 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e617e9c3-89ef-4369-b7e7-7f85b8c378ef tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1831 microversion: 2.1 time: 0.147599 Oct 15 21:24:44.329790 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1525/5292] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:44 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1831 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:44.354367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:44.355067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:44.361872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.362530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.363010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.365149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.365613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.366042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.494555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:44.495170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:44.497158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:44.501214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.501702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.502137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.502845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.503296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.503876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.505311 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c48b5a7a-0235-4434-952e-e16321f2b0eb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1680 microversion: 2.1 time: 0.155378 Oct 15 21:24:44.505892 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3768/5293] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:44 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:44.601933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:44.604443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:44.610462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.610945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.611380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.612071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.612520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.612946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.637741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:44.646706 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c2676ba3-d1b0-4eb0-af37-4612907b71c3 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.156018 Oct 15 21:24:44.647279 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1526/5294] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:44 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:44.667066 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fdad97b-f7c1-4bcc-a685-d948d7b3d6e5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action" status: 200 len: 32638 microversion: 2.1 time: 0.069434 Oct 15 21:24:44.667714 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3769/5295] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:24:44 2022] POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action => generated 32638 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 21:24:44.818681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:44.819185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:44.825746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.826246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.826689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.827401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.827858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.828293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.856067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '4641b6af6bf940439be4f38054c6f6fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4405aa20ee8a4987878be13acb9b0029', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:44.856665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 8449b4ee-3824-4555-9650-19489f33897b] flavor_id is None. Assuming migration. {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4196}} Oct 15 21:24:44.857207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:24:44.966066 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dd92e019-6b02-4e09-9d6f-e3e8dde9c6d7 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action" status: 202 len: 0 microversion: 2.1 time: 0.151793 Oct 15 21:24:44.966784 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1527/5296] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:24:44 2022] POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:24:44.976039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:44.976752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:44.984111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.984605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.985078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:44.985797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:44.986261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:44.986699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:45.131091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:45.140085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d9a5c84c-cf2f-4e23-bdf7-7f8084eb970c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1689 microversion: 2.1 time: 0.168841 Oct 15 21:24:45.140752 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3770/5297] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:44 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1689 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:45.671997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:45.672699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:45.680882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:45.681631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:45.681985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:45.682896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:45.683569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:45.684006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:45.814225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:45.822135 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-39a3b02e-b956-4761-9cc8-3360a5c9b031 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.155992 Oct 15 21:24:45.822712 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1528/5298] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:45 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:46.165387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:46.167008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:46.174718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:46.175385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:46.175881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:46.177205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:46.178633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:46.179077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:46.398497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:46.406416 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d170c2d4-3f67-4426-a1f9-ac38dc9e6d4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1689 microversion: 2.1 time: 0.245681 Oct 15 21:24:46.407046 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3771/5299] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:46 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1689 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:46.845234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:46.845756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:46.852920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:46.853442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:46.853880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:46.854576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:46.855030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:46.855458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:47.056697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:47.064607 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-88785247-3acd-4f2c-9829-6f07405056aa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.223507 Oct 15 21:24:47.065208 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1529/5300] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:46 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:47.430898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:47.431515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:47.438812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:47.439294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:47.439742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:47.440437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:47.440894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:47.441352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:47.573949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:47.581720 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43005914-5eef-4f85-8429-336c0c4a6f41 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.155286 Oct 15 21:24:47.582294 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3772/5301] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:47 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:48.088339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:48.088961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:48.096083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:48.096575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:48.097022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:48.097748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:48.098215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:48.098652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:48.231236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:48.239044 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d3031796-f641-49e2-a949-54816072befa tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.154667 Oct 15 21:24:48.240317 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1530/5302] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:48 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:48.606408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:48.607547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:48.614222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:48.614921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:48.615436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:48.616138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:48.616597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:48.617024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:48.749178 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:48.757679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b59d26f-2f98-4c5c-99bf-0f29e7e46874 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.155455 Oct 15 21:24:48.758241 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3773/5303] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:48 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:49.262996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:49.263623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:49.270547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:49.271137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:49.271573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:49.272282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:49.272750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:49.273227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:49.407508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:49.415446 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c38e4202-1878-441c-a9bc-55539bbfc19c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.156428 Oct 15 21:24:49.416021 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1531/5304] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:49 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:49.783293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:49.783980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:49.791475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:49.791966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:49.792408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:49.793132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:49.793596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:49.794026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:49.889266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:49.912131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:50.240828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:50.248716 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c9a576f6-d0d6-40cb-acef-0db8f9de94a5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.469416 Oct 15 21:24:50.249328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3774/5305] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:49 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:50.438859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:50.439401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:50.446409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:50.446899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:50.447338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:50.448036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:50.448497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:50.448929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:50.577537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:50.585822 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e12c24db-e6ee-49cb-aa67-d6eda0a0406f tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.150827 Oct 15 21:24:50.586486 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1532/5306] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:50 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:50.668910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c8d725be-1187-4b6f-b4ed-8ae1af11f86e req-07c68663-b1f2-4ba3-9938-27a182d5dabd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8d41dd49-2e79-410b-82cd-1c59be07093e", "name": "network-vif-plugged", "status": "completed", "tag": "4c8af1c9-bda4-4f36-9a8a-272f53f19356"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:50.683843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c8d725be-1187-4b6f-b4ed-8ae1af11f86e req-07c68663-b1f2-4ba3-9938-27a182d5dabd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:50.684331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c8d725be-1187-4b6f-b4ed-8ae1af11f86e req-07c68663-b1f2-4ba3-9938-27a182d5dabd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:50.684769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c8d725be-1187-4b6f-b4ed-8ae1af11f86e req-07c68663-b1f2-4ba3-9938-27a182d5dabd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:50.699591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c8d725be-1187-4b6f-b4ed-8ae1af11f86e req-07c68663-b1f2-4ba3-9938-27a182d5dabd service nova] Creating event network-vif-plugged:4c8af1c9-bda4-4f36-9a8a-272f53f19356 for instance 8d41dd49-2e79-410b-82cd-1c59be07093e on nch2-master-1 Oct 15 21:24:50.704552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c8d725be-1187-4b6f-b4ed-8ae1af11f86e req-07c68663-b1f2-4ba3-9938-27a182d5dabd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.117009 Oct 15 21:24:50.705172 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3775/5307] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:24:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:24:51.272847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:51.273486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:51.275369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:51.280759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:51.281278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:51.281716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:51.282406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:51.282857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:51.283316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:51.415553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:51.423293 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-30ce0f5c-9a78-4c5c-8bcc-5c201892eea5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.155129 Oct 15 21:24:51.423853 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1533/5308] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:51 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:51.611543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:51.612237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:51.619459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:51.619941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:51.620379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:51.621091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:51.621549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:51.621973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:51.757669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:51.765490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d013214f-550e-4092-95f1-1c2437405dca tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1319 microversion: 2.1 time: 0.159009 Oct 15 21:24:51.766061 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3776/5309] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:51 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:52.447682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:52.448234 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:52.455110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:52.455598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:52.456041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:52.456735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:52.457227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:52.457659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:52.590171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:52.598007 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d48a4cf7-f0fa-4152-9a79-628e347043ef tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.155136 Oct 15 21:24:52.598578 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1534/5310] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:52 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:52.789204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:52.789739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:52.796792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:52.797312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:52.797750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:52.798436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:52.798893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:52.799317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:52.931490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:52.939304 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4fb743d-9cb6-4b56-a467-d7d1faf8b0ee tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1508 microversion: 2.1 time: 0.154236 Oct 15 21:24:52.939901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3777/5311] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:52 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1508 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:53.621810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:53.622871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:53.630483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:53.631061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:53.631571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:53.632358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:53.632892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:53.633360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:53.765975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:53.774464 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ffcf1d2-1bdf-411e-be71-a115ef072ba6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.156605 Oct 15 21:24:53.775036 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1535/5312] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:53 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:54.431771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6ffb766d-75cc-49b7-af00-612bf703ee80", "device": "/dev/sdb"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:24:54.434531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:54.441479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:54.441963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:54.442400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:54.443750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:54.444209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:54.444635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:54.687452 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2830c71d-4827-4437-ab19-853f5c4709f7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "POST /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.260655 Oct 15 21:24:54.688126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3778/5313] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:24:54 2022] POST /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e/os-volume_attachments => generated 194 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:24:54.801933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:54.802555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:54.810826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:54.811318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:54.811761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:54.812471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:54.812939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:54.813396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:54.949778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:54.957939 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f53842f0-1892-4f9b-af6d-1db1f107bdab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.159964 Oct 15 21:24:54.958704 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1536/5314] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:54 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:55.983072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:55.983772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:55.990098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:55.991110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:55.991555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:55.992264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:55.992723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:55.993186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:56.130615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:56.139196 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24c21ba7-dffc-47a1-a7ef-85847fdb1726 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.161686 Oct 15 21:24:56.139770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3779/5315] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:55 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:24:57.162585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:57.163212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:57.170236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:57.170732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:57.171179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:57.171888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:57.172353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:57.172792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:57.305784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:57.313562 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f7f68265-2f3b-4de2-91e2-9296e121a7d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.154921 Oct 15 21:24:57.314127 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1537/5316] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:57 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:58.338586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:58.339204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:58.346199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:58.346691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:58.347155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:58.347881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:58.348348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:58.348779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:58.485854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:58.493902 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a3069fc-1541-4674-9eaa-f72e1ef72bab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.159955 Oct 15 21:24:58.494476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3780/5317] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:58 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:24:59.518979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:24:59.519534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:24:59.527045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:59.527531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:59.527970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:59.528660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:24:59.529150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:24:59.529578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:24:59.663470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:24:59.667596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:24:59.672657 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-af29019f-cf28-4f9b-b3da-b0cbd67c2ddb tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.157843 Oct 15 21:24:59.673255 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1538/5318] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:24:59 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:00.695758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:00.696398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:00.703600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:00.704086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:00.704528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:00.705259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:00.705722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:00.706154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:00.840423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:00.848246 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-596bb15a-a979-4bb2-8cb6-e809c0af24d0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.156472 Oct 15 21:25:00.848820 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3781/5319] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:00 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:01.872464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:01.873097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:01.881726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:01.882212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:01.882681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:01.883375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:01.884580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:01.885010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:02.021412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:02.029387 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dd331cf8-e4ae-4da5-ae79-9d0fc91f5e68 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.161905 Oct 15 21:25:02.029962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1539/5320] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:01 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:03.053176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:03.053817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:03.060689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:03.061223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:03.061667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:03.062367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:03.062826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:03.063258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:03.081900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:03.196075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:03.205136 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c0637bfb-9189-4255-8d21-6624fd13c9e7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1694 microversion: 2.1 time: 0.156103 Oct 15 21:25:03.205917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3782/5321] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:03 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1694 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:04.230148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:04.230770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:04.233593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=41 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:04.239815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:04.240312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:04.240760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:04.241547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:04.242012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:04.242447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:04.486151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:04.494175 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dbdc85c0-98a5-4a43-a580-9fcbfddd9086 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1693 microversion: 2.1 time: 0.268080 Oct 15 21:25:04.495061 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1540/5322] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:04 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1693 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:04.581808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "name": "network-vif-unplugged", "status": "completed", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:04.593375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:04.593867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:04.594312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:04.606785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] Creating event network-vif-unplugged:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch2-master-1 Oct 15 21:25:04.612267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] Instance 8449b4ee-3824-4555-9650-19489f33897b is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:25:04.623579 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-44f32311-8261-4318-85bb-e0953b8f01be req-9552433c-14af-4975-ad8c-b0d8dfceb204 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045504 Oct 15 21:25:04.624336 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3783/5323] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:25:04 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:25:05.518326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:05.518908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:05.526507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:05.527000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:05.527449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:05.528161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:05.528626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:05.529093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:05.672264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:05.680246 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-150cd6fb-ebc4-42bb-abfa-0be2fbddb824 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1693 microversion: 2.1 time: 0.165892 Oct 15 21:25:05.680839 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1541/5324] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:05 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1693 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:06.665113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:06.674567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:06.675045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:06.675476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:06.686830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] Creating event network-changed:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch2-master-1 Oct 15 21:25:06.692065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] Instance 8449b4ee-3824-4555-9650-19489f33897b is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:25:06.703445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9ef4bdf3-a5a6-4ac4-a44b-e5d91f4745fa req-da496e5b-5f74-4303-bd0d-0dfbfb79714a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043298 Oct 15 21:25:06.703922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:06.704358 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3784/5325] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:25:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:25:06.704611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:06.711795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:06.712290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:06.713577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:06.714449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:06.714919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:06.715355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:06.921397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:06.929220 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-16de3f90-6cfb-4b7c-a05e-4916361262e6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1691 microversion: 2.1 time: 0.229073 Oct 15 21:25:06.929800 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1542/5326] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:06 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1691 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:25:07.954735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:07.955444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:07.962003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:07.962621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:07.963082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:07.963786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:07.964246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:07.964673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:08.140985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:08.149104 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dcc2156c-7eea-45cc-a18c-bf2a2d3aaa94 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1691 microversion: 2.1 time: 0.199434 Oct 15 21:25:08.149694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3785/5327] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:07 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1691 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:09.173702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:09.174327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:09.181919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:09.182409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:09.182848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:09.183554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:09.184015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:09.184446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:09.316099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:09.324704 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-99fc25c5-9308-4a91-b1ef-98da81021c65 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1688 microversion: 2.1 time: 0.154910 Oct 15 21:25:09.325301 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1543/5328] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:09 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1688 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:09.347206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:09.347758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:09.354929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:09.355421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:09.355865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:09.356572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:09.357083 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:09.357520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:09.472395 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c908f1b1-6773-4225-9d7b-5ec99bf43342 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action" status: 202 len: 0 microversion: 2.1 time: 0.128768 Oct 15 21:25:09.473319 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3786/5329] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:25:09 2022] POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:25:09.483171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:09.483791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:09.489809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:09.490976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:09.491487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:09.492404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:09.492969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:09.493960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:09.692626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:09.701439 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df62eef8-1561-40f6-8f79-c33916de3541 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1687 microversion: 2.1 time: 0.223579 Oct 15 21:25:09.702022 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1544/5330] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:09 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1687 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:10.725915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:10.726631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:10.730264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:10.737908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:10.738242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:10.738682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:10.739438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:10.739818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:10.740243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:10.878539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:10.888289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d8348a80-8e9a-4f1d-9e24-adedd80374dd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1687 microversion: 2.1 time: 0.166316 Oct 15 21:25:10.889096 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3787/5331] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:10 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1687 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:11.914759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:11.916688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:11.924974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:11.925552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:11.925991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:11.926778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:11.927251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:11.927687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:12.074141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:12.082491 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f53a56e-6d3d-4dac-8231-228746d80e82 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1687 microversion: 2.1 time: 0.173528 Oct 15 21:25:12.083088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1545/5332] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:11 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1687 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:13.112069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:13.112934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:13.120922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:13.121434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:13.121875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:13.122587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:13.123046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:13.123469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:13.510135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:13.518581 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4100e10-85e5-4e11-8839-54ff3a46406b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1687 microversion: 2.1 time: 0.410563 Oct 15 21:25:13.533062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3788/5333] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:13 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1687 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:13.653199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:13.664005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:13.664490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:13.664924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:13.677007 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] Creating event network-changed:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:25:13.684169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] Instance 8449b4ee-3824-4555-9650-19489f33897b is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:25:13.695915 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d1af8786-7d41-4c03-875d-8af492f03cfd req-9dd21d13-5eba-4db4-a94f-ddf5f4d4f4fe service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046874 Oct 15 21:25:13.697299 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1546/5334] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:25:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:25:14.542969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:14.544324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:14.551924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:14.552653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:14.553331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:14.554374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:14.555198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:14.556136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:14.688554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:14.697162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1bae3605-c9a0-4ee0-a9ca-95842181309f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1687 microversion: 2.1 time: 0.157500 Oct 15 21:25:14.697757 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3789/5335] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:14 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1687 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:15.722111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:15.722826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:15.729597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:15.730089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:15.730534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:15.731239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:15.731706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:15.732146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:15.924743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:15.934316 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a8c8f605-1586-495a-8a5c-b1d5428a1094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1687 microversion: 2.1 time: 0.217687 Oct 15 21:25:15.934900 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1547/5336] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:15 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1687 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:16.958002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:16.958623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:16.965476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:16.965960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:16.966401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:16.967092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:16.967549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:16.968004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:17.098910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:17.107625 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-63bc9069-b166-44b7-ad7f-01baa82721a4 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1680 microversion: 2.1 time: 0.153547 Oct 15 21:25:17.108200 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3790/5337] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:16 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:19.769876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ed296b8a-cd52-435d-aa5f-23bea9ea375b req-03c6c9fb-574e-407e-b67b-e62748cf12d9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "name": "network-vif-plugged", "status": "completed", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:19.780669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ed296b8a-cd52-435d-aa5f-23bea9ea375b req-03c6c9fb-574e-407e-b67b-e62748cf12d9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:19.781399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ed296b8a-cd52-435d-aa5f-23bea9ea375b req-03c6c9fb-574e-407e-b67b-e62748cf12d9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:19.781827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ed296b8a-cd52-435d-aa5f-23bea9ea375b req-03c6c9fb-574e-407e-b67b-e62748cf12d9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:19.796615 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ed296b8a-cd52-435d-aa5f-23bea9ea375b req-03c6c9fb-574e-407e-b67b-e62748cf12d9 service nova] Creating event network-vif-plugged:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:25:19.802232 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ed296b8a-cd52-435d-aa5f-23bea9ea375b req-03c6c9fb-574e-407e-b67b-e62748cf12d9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035523 Oct 15 21:25:19.802523 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1548/5338] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:25:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:25:41.307563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:41.309830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:41.313191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:41.314064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:41.320365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:41.321296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:41.321753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:41.322461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:41.322929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:41.323363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:41.380863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07e341bd-39c9-4bc2-996e-128e3713b955 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action" status: 200 len: 63216 microversion: 2.1 time: 0.077296 Oct 15 21:25:41.381574 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3791/5339] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:25:41 2022] POST /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b/action => generated 63216 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:25:47.860157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=17 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:47.860913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=54 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:25:47.865206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:47.865972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:47.873743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:47.874229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:47.874669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:47.875368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:47.875826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:47.876254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:48.064729 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e65ab7cd-259f-4edb-b12d-659e81a32679 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1831 microversion: 2.1 time: 0.206771 Oct 15 21:25:48.065337 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1549/5340] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:47 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1831 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:48.912031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:48.912616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:48.914144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-45c647f9-bca6-4949-9e85-da014eff30f0 req-62c483fd-29f7-41d5-a72f-11212b73bd9e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:48.920624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:48.921376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:48.921987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:48.922871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:48.923331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:48.923761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:48.925256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-45c647f9-bca6-4949-9e85-da014eff30f0 req-62c483fd-29f7-41d5-a72f-11212b73bd9e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:48.925749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-45c647f9-bca6-4949-9e85-da014eff30f0 req-62c483fd-29f7-41d5-a72f-11212b73bd9e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:48.926184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-45c647f9-bca6-4949-9e85-da014eff30f0 req-62c483fd-29f7-41d5-a72f-11212b73bd9e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:48.937963 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-45c647f9-bca6-4949-9e85-da014eff30f0 req-62c483fd-29f7-41d5-a72f-11212b73bd9e service nova] Creating event network-changed:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:25:48.943168 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-45c647f9-bca6-4949-9e85-da014eff30f0 req-62c483fd-29f7-41d5-a72f-11212b73bd9e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033589 Oct 15 21:25:48.943948 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1550/5341] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:25:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:25:48.951447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:25:49.050379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1cb78130-44ed-43e7-b304-ef0fb27ac663 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 204 len: 0 microversion: 2.1 time: 0.142559 Oct 15 21:25:49.054707 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3792/5342] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:25:48 2022] DELETE /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:25:49.060472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:49.061503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:49.068306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:49.069073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:49.069762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:49.070682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:49.071497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:49.071997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:49.227343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:49.235224 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ae8ea30e-9ea8-4000-acf1-e5ab1f9a73cc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1686 microversion: 2.1 time: 0.179044 Oct 15 21:25:49.236012 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1551/5343] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:49 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1686 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:50.259962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:50.260670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:50.268417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:50.268914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:50.269388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:50.270092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:50.270553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:50.270989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:50.872009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:25:50.879971 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-28455555-96dd-41f7-9c9a-8288c3a41b2c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 200 len: 1686 microversion: 2.1 time: 0.625049 Oct 15 21:25:50.880554 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3793/5344] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:50 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 1686 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:25:51.243466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-3f7f7518-47e1-4acf-966b-8a86a74828c9 req-a659d3cf-bd5a-41d2-9101-38c9ee325c7a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8449b4ee-3824-4555-9650-19489f33897b", "name": "network-vif-deleted", "tag": "14c87224-221a-49dc-a26a-1ac604e6ea6f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:51.257377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3f7f7518-47e1-4acf-966b-8a86a74828c9 req-a659d3cf-bd5a-41d2-9101-38c9ee325c7a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:51.257990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3f7f7518-47e1-4acf-966b-8a86a74828c9 req-a659d3cf-bd5a-41d2-9101-38c9ee325c7a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:51.258426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3f7f7518-47e1-4acf-966b-8a86a74828c9 req-a659d3cf-bd5a-41d2-9101-38c9ee325c7a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:51.275022 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-3f7f7518-47e1-4acf-966b-8a86a74828c9 req-a659d3cf-bd5a-41d2-9101-38c9ee325c7a service nova] Creating event network-vif-deleted:14c87224-221a-49dc-a26a-1ac604e6ea6f for instance 8449b4ee-3824-4555-9650-19489f33897b on nch1-master-1 Oct 15 21:25:51.279853 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-3f7f7518-47e1-4acf-966b-8a86a74828c9 req-a659d3cf-bd5a-41d2-9101-38c9ee325c7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040536 Oct 15 21:25:51.280452 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1552/5345] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:25:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:25:51.906662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:51.907302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8449b4ee-3824-4555-9650-19489f33897b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:25:51.914941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:51.915420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:51.915859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:51.916547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:25:51.917000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:25:51.917447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:25:51.931553 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] HTTP exception thrown: Instance 8449b4ee-3824-4555-9650-19489f33897b could not be found. Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Request method failure captured: Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b HTTP/1.1 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRbA0cLyHQrk49g1yDTx2mawYGe7wmuR6AIZfAxBzfijpxhir3KV-4WCV7R2Uxj7n0-Vd4Epp7EBbcjQo861nJwrUnuYBBc0VHGRvHVNbDWK0zr0_BWwaxNQHYbUerlTCJoIXcNVviBEtMXGQni285yOrYoEZhsCaxrPEmh_p7oayg Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a0d6261b46824acf958ef749425ae421 Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8449b4ee-3824-4555-9650-19489f33897b could not be found. Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8449b4ee-3824-4555-9650-19489f33897b'} Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8449b4ee-3824-4555-9650-19489f33897b could not be found. Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8449b4ee-3824-4555-9650-19489f33897b could not be found. Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:25:51.933238 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8449b4ee-3824-4555-9650-19489f33897b could not be found. Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Returning 404 to user: Instance 8449b4ee-3824-4555-9650-19489f33897b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80b1d03f-663b-44e8-a18a-c14e9376ac6a tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b" status: 404 len: 111 microversion: 2.1 time: 0.031955 Oct 15 21:25:51.937059 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3794/5346] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:25:51 2022] GET /compute/v2.1/servers/8449b4ee-3824-4555-9650-19489f33897b => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:25:55.029756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d924f0d-33ce-406d-ad56-bda4a25636b2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '.version_select at 0x7fe74f018940>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:25:55.052854 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2d924f0d-33ce-406d-ad56-bda4a25636b2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-62898679" status: 202 len: 0 microversion: 2.1 time: 0.027320 Oct 15 21:25:55.053983 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1553/5347] 10.222.0.16 () {58 vars in 1278 bytes} [Sat Oct 15 21:25:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-62898679 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:25:55.090326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cddc0b02-0a5b-4481-a151-43dc186d12a7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: .version_select at 0x7fe74e13e670>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-160637841", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIJ7iUpaTd1DhlK7FxRuACvzKUHj6OHoa4lDCVYvRUg1fe14xSmVuvzFp4PKLZgA16LgkFqmVDrgQZ0jF/ssDbIHq5bpFk2cKkaM5eAkTOwZZxKyV7W4DYo4XutNIe5T5A=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:25:55.107326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cddc0b02-0a5b-4481-a151-43dc186d12a7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:25:55.113364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cddc0b02-0a5b-4481-a151-43dc186d12a7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:25:55.136685 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cddc0b02-0a5b-4481-a151-43dc186d12a7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.050163 Oct 15 21:25:55.137238 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3795/5348] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:25:55 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:26:00.951082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1584347895", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7616de16-8201-42d3-88f3-5568b69b710d"}], "key_name": "tempest-TestNetworkAdvancedServerOps-160637841", "security_groups": [{"name": "tempest-secgroup-smoke-848345723"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:01.094274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] validate_networks() for [('7616de16-8201-42d3-88f3-5568b69b710d', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:26:01.263424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:01.542025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:26:01.542630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:26:01.543819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:26:01.544389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:26:01.544966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:26:01.555871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:01.557545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:01.558140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:01.559041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:01.559844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:01.560504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:01.588079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:26:01.596737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:26:01.611648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:26:01.614440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:26:01.674459 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7d7accff-7f98-4b34-a38d-34d759f62fa5 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.727509 Oct 15 21:26:01.675338 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1554/5349] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:26:00 2022] POST /compute/v2.1/servers => generated 405 bytes in 729 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:26:01.688265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eccc688d-5912-40be-b2f0-563bb82fc76d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:01.689002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eccc688d-5912-40be-b2f0-563bb82fc76d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:01.791538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eccc688d-5912-40be-b2f0-563bb82fc76d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:01.799700 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eccc688d-5912-40be-b2f0-563bb82fc76d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1266 microversion: 2.1 time: 0.116164 Oct 15 21:26:01.800688 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3796/5350] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:01 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1266 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:02.825853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:02.827735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:02.837003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:02.837696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:02.838437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:02.839417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:02.840154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:02.840608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:02.925320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:02.934428 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b0f27752-b8ee-429e-8579-6c06d447a8ac tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1336 microversion: 2.1 time: 0.114148 Oct 15 21:26:02.935017 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1555/5351] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:02 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1336 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:03.958108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:03.958936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:03.961625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:03.969189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:03.969691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:03.970139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:03.970868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:03.971336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:03.971775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:04.213887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:04.221756 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04edcb6c-68a5-45e6-a119-f69828dd96d2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.267644 Oct 15 21:26:04.222346 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3797/5352] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:03 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:04.415855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d69f240d-ddb7-47e9-bbff-fdc710414c2b req-1e673c7c-b302-4ccf-b033-42639df177ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:04.442461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d69f240d-ddb7-47e9-bbff-fdc710414c2b req-1e673c7c-b302-4ccf-b033-42639df177ba service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:04.443389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d69f240d-ddb7-47e9-bbff-fdc710414c2b req-1e673c7c-b302-4ccf-b033-42639df177ba service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:04.443942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d69f240d-ddb7-47e9-bbff-fdc710414c2b req-1e673c7c-b302-4ccf-b033-42639df177ba service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:04.456253 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d69f240d-ddb7-47e9-bbff-fdc710414c2b req-1e673c7c-b302-4ccf-b033-42639df177ba service nova] Creating event network-changed:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch1-master-1 Oct 15 21:26:04.461959 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d69f240d-ddb7-47e9-bbff-fdc710414c2b req-1e673c7c-b302-4ccf-b033-42639df177ba service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055876 Oct 15 21:26:04.462700 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1556/5353] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:26:05.246851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:05.247526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:05.254407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:05.254889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:05.255327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:05.256091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:05.256659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:05.257007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:05.393647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:05.402405 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6060dcb9-8542-4268-b68d-dd651a6b4b96 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.159374 Oct 15 21:26:05.402990 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3798/5354] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:05 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:06.428219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:06.428978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:06.435528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:06.436058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:06.436512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:06.437261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:06.437738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:06.438166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:06.578278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:06.586083 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28d0096c-e6fa-4a75-91a4-b48d018010c9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.163135 Oct 15 21:26:06.588682 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1557/5355] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:06 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:07.610209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:07.610751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:07.618503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:07.618995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:07.619440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:07.620144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:07.620630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:07.621089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:07.754171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:07.762929 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f7e0a05-7854-4400-b545-1d5200566838 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.156748 Oct 15 21:26:07.763501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3799/5356] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:07 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:08.785614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:08.786147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:08.793595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:08.794082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:08.795518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:08.796217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:08.796676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:08.797136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:08.933103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:08.940751 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec5938f5-2dbd-482e-9596-86a62a545fd3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.159184 Oct 15 21:26:08.941352 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1558/5357] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:08 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:09.964395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:09.965013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:09.972286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:09.972781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:09.973259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:09.973979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:09.974450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:09.975516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:10.114162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:10.122239 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-842ae2d7-20dd-4072-99d6-50f010bd7bd0 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.161689 Oct 15 21:26:10.123024 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3800/5358] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:09 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:10.126294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5a665c1f-2e8b-4512-8428-d135b2679bd9 req-928f2e78-d07d-454f-9408-81edc803342d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "name": "network-vif-plugged", "status": "completed", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:10.137528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a665c1f-2e8b-4512-8428-d135b2679bd9 req-928f2e78-d07d-454f-9408-81edc803342d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:10.138193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a665c1f-2e8b-4512-8428-d135b2679bd9 req-928f2e78-d07d-454f-9408-81edc803342d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:10.138772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a665c1f-2e8b-4512-8428-d135b2679bd9 req-928f2e78-d07d-454f-9408-81edc803342d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:10.151289 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5a665c1f-2e8b-4512-8428-d135b2679bd9 req-928f2e78-d07d-454f-9408-81edc803342d service nova] Creating event network-vif-plugged:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch1-master-1 Oct 15 21:26:10.156264 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5a665c1f-2e8b-4512-8428-d135b2679bd9 req-928f2e78-d07d-454f-9408-81edc803342d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034067 Oct 15 21:26:10.156861 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1559/5359] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:26:11.146731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:11.147452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:11.154418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:11.154910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:11.155357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:11.156063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:11.156530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:11.156968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:11.294381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:11.302158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-650adc19-95f6-4aa7-b790-6666f5f6f511 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1391 microversion: 2.1 time: 0.159425 Oct 15 21:26:11.303389 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3801/5360] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:11 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1391 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:12.324361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:12.325155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:12.329478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:12.330184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:12.337554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:12.338043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:12.338488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:12.339186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:12.339650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:12.340080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:12.478837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:12.486657 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87e86ace-49fe-4a89-8714-2da148cb5ebc tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1564 microversion: 2.1 time: 0.164138 Oct 15 21:26:12.487247 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1560/5361] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:12 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1564 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:12.509550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:12.510176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:12.512042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:12.513755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:12.519718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:12.520223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:12.520675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:12.521423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:12.521894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:12.522347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:12.657830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:12.666576 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2591f9ce-2c6e-40a3-b8e9-22b871d7bdba tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1564 microversion: 2.1 time: 0.160851 Oct 15 21:26:12.667162 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3802/5362] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:12 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1564 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:14.365787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-133d9ace-7ba3-483c-99ac-3d85bf9bb1e1 req-e1f7a198-37cb-45ed-86f1-f3eaab561c25 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:14.367785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:14.368665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:14.379729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:14.380235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:14.380679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:14.381435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:14.381930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:14.383016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:14.392310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-133d9ace-7ba3-483c-99ac-3d85bf9bb1e1 req-e1f7a198-37cb-45ed-86f1-f3eaab561c25 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:14.392310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-133d9ace-7ba3-483c-99ac-3d85bf9bb1e1 req-e1f7a198-37cb-45ed-86f1-f3eaab561c25 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:14.392310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-133d9ace-7ba3-483c-99ac-3d85bf9bb1e1 req-e1f7a198-37cb-45ed-86f1-f3eaab561c25 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:14.404591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-133d9ace-7ba3-483c-99ac-3d85bf9bb1e1 req-e1f7a198-37cb-45ed-86f1-f3eaab561c25 service nova] Creating event network-changed:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch1-master-1 Oct 15 21:26:14.409826 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-133d9ace-7ba3-483c-99ac-3d85bf9bb1e1 req-e1f7a198-37cb-45ed-86f1-f3eaab561c25 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050173 Oct 15 21:26:14.413195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3803/5363] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:26:14.620850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:14.630158 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3115f91e-0319-44cc-bb6b-42bf5ea194c2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1564 microversion: 2.1 time: 0.274114 Oct 15 21:26:14.630874 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1561/5364] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:14 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1564 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:35.641017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:35.641638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:35.643978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:35.644630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=38 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:35.650670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:35.651161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:35.651608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:35.652311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:35.652777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:35.654593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:35.738070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lazy-loading 'services' on Instance uuid 8d41dd49-2e79-410b-82cd-1c59be07093e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:26:35.808962 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83394fc4-b8aa-468f-845f-6b551d2fa354 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e/os-volume_attachments/6ffb766d-75cc-49b7-af00-612bf703ee80" status: 202 len: 0 microversion: 2.1 time: 0.172220 Oct 15 21:26:35.809592 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3804/5365] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:26:35 2022] DELETE /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e/os-volume_attachments/6ffb766d-75cc-49b7-af00-612bf703ee80 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:26:35.892740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=40,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:35.893515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=36 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:35.952572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:35.954985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:35.961012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:35.961533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:35.961980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:35.962757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:35.963228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:35.964290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:36.020521 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-254e0fc5-4e6a-47fb-80c4-81434763b9b7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action" status: 200 len: 24571 microversion: 2.1 time: 0.132621 Oct 15 21:26:36.021157 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1562/5366] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:26:35 2022] POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action => generated 24571 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:26:36.936196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:36.936793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:36.943108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:36.943596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:36.944036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:36.944738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:36.945242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:36.945681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:36.971302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:26:37.046828 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b27ff0c-b124-46b3-9fb9-82246359d0d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 204 len: 0 microversion: 2.1 time: 0.115898 Oct 15 21:26:37.047403 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3805/5367] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:26:36 2022] DELETE /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:26:37.056635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:37.057935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:37.065608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:37.066100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:37.066582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:37.067296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:37.067781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:37.068218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:37.208057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:37.216336 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65349bd1-c800-4efd-800b-a67f395376b2 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1514 microversion: 2.1 time: 0.164076 Oct 15 21:26:37.216897 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1563/5368] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:37 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1514 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:26:38.240882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:38.241620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:38.249236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:38.249756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:38.250205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:38.250911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:38.251374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:38.251808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:38.382565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:38.390603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-64200c42-9522-4912-8674-a1a53108e934 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1514 microversion: 2.1 time: 0.153927 Oct 15 21:26:38.391175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3806/5369] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:38 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1514 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:39.164915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-670609fb-0ccf-4bb0-a028-137e54702770 req-a05f8b9b-e8e4-44d8-85b2-605c2187645e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8d41dd49-2e79-410b-82cd-1c59be07093e", "name": "network-vif-deleted", "tag": "4c8af1c9-bda4-4f36-9a8a-272f53f19356"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:39.176587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-670609fb-0ccf-4bb0-a028-137e54702770 req-a05f8b9b-e8e4-44d8-85b2-605c2187645e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:39.176995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-670609fb-0ccf-4bb0-a028-137e54702770 req-a05f8b9b-e8e4-44d8-85b2-605c2187645e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:39.177458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-670609fb-0ccf-4bb0-a028-137e54702770 req-a05f8b9b-e8e4-44d8-85b2-605c2187645e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:39.188589 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-670609fb-0ccf-4bb0-a028-137e54702770 req-a05f8b9b-e8e4-44d8-85b2-605c2187645e service nova] Creating event network-vif-deleted:4c8af1c9-bda4-4f36-9a8a-272f53f19356 for instance 8d41dd49-2e79-410b-82cd-1c59be07093e on nch2-master-1 Oct 15 21:26:39.193640 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-670609fb-0ccf-4bb0-a028-137e54702770 req-a05f8b9b-e8e4-44d8-85b2-605c2187645e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032352 Oct 15 21:26:39.194243 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1564/5370] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:26:39.413884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:39.414531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:39.422272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:39.422763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:39.423205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:39.423906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:39.424365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:39.424803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:39.459279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:39.459695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:39.460141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:39.460587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lazy-loading 'fault' on Instance uuid 8d41dd49-2e79-410b-82cd-1c59be07093e {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:26:39.501839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:39.509958 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-88fa0e47-3c6f-4670-9403-982b2815e322 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 200 len: 1309 microversion: 2.1 time: 0.099953 Oct 15 21:26:39.510543 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3807/5371] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:39 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 1309 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:40.535604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:40.536313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: 8d41dd49-2e79-410b-82cd-1c59be07093e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:40.542657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:40.543142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:40.543582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:40.544413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:40.545025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:40.545480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:40.558933 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] HTTP exception thrown: Instance 8d41dd49-2e79-410b-82cd-1c59be07093e could not be found. Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Request method failure captured: Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e HTTP/1.1 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyUJeBQH6P-a9dnU9biwtBaxfL2qEmsNBiXdbdPgkTWZhQ3eI4DNXNVBpAHOLKsspEY3KP8dYGKYNKvBvan1HFWI1HDriGddTxuT6yzIB_aR5yXlnnGB8-O-OTqhDEYlsJl8E-5KlU4PnGwALGn7pHfX8JblCC4ehYltCY6nP9X-718 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 73d7ccfabbb44b5d8b1aa1253b2048e1 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-VolumesSnapshotTestJSON-153295047 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-VolumesSnapshotTestJSON-153295047 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 73d7ccfabbb44b5d8b1aa1253b2048e1 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-153295047 Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-VolumesSnapshotTestJSON-153295047-project Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: d159c11f06c4482ebf4e600305bfe0ee Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-VolumesSnapshotTestJSON-153295047-project Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 8d41dd49-2e79-410b-82cd-1c59be07093e could not be found. Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '8d41dd49-2e79-410b-82cd-1c59be07093e'} Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8d41dd49-2e79-410b-82cd-1c59be07093e could not be found. Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8d41dd49-2e79-410b-82cd-1c59be07093e could not be found. Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:26:40.561276 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8d41dd49-2e79-410b-82cd-1c59be07093e could not be found. Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Returning 404 to user: Instance 8d41dd49-2e79-410b-82cd-1c59be07093e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9e0ba2ba-4d72-429f-bf35-be746a72039a tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e" status: 404 len: 111 microversion: 2.1 time: 0.032342 Oct 15 21:26:40.565452 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1565/5372] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:40 2022] GET /compute/v2.1/servers/8d41dd49-2e79-410b-82cd-1c59be07093e => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:26:42.422107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:42.422719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:42.429917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:42.430402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:42.430839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:42.431528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:42.431985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:42.432407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:42.567296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:42.576337 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4e2202c-05c5-4d10-a8ec-7900b4919a32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1683 microversion: 2.1 time: 0.158046 Oct 15 21:26:42.576996 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3808/5373] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:42 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1683 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:42.677778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:42.680248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:42.689545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:42.690110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:42.690549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:42.691265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:42.691732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:42.692170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:42.744655 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8615a727-0dff-4552-aa38-8ef665d2b4e8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action" status: 200 len: 32643 microversion: 2.1 time: 0.071084 Oct 15 21:26:42.745316 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1566/5374] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:26:42 2022] POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action => generated 32643 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:26:43.048607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:43.049246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:43.055222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:43.055710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:43.056154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:43.056848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:43.057350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:43.057781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:43.212855 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-edf36caf-0a85-468b-8fa5-0ceeb3911033 tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1834 microversion: 2.1 time: 0.284407 Oct 15 21:26:43.213779 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3809/5375] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:42 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1834 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:43.244134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": null, "block_migration": true, "disk_over_commit": false}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:43.246767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:43.254376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:43.254874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:43.255320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:43.256033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:43.256498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:43.257094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:43.285645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lazy-loading 'flavor' on Instance uuid 29605bae-becd-4e1b-96bc-84aee0ba8be2 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:26:43.298259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Going to try to live migrate instance to another host {{(pid=138689) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Oct 15 21:26:44.251040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-696646191", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "c44a3d28-5fb1-4d4f-a5d6-b74bdb233a9e"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:44.305587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] validate_networks() for [('c44a3d28-5fb1-4d4f-a5d6-b74bdb233a9e', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:26:44.727156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:26:44.727589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:26:44.728070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:26:44.728488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:26:44.729285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:26:44.738825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:44.739320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:44.739763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:44.740500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:44.740962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:44.741421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:44.759573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Getting quotas for project 73d7ccfabbb44b5d8b1aa1253b2048e1. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:26:44.765967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Getting quotas for user d159c11f06c4482ebf4e600305bfe0ee and project 73d7ccfabbb44b5d8b1aa1253b2048e1. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:26:44.779840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:26:44.782222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:26:44.821521 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28787c6e-5310-4c92-8cfd-93b77bd1d26a tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action" status: 202 len: 0 microversion: 2.1 time: 1.581388 Oct 15 21:26:44.825069 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1567/5376] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:26:43 2022] POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action => generated 0 bytes in 1583 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:26:44.832024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:44.832758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:44.833240 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0132a174-fa07-42a8-b594-b66eb2bd4043 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.586119 Oct 15 21:26:44.834309 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3810/5377] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:26:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 589 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:26:44.852743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9d21b2c-21cb-462d-b16c-aec2c8a75884 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:44.853385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:44.854137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:44.854280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9d21b2c-21cb-462d-b16c-aec2c8a75884 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:44.854584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:44.855354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:44.855824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:44.857267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:44.962747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9d21b2c-21cb-462d-b16c-aec2c8a75884 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:44.971672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9d21b2c-21cb-462d-b16c-aec2c8a75884 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1218 microversion: 2.1 time: 0.125095 Oct 15 21:26:44.973649 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3811/5378] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:44 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1218 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:45.409989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:45.417873 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc562c5b-71b7-4bd9-93e4-93538ea65a64 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.589921 Oct 15 21:26:45.418458 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1568/5379] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:44 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:26:45.995679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:45.996988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:46.004710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:46.005257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:46.005726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:46.006427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:46.007064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:46.007504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:46.122084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:46.131160 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02a8a99d-36b7-4608-b0a8-84081dcb7387 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1276 microversion: 2.1 time: 0.139559 Oct 15 21:26:46.132037 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3812/5380] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:45 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1276 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:46.442836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:46.443482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:46.451839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:46.452336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:46.452786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:46.453536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:46.454002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:46.454439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:46.724246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:46.733116 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3bdf1b6c-6fb7-4571-9892-028bc1e65f67 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.294181 Oct 15 21:26:46.733773 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1569/5381] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:46 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:47.041567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-219d6ea2-944c-468c-9b35-e9b07a340e44 req-69320e26-d037-41fe-bebd-10121838dc1b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "name": "network-vif-unplugged", "status": "completed", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:47.055742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-219d6ea2-944c-468c-9b35-e9b07a340e44 req-69320e26-d037-41fe-bebd-10121838dc1b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:47.056400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-219d6ea2-944c-468c-9b35-e9b07a340e44 req-69320e26-d037-41fe-bebd-10121838dc1b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:47.056953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-219d6ea2-944c-468c-9b35-e9b07a340e44 req-69320e26-d037-41fe-bebd-10121838dc1b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:47.070289 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-219d6ea2-944c-468c-9b35-e9b07a340e44 req-69320e26-d037-41fe-bebd-10121838dc1b service nova] Creating event network-vif-unplugged:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch1-master-1 Oct 15 21:26:47.076471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-219d6ea2-944c-468c-9b35-e9b07a340e44 req-69320e26-d037-41fe-bebd-10121838dc1b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.038565 Oct 15 21:26:47.078693 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3813/5382] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:26:47.154795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:47.155759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:47.162059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:47.162617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:47.163123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:47.163893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:47.164423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:47.164937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:47.453302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:47.461005 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a6193674-ac37-4d66-811f-8d0ff8dd5b23 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.310192 Oct 15 21:26:47.461607 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1570/5383] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:47 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:47.758107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:47.758975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:47.765598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:47.766091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:47.766532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:47.767227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:47.767687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:47.768114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:48.036656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:48.045379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0ce0fab-3ec9-437e-b947-23877ba5d384 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.290973 Oct 15 21:26:48.046245 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3814/5384] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:47 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:48.484016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:48.484648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:48.491581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:48.492076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:48.492515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:48.493247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:48.493712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:48.494143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:48.707022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:48.715054 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-80841cd2-cc0a-4933-967e-ef4e8878fb3e tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.234930 Oct 15 21:26:48.715593 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1571/5385] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:48 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:49.069705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:49.070216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:49.077514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:49.077995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:49.078432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:49.079125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:49.079582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:49.080009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:49.091550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}, {"name": "network-changed", "server_uuid": "d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4", "tag": "418f750b-b477-45ec-bd46-66db978b34e0"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:49.101853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:49.102340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:49.102779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:49.118042 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] Creating event network-changed:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch1-master-1 Oct 15 21:26:49.118557 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] Creating event network-changed:418f750b-b477-45ec-bd46-66db978b34e0 for instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 on nch2-master-1 Oct 15 21:26:49.129984 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-68875926-d050-4d97-9f7f-a74f655a1dbf req-76c2f8ae-ce8a-4f47-a168-102b38bc185f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.042126 Oct 15 21:26:49.130602 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1572/5386] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:49 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:26:49.268290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:49.276543 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26e8aaac-b4aa-4220-bb20-7c473e7c3b66 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.210787 Oct 15 21:26:49.277315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3815/5387] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:49 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:49.740594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:49.741338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:49.748054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:49.748535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:49.749078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:49.749783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:49.750242 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:49.750668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:49.884261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:49.892094 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a7350471-3d0b-4e99-984c-c30f88de15fe tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.156763 Oct 15 21:26:49.892657 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1573/5388] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:49 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:50.300646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:50.301290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:50.308255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:50.308738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:50.309209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:50.309901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:50.310392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:50.310788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:50.449261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:50.457647 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95afa669-764c-43d7-a8b8-088edd141885 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.160970 Oct 15 21:26:50.458215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3816/5389] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:50 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:50.915568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:50.916185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:50.923661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:50.924159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:50.924596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:50.925320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:50.925778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:50.926219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:51.064337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:51.072374 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-004bcc25-ecc1-4c94-bc1c-228b08b190ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.160669 Oct 15 21:26:51.072953 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1574/5390] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:50 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:51.481672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:51.483624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:51.490852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:51.491353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:51.491793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:51.492514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:51.492982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:51.493444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:51.757452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:51.768270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-114da5a8-c3d5-43cf-a47e-df82e4c0d02d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.290464 Oct 15 21:26:51.768531 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3817/5391] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:51 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:52.095972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:52.096519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:52.103333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:52.103822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:52.104510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:52.105253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:52.105713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:52.106145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:52.241253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:52.249818 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0799cfbb-6d52-4cdd-ae5f-71d850535d69 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.158416 Oct 15 21:26:52.250489 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1575/5392] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:52 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:52.791547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:52.792086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:52.799142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:52.799628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:52.800066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:52.800756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:52.801249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:52.801685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:52.931902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:52.940490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1c3e514-c307-4d7e-807f-d62ae32bbedd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.152972 Oct 15 21:26:52.941084 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3818/5393] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:52 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:53.272785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:53.273454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:53.281030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:53.281555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:53.281995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:53.282786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:53.283248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:53.283708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:53.419157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:53.427747 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc9dcbe6-8d68-47aa-baab-91287be2fb29 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.158775 Oct 15 21:26:53.428325 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1576/5394] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:53 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:53.964505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:53.965069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:53.972548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:53.973064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:53.973521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:53.974228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:53.974693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:53.975129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:54.106405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:54.114823 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-996300c0-f3d7-4201-8fca-bc4e02548bda tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.154277 Oct 15 21:26:54.115400 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3819/5395] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:53 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:54.450773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:54.452094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:54.458926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:54.459409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:54.459853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:54.460528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:54.460981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:54.461430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:54.597057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:54.605121 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f294a5ac-d490-490f-ba3f-a0369be02954 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.157969 Oct 15 21:26:54.605689 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1577/5396] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:54 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:55.115117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-765998bb-a93b-4f1d-a146-009a808977e8 req-66aaffec-dc4e-47aa-b735-5bdf4658b66d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4", "name": "network-vif-plugged", "status": "completed", "tag": "418f750b-b477-45ec-bd46-66db978b34e0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:55.120617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:55.122700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:55.132198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-765998bb-a93b-4f1d-a146-009a808977e8 req-66aaffec-dc4e-47aa-b735-5bdf4658b66d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:55.132700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-765998bb-a93b-4f1d-a146-009a808977e8 req-66aaffec-dc4e-47aa-b735-5bdf4658b66d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:55.133200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-765998bb-a93b-4f1d-a146-009a808977e8 req-66aaffec-dc4e-47aa-b735-5bdf4658b66d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:55.137624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:55.138377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:55.138921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:55.145136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:55.145288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-765998bb-a93b-4f1d-a146-009a808977e8 req-66aaffec-dc4e-47aa-b735-5bdf4658b66d service nova] Creating event network-vif-plugged:418f750b-b477-45ec-bd46-66db978b34e0 for instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 on nch2-master-1 Oct 15 21:26:55.145628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:55.146067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:55.146783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:55.147227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:55.147653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:55.150292 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-765998bb-a93b-4f1d-a146-009a808977e8 req-66aaffec-dc4e-47aa-b735-5bdf4658b66d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038921 Oct 15 21:26:55.151094 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3820/5397] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:26:55.352045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:55.361403 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ea4cf27d-ce5f-4deb-9ce6-002b9d7e5203 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.227034 Oct 15 21:26:55.361977 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1578/5398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:55 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:26:55.629527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:55.630156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:55.637062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:55.637671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:55.638107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:55.638854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:55.639318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:55.639805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:55.774119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:55.782363 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-223e545b-6d06-444c-9b24-34d6dfcf19ff tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1318 microversion: 2.1 time: 0.158316 Oct 15 21:26:55.782940 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3821/5399] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:55 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:56.386136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:56.386852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:56.394960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:56.395450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:56.395950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:56.396694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:56.397235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:56.397664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:56.416921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:26:56.585499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:56.593499 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef58ec58-5fa4-48bf-85c2-ceeff2bf1e5e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.211216 Oct 15 21:26:56.594067 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1579/5400] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:56 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:26:56.806418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:56.807006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:56.814533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:56.815059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:56.815500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:56.816207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:56.816687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:56.817143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:57.046713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:57.054586 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4ad4b7b-6d13-411c-aac3-ac25944932cf tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1506 microversion: 2.1 time: 0.252030 Oct 15 21:26:57.055476 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3822/5401] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:56 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1506 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:57.079494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7dd6e9c3-3066-4643-8d79-139785176d7f", "device": "/dev/sdb"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:57.082232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:57.088406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:57.088902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:57.089379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:57.090373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:57.090836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:57.091268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:57.367615 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78c6b770-17a8-44a0-b2ee-3dd30e5a80db tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "POST /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.293247 Oct 15 21:26:57.388748 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1580/5402] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:26:57 2022] POST /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4/os-volume_attachments => generated 194 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:26:57.617881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:57.618597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:57.625454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:57.625960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:57.626594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:57.627327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:57.627799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:57.628236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:57.835071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:57.844244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5821bd81-00e0-48ac-8c44-446ef88515e2 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1693 microversion: 2.1 time: 0.230286 Oct 15 21:26:57.844930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3823/5403] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:57 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1693 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:58.869406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:58.870046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:58.876474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:58.876986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:58.877465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:58.878163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:58.878616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:58.879043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.018063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:59.027455 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bab0393d-86e3-465a-b0a6-7b60cc43c951 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1683 microversion: 2.1 time: 0.163232 Oct 15 21:26:59.028138 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1581/5404] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:58 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1683 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:59.051948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:59.052679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:59.059474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.060043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.060559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.061448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.061941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.062364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.159814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d7afffce-e87e-447a-9134-79d17ffcf796 req-4eb11acb-18bb-432b-8da3-322f6e5c23dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "name": "network-vif-plugged", "status": "completed", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:59.171371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d7afffce-e87e-447a-9134-79d17ffcf796 req-4eb11acb-18bb-432b-8da3-322f6e5c23dc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.171933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d7afffce-e87e-447a-9134-79d17ffcf796 req-4eb11acb-18bb-432b-8da3-322f6e5c23dc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.172438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d7afffce-e87e-447a-9134-79d17ffcf796 req-4eb11acb-18bb-432b-8da3-322f6e5c23dc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.185003 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d7afffce-e87e-447a-9134-79d17ffcf796 req-4eb11acb-18bb-432b-8da3-322f6e5c23dc service nova] Creating event network-vif-plugged:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch2-master-1 Oct 15 21:26:59.199359 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d7afffce-e87e-447a-9134-79d17ffcf796 req-4eb11acb-18bb-432b-8da3-322f6e5c23dc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043374 Oct 15 21:26:59.201980 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1582/5405] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:26:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:26:59.209315 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f6dd30b2-bb6e-4fc2-bd18-99d160bbabba tempest-TestNetworkAdvancedServerOps-1153367167 tempest-TestNetworkAdvancedServerOps-1153367167-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1834 microversion: 2.1 time: 0.161020 Oct 15 21:26:59.210175 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3824/5406] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:59 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1834 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:59.234812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:26:59.235528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:59.241871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.242355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.242795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.243500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.243961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.244388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.374337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:26:59.382383 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-00938502-a12c-460f-af05-3a43bd74320f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1683 microversion: 2.1 time: 0.152804 Oct 15 21:26:59.382953 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1583/5407] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:26:59 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1683 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:26:59.481204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:26:59.483615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:26:59.489665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.490153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.490595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.491290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:26:59.491758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:26:59.492186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:26:59.587187 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e5436cc0-c497-4e25-97c0-9ac74ef312a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action" status: 200 len: 38489 microversion: 2.1 time: 0.109870 Oct 15 21:26:59.588094 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3825/5408] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:26:59 2022] POST /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2/action => generated 38489 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:27:04.071247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:04.071743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:04.074293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-97251c00-be79-4ac6-b445-fb6dffa7be90 req-62f700b9-4dab-4ba0-b3db-b1ea1865b798 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:04.079105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:04.079596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:04.080034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:04.080734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:04.081227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:04.081654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:04.087631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-97251c00-be79-4ac6-b445-fb6dffa7be90 req-62f700b9-4dab-4ba0-b3db-b1ea1865b798 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:04.088370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-97251c00-be79-4ac6-b445-fb6dffa7be90 req-62f700b9-4dab-4ba0-b3db-b1ea1865b798 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:04.088907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-97251c00-be79-4ac6-b445-fb6dffa7be90 req-62f700b9-4dab-4ba0-b3db-b1ea1865b798 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:04.102071 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-97251c00-be79-4ac6-b445-fb6dffa7be90 req-62f700b9-4dab-4ba0-b3db-b1ea1865b798 service nova] Creating event network-changed:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch2-master-1 Oct 15 21:27:04.107703 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-97251c00-be79-4ac6-b445-fb6dffa7be90 req-62f700b9-4dab-4ba0-b3db-b1ea1865b798 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037672 Oct 15 21:27:04.108506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:27:04.109687 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3826/5409] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:27:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:27:04.211114 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bd8620b7-77ed-4042-a0c7-244a6af83950 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 204 len: 0 microversion: 2.1 time: 0.142562 Oct 15 21:27:04.212346 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1584/5410] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:27:04 2022] DELETE /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:27:04.224071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:04.224967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:04.233550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:04.234233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:04.234448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:04.235233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:04.235681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:04.236202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:04.435185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:04.444297 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b84608e6-2180-45b1-b44c-c19becb102a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1689 microversion: 2.1 time: 0.227541 Oct 15 21:27:04.451540 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3827/5411] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:04 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1689 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:27:05.324104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-93849d48-7db0-4a85-8acc-ff40304adba2 req-fbd1fda6-3bc2-4f82-aa10-68d553a3599f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29605bae-becd-4e1b-96bc-84aee0ba8be2", "name": "network-vif-deleted", "tag": "90aa4401-4312-4be3-99a4-5da83136f7bf"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:05.335331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-93849d48-7db0-4a85-8acc-ff40304adba2 req-fbd1fda6-3bc2-4f82-aa10-68d553a3599f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:05.335827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-93849d48-7db0-4a85-8acc-ff40304adba2 req-fbd1fda6-3bc2-4f82-aa10-68d553a3599f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:05.336260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-93849d48-7db0-4a85-8acc-ff40304adba2 req-fbd1fda6-3bc2-4f82-aa10-68d553a3599f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:05.347814 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-93849d48-7db0-4a85-8acc-ff40304adba2 req-fbd1fda6-3bc2-4f82-aa10-68d553a3599f service nova] Creating event network-vif-deleted:90aa4401-4312-4be3-99a4-5da83136f7bf for instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 on nch2-master-1 Oct 15 21:27:05.352661 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-93849d48-7db0-4a85-8acc-ff40304adba2 req-fbd1fda6-3bc2-4f82-aa10-68d553a3599f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032613 Oct 15 21:27:05.353367 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1585/5412] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:27:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:27:05.468759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:05.469342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:05.476450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:05.476946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:05.477426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:05.478184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:05.478658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:05.479095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:05.513396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:05.514138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:05.515333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:05.515839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lazy-loading 'fault' on Instance uuid 29605bae-becd-4e1b-96bc-84aee0ba8be2 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:27:05.558129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:05.565895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fdf4e902-5492-4931-b68e-a6a3118312d6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 200 len: 1356 microversion: 2.1 time: 0.101158 Oct 15 21:27:05.566469 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3828/5413] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:05 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 1356 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:06.591368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:06.591994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 29605bae-becd-4e1b-96bc-84aee0ba8be2] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:06.594725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:06.600217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:06.600699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:06.601159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:06.601857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:06.602307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:06.602729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:06.617782 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] HTTP exception thrown: Instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 could not be found. Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Request method failure captured: Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 HTTP/1.1 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyRbA0cLyHQrk49g1yDTx2mawYGe7wmuR6AIZfAxBzfijpxhir3KV-4WCV7R2Uxj7n0-Vd4Epp7EBbcjQo861nJwrUnuYBBc0VHGRvHVNbDWK0zr0_BWwaxNQHYbUerlTCJoIXcNVviBEtMXGQni285yOrYoEZhsCaxrPEmh_p7oayg Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a0d6261b46824acf958ef749425ae421 Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 could not be found. Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '29605bae-becd-4e1b-96bc-84aee0ba8be2'} Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 could not be found. Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 could not be found. Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:27:06.619465 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 could not be found. Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Returning 404 to user: Instance 29605bae-becd-4e1b-96bc-84aee0ba8be2 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a253beeb-22cf-4eae-bb35-af89d4d54cdd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2" status: 404 len: 111 microversion: 2.1 time: 0.034755 Oct 15 21:27:06.623977 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1586/5414] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:06 2022] GET /compute/v2.1/servers/29605bae-becd-4e1b-96bc-84aee0ba8be2 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:27:09.903001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5dac919a-baae-49e6-a8de-89701aff2175 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '.version_select at 0x7fe74e15ddc0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:09.911113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:09.924406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5dac919a-baae-49e6-a8de-89701aff2175 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-160637841" status: 202 len: 0 microversion: 2.1 time: 0.025309 Oct 15 21:27:09.925006 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3829/5415] 10.222.0.16 () {58 vars in 1281 bytes} [Sat Oct 15 21:27:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-160637841 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:27:09.947644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-25568ccb-53bf-474a-b562-3492e7c797c3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: .version_select at 0x7fe75847d310>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-2003145326", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOwjCslk5FloFt4pdac2d3+urpltYKHI06jb8wUOsrrB2pMx85SR7hiCw+js9M0DyIQ7HfjdZzEaaNOMatj8hubktAL8pF8Q2v41NyyMx3D7ElZdPJkXdHZNoIRpCv/0Ow=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:09.966779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-25568ccb-53bf-474a-b562-3492e7c797c3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:27:09.972378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-25568ccb-53bf-474a-b562-3492e7c797c3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:27:10.002208 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-25568ccb-53bf-474a-b562-3492e7c797c3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.058389 Oct 15 21:27:10.002830 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1587/5416] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:27:09 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:27:15.550428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-344324696", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "735307a3-c350-4102-a048-2e781216aea7"}], "key_name": "tempest-TestNetworkAdvancedServerOps-2003145326", "security_groups": [{"name": "tempest-secgroup-smoke-1845226998"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:15.563815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=44 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:15.706200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] validate_networks() for [('735307a3-c350-4102-a048-2e781216aea7', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:27:16.208165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:27:16.209315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:27:16.209810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:27:16.210231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:27:16.210655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:27:16.221067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:16.221581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:16.222031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:16.222780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:16.223745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:16.223920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:16.245283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:27:16.251320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Getting quotas for user a0d6261b46824acf958ef749425ae421 and project cdd706928da546cf8e5b0fcbf6ec94bd. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:27:16.264950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:27:16.268176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:27:16.327351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-35e75006-f20b-4d7c-82bd-3f8a274c00ad tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.783157 Oct 15 21:27:16.328089 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3830/5417] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:27:15 2022] POST /compute/v2.1/servers => generated 406 bytes in 785 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:27:16.341241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8fbd152c-2aa8-4848-b3fd-354f6ec7a5ab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:16.342347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8fbd152c-2aa8-4848-b3fd-354f6ec7a5ab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:16.408775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8fbd152c-2aa8-4848-b3fd-354f6ec7a5ab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:16.417563 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8fbd152c-2aa8-4848-b3fd-354f6ec7a5ab tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1266 microversion: 2.1 time: 0.081318 Oct 15 21:27:16.418499 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1588/5418] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:16 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1266 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:17.445326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:17.447220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:17.457351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:17.458273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:17.458920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:17.460073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:17.460714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:17.462118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:17.547562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:17.555921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d3b1660e-66cb-40a6-972c-676596164b74 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1336 microversion: 2.1 time: 0.119287 Oct 15 21:27:17.556843 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3831/5419] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:17 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1336 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:18.580560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:18.581697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:18.588051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=45,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:18.592324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:18.593063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:18.593660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:18.596135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:18.597250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:18.598277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:19.170140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:19.180124 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aaa95757-35f9-4393-a229-5cf59eb143c6 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1392 microversion: 2.1 time: 0.604215 Oct 15 21:27:19.180908 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1589/5420] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:18 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1392 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:19.305611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3d61b7e8-acc3-43ad-a318-484a54d25458 req-81831ace-ecfe-4d28-a0a2-5937238b982d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a8428d1-3550-4285-a310-448a1f725c4c", "tag": "bf0034de-2ae9-4efd-b594-6db5e76f4846"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:19.319344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3d61b7e8-acc3-43ad-a318-484a54d25458 req-81831ace-ecfe-4d28-a0a2-5937238b982d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:19.320216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3d61b7e8-acc3-43ad-a318-484a54d25458 req-81831ace-ecfe-4d28-a0a2-5937238b982d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:19.320938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3d61b7e8-acc3-43ad-a318-484a54d25458 req-81831ace-ecfe-4d28-a0a2-5937238b982d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:19.333793 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3d61b7e8-acc3-43ad-a318-484a54d25458 req-81831ace-ecfe-4d28-a0a2-5937238b982d service nova] Creating event network-changed:bf0034de-2ae9-4efd-b594-6db5e76f4846 for instance 8a8428d1-3550-4285-a310-448a1f725c4c on nch1-master-1 Oct 15 21:27:19.339133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3d61b7e8-acc3-43ad-a318-484a54d25458 req-81831ace-ecfe-4d28-a0a2-5937238b982d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037956 Oct 15 21:27:19.340363 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3832/5421] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:27:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:27:20.203936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:20.204727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:20.211767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:20.212257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:20.212700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:20.213452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:20.213917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:20.214348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:20.353116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:20.361858 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-275e8cef-a57e-4e69-92fd-e4ca75003b81 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1392 microversion: 2.1 time: 0.162025 Oct 15 21:27:20.362449 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1590/5422] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:20 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1392 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:21.387133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:21.387834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:21.395113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:21.395598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:21.396034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:21.396726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:21.397217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:21.397644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:21.528458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:21.536448 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad4f4e6e-9ed0-462c-b068-00312f73ae97 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1392 microversion: 2.1 time: 0.154917 Oct 15 21:27:21.537062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3833/5423] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:21 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1392 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:27:22.559850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:22.560497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:22.567928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:22.568416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:22.568859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:22.569613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:22.570078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:22.570489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:22.708268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:22.716020 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-80a2d831-5d25-4160-9103-969b6e37cac8 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1392 microversion: 2.1 time: 0.160181 Oct 15 21:27:22.716590 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1591/5424] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:22 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1392 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:23.738818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:23.739437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:23.746344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:23.746834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:23.747274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:23.747975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:23.748433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:23.748868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:23.967700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:23.975547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-06875f0f-dd77-4f34-8653-f57bf6722f19 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1392 microversion: 2.1 time: 0.240770 Oct 15 21:27:23.976124 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3834/5425] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:23 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1392 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:24.209380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-77fe4323-ed8a-438b-aa02-3759754bf309 req-a7b51e39-80ec-4168-b52a-78f6cd67ec26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a8428d1-3550-4285-a310-448a1f725c4c", "name": "network-vif-plugged", "status": "completed", "tag": "bf0034de-2ae9-4efd-b594-6db5e76f4846"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:24.219966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-77fe4323-ed8a-438b-aa02-3759754bf309 req-a7b51e39-80ec-4168-b52a-78f6cd67ec26 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:24.220525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-77fe4323-ed8a-438b-aa02-3759754bf309 req-a7b51e39-80ec-4168-b52a-78f6cd67ec26 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:24.220956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-77fe4323-ed8a-438b-aa02-3759754bf309 req-a7b51e39-80ec-4168-b52a-78f6cd67ec26 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:24.232505 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-77fe4323-ed8a-438b-aa02-3759754bf309 req-a7b51e39-80ec-4168-b52a-78f6cd67ec26 service nova] Creating event network-vif-plugged:bf0034de-2ae9-4efd-b594-6db5e76f4846 for instance 8a8428d1-3550-4285-a310-448a1f725c4c on nch1-master-1 Oct 15 21:27:24.237624 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-77fe4323-ed8a-438b-aa02-3759754bf309 req-a7b51e39-80ec-4168-b52a-78f6cd67ec26 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031835 Oct 15 21:27:24.238211 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1592/5426] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:27:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:27:24.998379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:24.998924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:25.005869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:25.006360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:25.006801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:25.007500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:25.007959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:25.008389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:25.139470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:25.147818 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc708496-1b2a-4037-b7c1-8c4bbbfc907f tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1392 microversion: 2.1 time: 0.153453 Oct 15 21:27:25.148392 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3835/5427] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:24 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1392 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:26.173126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:26.173839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:26.180793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:26.181314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:26.181759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:26.182457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:26.182917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:26.183346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:26.319267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:26.327232 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5000d3e6-7934-416e-8529-1680d64ad7ee tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1563 microversion: 2.1 time: 0.159225 Oct 15 21:27:26.327811 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1593/5428] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:26 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1563 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:26.350313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:26.351593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:26.354476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:26.355164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:26.361760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:26.362896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:26.363349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:26.364056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:26.364554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:26.364990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:26.499369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:26.507140 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee1315ba-a6af-429b-bad7-ab45568c17aa tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1563 microversion: 2.1 time: 0.161206 Oct 15 21:27:26.507726 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3836/5429] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:26 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1563 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:28.120039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-154c37cb-0384-4433-91ed-4679e9ca35b1 req-3a4ec9c7-9e66-4d5c-997c-4661e260f4f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a8428d1-3550-4285-a310-448a1f725c4c", "tag": "bf0034de-2ae9-4efd-b594-6db5e76f4846"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:28.123323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:27:28.130542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:28.132896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:28.139063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:28.139564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:28.140008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:28.140719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:28.141922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:28.142363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:28.146519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-154c37cb-0384-4433-91ed-4679e9ca35b1 req-3a4ec9c7-9e66-4d5c-997c-4661e260f4f8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:28.147155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-154c37cb-0384-4433-91ed-4679e9ca35b1 req-3a4ec9c7-9e66-4d5c-997c-4661e260f4f8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:28.147667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-154c37cb-0384-4433-91ed-4679e9ca35b1 req-3a4ec9c7-9e66-4d5c-997c-4661e260f4f8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:28.161739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-154c37cb-0384-4433-91ed-4679e9ca35b1 req-3a4ec9c7-9e66-4d5c-997c-4661e260f4f8 service nova] Creating event network-changed:bf0034de-2ae9-4efd-b594-6db5e76f4846 for instance 8a8428d1-3550-4285-a310-448a1f725c4c on nch1-master-1 Oct 15 21:27:28.167281 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-154c37cb-0384-4433-91ed-4679e9ca35b1 req-3a4ec9c7-9e66-4d5c-997c-4661e260f4f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050772 Oct 15 21:27:28.168311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3837/5430] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:27:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:27:28.373426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:27:28.384544 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-93fb436d-65c1-43e7-bf53-fa5b4a1e3326 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1563 microversion: 2.1 time: 0.271113 Oct 15 21:27:28.388707 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1594/5431] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:27:28 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1563 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:27:49.642764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:27:49.645026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:27:49.647962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=31,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:49.648617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=36 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:27:49.655179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:49.655664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:49.656104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:49.656799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:27:49.657284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:27:49.657712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:27:49.709020 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7fb7b2d7-fa78-48db-ae59-91fb07f8dc7c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c/action" status: 200 len: 24572 microversion: 2.1 time: 0.070012 Oct 15 21:27:49.709657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3838/5432] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:27:49 2022] POST /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c/action => generated 24572 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:28:00.692903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:28:00.695638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:00.697961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:00.698672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=32 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:00.704431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:00.704856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:00.705321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:00.706017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:00.706472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:00.706898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:00.787758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:28:00.788179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:28:00.788650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:28:00.789080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:28:00.789501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:28:00.850470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:00.856413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:00.856903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:00.857373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:00.858090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:00.858531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:00.858985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:00.890522 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-29cc8ae4-7002-4f93-828d-0c4505e0f8f7 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c/action" status: 202 len: 1136 microversion: 2.1 time: 0.201431 Oct 15 21:28:00.891212 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1595/5433] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:28:00 2022] POST /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c/action => generated 1136 bytes in 203 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Oct 15 21:28:00.910853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:00.911600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:00.915699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:00.916412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:00.922383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:00.922878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:00.923328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:00.924189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:00.924684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:00.925244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:01.064148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:01.072528 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b42e073f-9dda-4326-95d0-416083780567 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.165988 Oct 15 21:28:01.073347 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3839/5434] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:00 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:02.099052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:02.100650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:02.107786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:02.108275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:02.108714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:02.109446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:02.110161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:02.110592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:02.246195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:02.253981 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-604321cd-c9ab-4faf-8046-a2c15c14b220 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.158865 Oct 15 21:28:02.254547 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1596/5435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:02 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:03.278516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:03.279174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:03.286187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:03.286679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:03.287750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:03.288457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:03.288927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:03.289422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:03.422469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:03.430838 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8097459b-d983-4876-9d65-2133a69ed094 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.156318 Oct 15 21:28:03.431409 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3840/5436] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:03 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:04.454924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:04.455459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:04.462236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:04.462717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:04.463158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:04.463848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:04.464301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:04.464723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:04.600865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:04.608744 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87ba10bf-44be-4449-ae99-85a9d6fa19a1 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.158225 Oct 15 21:28:04.609349 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1597/5437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:04 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:05.632345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:05.632966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:05.640055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:05.640542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:05.640980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:05.641699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:05.642157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:05.642593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:05.772885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:05.780845 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5f426e2-4779-4ab8-a0fb-57b6b51b87b3 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.152300 Oct 15 21:28:05.781460 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3841/5438] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:05 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:06.805629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:06.806204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:06.813492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:06.813983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:06.814426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:06.815127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:06.815590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:06.816024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:06.961649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:06.969439 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1e0a6401-bee3-4049-8868-f9ec98743219 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.168337 Oct 15 21:28:06.970013 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1598/5439] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:06 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:07.993690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:07.994301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:08.001266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:08.001755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:08.002196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:08.002900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:08.003360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:08.003794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:08.138681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:08.147536 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aef6aaf9-6e37-4b66-9f1e-6bb87d4e6a4b tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.157655 Oct 15 21:28:08.148127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3842/5440] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:07 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:09.170934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:09.171551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:09.179520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:09.180006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:09.180442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:09.181243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:09.181702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:09.182136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:09.311914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:09.319936 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-804bf983-adf5-483b-9309-f06b45776cbd tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.152845 Oct 15 21:28:09.320507 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1599/5441] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:09 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:10.343664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:10.344275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:10.351846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:10.352325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:10.352766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:10.353477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:10.353938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:10.354632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:10.492374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:10.500217 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d06c312e-dac5-4cff-988c-8b1b8ee59557 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.160451 Oct 15 21:28:10.500790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3843/5442] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:10 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:11.523341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:11.523861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:11.530887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:11.531381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:11.531824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:11.532536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:11.532998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:11.533457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:11.668207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:11.676147 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c6801c02-b4b4-4558-833a-ac5099c51044 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.156908 Oct 15 21:28:11.676865 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1600/5443] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:11 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:12.699710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:12.700331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:12.707393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:12.707817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:12.708254 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:12.708956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:12.709529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:12.709959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:12.840562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:12.848380 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a81ee495-b30e-41df-8e50-4cecc5f0c152 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.152558 Oct 15 21:28:12.848958 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3844/5444] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:12 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:13.872685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:13.873354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:13.880381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:13.881458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:13.881961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:13.882795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:13.883280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:13.883738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:14.016295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:14.024386 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8747b9db-7406-409f-8e63-75f4837c7d32 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.155635 Oct 15 21:28:14.024955 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1601/5445] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:13 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:15.048024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:15.048643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:15.056299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:15.056795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:15.057276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:15.057988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:15.058457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:15.058894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:15.188828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:15.197506 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c99c8f61-2df7-4b7f-be0d-bdb425f1f390 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.153268 Oct 15 21:28:15.198098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3845/5446] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:15 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:16.221585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:16.222211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:16.229936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:16.230431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:16.230872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:16.231572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:16.232035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:16.232469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:16.368288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:16.376322 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-10269a7d-bcd9-405f-9de3-53e945ed0b75 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.158686 Oct 15 21:28:16.376899 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1602/5447] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:16 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:17.400514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:17.401198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:17.408133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:17.408625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:17.409094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:17.409803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:17.410297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:17.410732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:17.546258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:17.554710 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f585413-cfa4-451f-8a12-21c986311a9c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1691 microversion: 2.1 time: 0.158818 Oct 15 21:28:17.555281 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3846/5448] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:17 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1691 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:18.579121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:18.579775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:18.588034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:18.588539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:18.589146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:18.589866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:18.590334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:18.590769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:19.036125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:19.044039 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0af8c09f-b0cd-4ec1-8cac-7ac4ae50e23d tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1697 microversion: 2.1 time: 0.468987 Oct 15 21:28:19.044617 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1603/5449] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:18 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1697 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:20.068207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:20.068900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:20.075667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:20.076158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:20.076610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:20.077353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:20.077826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:20.078265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:20.242555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:20.250370 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1aaf0735-f92c-4263-8124-6a367b301163 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1682 microversion: 2.1 time: 0.186035 Oct 15 21:28:20.250947 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3847/5450] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:20 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1682 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:22.529956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-f7dbaf4b-5968-4971-976b-4c0aaa88972c req-67d072bd-b39b-4c74-a212-d56ab1d54539 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a8428d1-3550-4285-a310-448a1f725c4c", "name": "network-vif-plugged", "status": "completed", "tag": "bf0034de-2ae9-4efd-b594-6db5e76f4846"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:28:22.542857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f7dbaf4b-5968-4971-976b-4c0aaa88972c req-67d072bd-b39b-4c74-a212-d56ab1d54539 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:22.543353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f7dbaf4b-5968-4971-976b-4c0aaa88972c req-67d072bd-b39b-4c74-a212-d56ab1d54539 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:22.543787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f7dbaf4b-5968-4971-976b-4c0aaa88972c req-67d072bd-b39b-4c74-a212-d56ab1d54539 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:22.556093 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-f7dbaf4b-5968-4971-976b-4c0aaa88972c req-67d072bd-b39b-4c74-a212-d56ab1d54539 service nova] Creating event network-vif-plugged:bf0034de-2ae9-4efd-b594-6db5e76f4846 for instance 8a8428d1-3550-4285-a310-448a1f725c4c on nch1-master-1 Oct 15 21:28:22.561935 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-f7dbaf4b-5968-4971-976b-4c0aaa88972c req-67d072bd-b39b-4c74-a212-d56ab1d54539 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035258 Oct 15 21:28:22.562865 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1604/5451] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:28:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:28:35.949492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:35.949994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:35.952929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=18 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:35.953618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=54 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:35.959360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:35.959847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:35.960285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:35.960979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:35.962196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:35.963414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:36.042159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lazy-loading 'services' on Instance uuid d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:28:36.111765 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-101c222e-68f2-4386-87a4-2f88609f8bd7 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4/os-volume_attachments/7dd6e9c3-3066-4643-8d79-139785176d7f" status: 202 len: 0 microversion: 2.1 time: 0.166701 Oct 15 21:28:36.112677 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3848/5452] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:28:35 2022] DELETE /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4/os-volume_attachments/7dd6e9c3-3066-4643-8d79-139785176d7f => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:28:37.236715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:37.237336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:37.240159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=24 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:37.240800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:37.247928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:37.248415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:37.248851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:37.249573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:37.250029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:37.250453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:37.278035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:28:37.354894 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-adea6e67-b958-4793-8c3b-49a82a2015d5 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 204 len: 0 microversion: 2.1 time: 0.122413 Oct 15 21:28:37.355673 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1605/5453] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:28:37 2022] DELETE /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:28:37.365601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:37.366316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:37.373500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:37.373991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:37.374434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:37.375139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:37.375603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:37.376037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:37.516509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:37.525162 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2455ea86-abbb-409f-983e-60e1b74669dd tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1512 microversion: 2.1 time: 0.164941 Oct 15 21:28:37.525744 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3849/5454] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:37 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1512 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:28:38.548527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:38.549245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:38.556901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:38.557407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:38.557842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:38.558525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:38.559010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:38.559430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:38.692951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:38.700894 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ad41b1f3-580c-4423-b7fd-2d937a4e5582 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1512 microversion: 2.1 time: 0.156716 Oct 15 21:28:38.701504 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1606/5455] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:38 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1512 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:39.503311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0c31e59a-10de-4303-94e5-2a31f3682697 req-4a422f6c-a015-478a-ac2c-9e888029256d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4", "name": "network-vif-deleted", "tag": "418f750b-b477-45ec-bd46-66db978b34e0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:28:39.514465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0c31e59a-10de-4303-94e5-2a31f3682697 req-4a422f6c-a015-478a-ac2c-9e888029256d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:39.514946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0c31e59a-10de-4303-94e5-2a31f3682697 req-4a422f6c-a015-478a-ac2c-9e888029256d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:39.515371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0c31e59a-10de-4303-94e5-2a31f3682697 req-4a422f6c-a015-478a-ac2c-9e888029256d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:39.527250 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0c31e59a-10de-4303-94e5-2a31f3682697 req-4a422f6c-a015-478a-ac2c-9e888029256d service nova] Creating event network-vif-deleted:418f750b-b477-45ec-bd46-66db978b34e0 for instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 on nch2-master-1 Oct 15 21:28:39.532132 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0c31e59a-10de-4303-94e5-2a31f3682697 req-4a422f6c-a015-478a-ac2c-9e888029256d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032514 Oct 15 21:28:39.532735 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3850/5456] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:28:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:28:39.724338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:39.724950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:39.732432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:39.732918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:39.733396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:39.734086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:39.734572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:39.734997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:39.766355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:39.766831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:39.767267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:39.767710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lazy-loading 'fault' on Instance uuid d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:28:39.809384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd159c11f06c4482ebf4e600305bfe0ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '73d7ccfabbb44b5d8b1aa1253b2048e1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:39.817385 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e8b6c4c2-4aec-46e2-b5ab-0d1ed0dc206c tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 200 len: 1308 microversion: 2.1 time: 0.097409 Oct 15 21:28:39.817943 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1607/5457] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:39 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 1308 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:40.842479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:40.843108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] [instance: d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:40.849448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:40.849937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:40.850382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:40.851081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:40.851541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:40.851971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:40.867125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] HTTP exception thrown: Instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 could not be found. Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Request method failure captured: Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 HTTP/1.1 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyUJeBQH6P-a9dnU9biwtBaxfL2qEmsNBiXdbdPgkTWZhQ3eI4DNXNVBpAHOLKsspEY3KP8dYGKYNKvBvan1HFWI1HDriGddTxuT6yzIB_aR5yXlnnGB8-O-OTqhDEYlsJl8E-5KlU4PnGwALGn7pHfX8JblCC4ehYltCY6nP9X-718 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 73d7ccfabbb44b5d8b1aa1253b2048e1 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-VolumesSnapshotTestJSON-153295047 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/73d7ccfabbb44b5d8b1aa1253b2048e1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-VolumesSnapshotTestJSON-153295047 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 73d7ccfabbb44b5d8b1aa1253b2048e1 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-153295047 Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-VolumesSnapshotTestJSON-153295047-project Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: d159c11f06c4482ebf4e600305bfe0ee Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-VolumesSnapshotTestJSON-153295047-project Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 could not be found. Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4'} Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 could not be found. Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 could not be found. Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:28:40.868792 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 could not be found. Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] Returning 404 to user: Instance d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3d032b3d-1ab0-431c-b71e-a280d6fbfd66 tempest-VolumesSnapshotTestJSON-153295047 tempest-VolumesSnapshotTestJSON-153295047-project] 10.222.0.16 "GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4" status: 404 len: 111 microversion: 2.1 time: 0.033552 Oct 15 21:28:40.873145 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3851/5458] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:40 2022] GET /compute/v2.1/servers/d5e76d3b-f9ad-4cfd-b750-eb4d0088a3f4 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:28:44.489419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:28:44.491630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:44.499187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:44.499683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:44.500127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:44.500832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:44.501332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:44.501769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:44.553570 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-171326c4-53af-4e7c-b425-c269594e7cc9 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "POST /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c/action" status: 200 len: 24569 microversion: 2.1 time: 0.068190 Oct 15 21:28:44.554205 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1608/5459] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:28:44 2022] POST /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c/action => generated 24569 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:28:51.763913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:51.764656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:51.765324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ac0ea930-0dc6-46bf-91b1-e7c4a685ac46 req-9a8c0195-2934-4ea1-8dce-173bd911a06b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a8428d1-3550-4285-a310-448a1f725c4c", "tag": "bf0034de-2ae9-4efd-b594-6db5e76f4846"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:28:51.768746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:51.769828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:28:51.774090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:51.774854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:51.776341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:51.777772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:51.778532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:51.779040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:51.779301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ac0ea930-0dc6-46bf-91b1-e7c4a685ac46 req-9a8c0195-2934-4ea1-8dce-173bd911a06b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:51.779805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ac0ea930-0dc6-46bf-91b1-e7c4a685ac46 req-9a8c0195-2934-4ea1-8dce-173bd911a06b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:51.780505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ac0ea930-0dc6-46bf-91b1-e7c4a685ac46 req-9a8c0195-2934-4ea1-8dce-173bd911a06b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:51.798104 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ac0ea930-0dc6-46bf-91b1-e7c4a685ac46 req-9a8c0195-2934-4ea1-8dce-173bd911a06b service nova] Creating event network-changed:bf0034de-2ae9-4efd-b594-6db5e76f4846 for instance 8a8428d1-3550-4285-a310-448a1f725c4c on nch1-master-1 Oct 15 21:28:51.804981 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ac0ea930-0dc6-46bf-91b1-e7c4a685ac46 req-9a8c0195-2934-4ea1-8dce-173bd911a06b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043723 Oct 15 21:28:51.807134 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1609/5460] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:28:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:28:51.826373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:28:51.925654 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c827df70-f7d7-4c4b-b86c-1cfeddb13368 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 204 len: 0 microversion: 2.1 time: 0.166930 Oct 15 21:28:51.929787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3852/5461] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:28:51 2022] DELETE /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:28:51.936251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:51.937266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:51.944642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:51.945401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:51.946114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:51.947990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:51.948780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:51.949471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:52.114055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:52.122713 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-caf665dc-37c0-43c9-bb00-f8ab39f9eb6c tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1688 microversion: 2.1 time: 0.191175 Oct 15 21:28:52.123290 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1610/5462] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:51 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1688 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:53.148078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:53.149010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:53.156336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:53.156824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:53.157294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:53.157993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:53.158456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:53.158886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:53.592590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a0d6261b46824acf958ef749425ae421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdd706928da546cf8e5b0fcbf6ec94bd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:28:53.600270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21dc386f-d355-4f5e-869c-00c4497297ce tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 200 len: 1688 microversion: 2.1 time: 0.456603 Oct 15 21:28:53.600837 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3853/5463] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:53 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 1688 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:28:53.954009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8391d739-acf6-4ce7-95c2-a225d510a786 req-8fa801db-805b-4c42-a12b-e516821a4e4a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a8428d1-3550-4285-a310-448a1f725c4c", "name": "network-vif-deleted", "tag": "bf0034de-2ae9-4efd-b594-6db5e76f4846"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:28:53.966988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8391d739-acf6-4ce7-95c2-a225d510a786 req-8fa801db-805b-4c42-a12b-e516821a4e4a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:53.967470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8391d739-acf6-4ce7-95c2-a225d510a786 req-8fa801db-805b-4c42-a12b-e516821a4e4a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:53.967899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8391d739-acf6-4ce7-95c2-a225d510a786 req-8fa801db-805b-4c42-a12b-e516821a4e4a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:53.979910 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8391d739-acf6-4ce7-95c2-a225d510a786 req-8fa801db-805b-4c42-a12b-e516821a4e4a service nova] Creating event network-vif-deleted:bf0034de-2ae9-4efd-b594-6db5e76f4846 for instance 8a8428d1-3550-4285-a310-448a1f725c4c on nch1-master-1 Oct 15 21:28:53.984834 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8391d739-acf6-4ce7-95c2-a225d510a786 req-8fa801db-805b-4c42-a12b-e516821a4e4a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034855 Oct 15 21:28:53.985470 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1611/5464] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:28:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:28:54.625310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:54.625927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] [instance: 8a8428d1-3550-4285-a310-448a1f725c4c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:28:54.632955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:54.633483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:54.633926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:54.634626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:28:54.635164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:28:54.635609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:28:54.649445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] HTTP exception thrown: Instance 8a8428d1-3550-4285-a310-448a1f725c4c could not be found. Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Request method failure captured: Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c HTTP/1.1 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyRbA0cLyHQrk49g1yDTx2mawYGe7wmuR6AIZfAxBzfijpxhir3KV-4WCV7R2Uxj7n0-Vd4Epp7EBbcjQo861nJwrUnuYBBc0VHGRvHVNbDWK0zr0_BWwaxNQHYbUerlTCJoIXcNVviBEtMXGQni285yOrYoEZhsCaxrPEmh_p7oayg Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cdd706928da546cf8e5b0fcbf6ec94bd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cdd706928da546cf8e5b0fcbf6ec94bd Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1799936752 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a0d6261b46824acf958ef749425ae421 Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1799936752-project Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 8a8428d1-3550-4285-a310-448a1f725c4c could not be found. Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '8a8428d1-3550-4285-a310-448a1f725c4c'} Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a8428d1-3550-4285-a310-448a1f725c4c could not be found. Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a8428d1-3550-4285-a310-448a1f725c4c could not be found. Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:54.651189 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a8428d1-3550-4285-a310-448a1f725c4c could not be found. Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Returning 404 to user: Instance 8a8428d1-3550-4285-a310-448a1f725c4c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c235ab6-436c-47c6-bbc0-0b614be71283 tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c" status: 404 len: 111 microversion: 2.1 time: 0.031341 Oct 15 21:28:54.655345 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3854/5465] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:28:54 2022] GET /compute/v2.1/servers/8a8428d1-3550-4285-a310-448a1f725c4c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:28:58.250114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e21fa8c-1087-49ad-a33e-2c0bad660e4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] Calling method '.version_select at 0x7fe74f3dad30>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:28:58.275315 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1e21fa8c-1087-49ad-a33e-2c0bad660e4e tempest-TestNetworkAdvancedServerOps-1799936752 tempest-TestNetworkAdvancedServerOps-1799936752-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-2003145326" status: 202 len: 0 microversion: 2.1 time: 0.029808 Oct 15 21:28:58.277294 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1612/5466] 10.222.0.16 () {58 vars in 1284 bytes} [Sat Oct 15 21:28:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-2003145326 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:29:02.670200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerAdvancedOps-server-1808115484", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "86007dbc-d448-41d1-8616-b17f8be95f88"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:02.728129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] validate_networks() for [('86007dbc-d448-41d1-8616-b17f8be95f88', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:29:03.112424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:29:03.112934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:29:03.113526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:29:03.114013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:29:03.114751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:29:03.125164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:03.125670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:03.126110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:03.126871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:03.127336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:03.127765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:03.147080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Getting quotas for project a82cb785a32d473db8e973a212715528. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:03.153048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Getting quotas for user 3df1b449adeb42c58878a8344c9ddd66 and project a82cb785a32d473db8e973a212715528. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:03.165580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:29:03.167936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:29:03.218194 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-76a11f9e-476b-45ad-a0cd-949c7d429c64 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.798352 Oct 15 21:29:03.225231 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3855/5467] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:29:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 800 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:29:03.230412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd30c60e-9c3b-4296-896e-7235a036b7e2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:03.231159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fd30c60e-9c3b-4296-896e-7235a036b7e2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:03.311854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fd30c60e-9c3b-4296-896e-7235a036b7e2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:03.319322 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fd30c60e-9c3b-4296-896e-7235a036b7e2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1215 microversion: 2.1 time: 0.092757 Oct 15 21:29:03.319933 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1613/5468] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:03 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1215 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:04.344762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:04.345390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:04.354865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:04.355679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:04.355983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:04.356700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:04.357190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:04.357624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:04.477967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:04.489062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1416b3cf-12ce-4cc2-b1fe-084979498ace tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1285 microversion: 2.1 time: 0.149318 Oct 15 21:29:04.490082 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3856/5469] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:04 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1285 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:05.513765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:05.515053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:05.517379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:05.528026 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:05.529525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:05.530854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:05.531814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:05.532685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:05.533365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:05.795731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:05.804555 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c184e033-2e19-499b-b55b-e16e4edac946 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.295222 Oct 15 21:29:05.817951 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1614/5470] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:05 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:05.911453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3e512995-1853-4667-ab6a-466390931a6f req-4ab4e13d-a9ab-4d9d-b6ea-37062f63e57a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce54bbef-d609-4599-8ae5-e46b0fa64f30", "tag": "e4661ac3-e27b-495f-ae9c-f8a6c9653719"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:05.957522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3e512995-1853-4667-ab6a-466390931a6f req-4ab4e13d-a9ab-4d9d-b6ea-37062f63e57a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:05.959062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3e512995-1853-4667-ab6a-466390931a6f req-4ab4e13d-a9ab-4d9d-b6ea-37062f63e57a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:05.959844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3e512995-1853-4667-ab6a-466390931a6f req-4ab4e13d-a9ab-4d9d-b6ea-37062f63e57a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:05.990809 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3e512995-1853-4667-ab6a-466390931a6f req-4ab4e13d-a9ab-4d9d-b6ea-37062f63e57a service nova] Creating event network-changed:e4661ac3-e27b-495f-ae9c-f8a6c9653719 for instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 on nch2-master-1 Oct 15 21:29:06.000984 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3e512995-1853-4667-ab6a-466390931a6f req-4ab4e13d-a9ab-4d9d-b6ea-37062f63e57a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095665 Oct 15 21:29:06.006229 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3857/5471] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:29:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:29:06.834245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:06.835050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:06.850456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:06.850995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:06.851474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:06.852204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:06.852673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:06.853124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:07.029468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:07.046145 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f350e06a-2be7-4e11-b100-605dfb915d0c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.219653 Oct 15 21:29:07.061752 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1615/5472] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:06 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:08.076577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:08.082942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:08.090269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:08.090762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:08.098160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:08.101203 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:08.101766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:08.102493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:08.294421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1698ee7f-8160-41ed-a437-5dd47bdbb0a7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'create', calling method: .version_select at 0x7fe74ee56820>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-2086908361", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAxTVTB5C6UqO26GZP02ikRWWSmj/G9Ax2U9d6s24lUNidHgkgkMW9CFFe5kUk8ELz5PPXqibjqxIQ5bcMt2lAf+53VcX1YYxnPWqr7Wut+ib8k4am7a94rFr8NjmI8HJw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:08.301129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:08.317650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1698ee7f-8160-41ed-a437-5dd47bdbb0a7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:08.324424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1698ee7f-8160-41ed-a437-5dd47bdbb0a7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for user bd14dd0707a1423d82328ac3a54ccb28 and project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:08.340920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:08.352314 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4915af7e-ea75-425c-a828-4f17a4c960d3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.281012 Oct 15 21:29:08.353898 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3858/5473] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:08 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:08.402758 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1698ee7f-8160-41ed-a437-5dd47bdbb0a7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.112874 Oct 15 21:29:08.403418 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1616/5474] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:29:08 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:29:09.378588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:09.381366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:09.398272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:09.399122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:09.399982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:09.401026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:09.403226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:09.403905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:09.577689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:09.587464 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c87b8bea-503c-49c0-928e-ad985a2edea6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.212964 Oct 15 21:29:09.588273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3859/5475] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:09 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:10.061311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-1461057923", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "8ed47ca5-ebee-4a9d-bc69-7328757ecebe"}], "key_name": "tempest-TestNetworkBasicOps-2086908361", "security_groups": [{"name": "tempest-secgroup-smoke-1915441306"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:10.181815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] validate_networks() for [('8ed47ca5-ebee-4a9d-bc69-7328757ecebe', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:29:10.612241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:10.615267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:10.627874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:10.628576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:10.629209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:10.630117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:10.630731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:10.631329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:10.696955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:29:10.697854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:29:10.698755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:29:10.699734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:29:10.700356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:29:10.712006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:10.714359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:10.715069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:10.716653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:10.717339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:10.717961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:10.747836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:10.753655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for user bd14dd0707a1423d82328ac3a54ccb28 and project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:10.768728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:29:10.771137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:29:10.808289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:10.817156 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5485f086-8181-4e34-8028-2a71d7d04fde tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.208597 Oct 15 21:29:10.818250 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3860/5476] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:10 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:10.837821 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c5c6efe-5bbf-46c2-a59a-f9ab25a832ac tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.780808 Oct 15 21:29:10.850998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f04c82cd-9ec0-4e1b-beaa-d8ac4494f62d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:10.852274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f04c82cd-9ec0-4e1b-beaa-d8ac4494f62d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:10.856832 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1617/5477] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:29:10 2022] POST /compute/v2.1/servers => generated 406 bytes in 801 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:29:10.934887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f04c82cd-9ec0-4e1b-beaa-d8ac4494f62d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:10.943457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f04c82cd-9ec0-4e1b-beaa-d8ac4494f62d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1249 microversion: 2.1 time: 0.096612 Oct 15 21:29:10.959332 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3861/5478] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:10 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1249 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:29:11.718440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-74af51fd-c505-4b49-895a-56a42733f2dd req-88697980-f291-4055-bf23-1d220f758a51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ce54bbef-d609-4599-8ae5-e46b0fa64f30", "name": "network-vif-plugged", "status": "completed", "tag": "e4661ac3-e27b-495f-ae9c-f8a6c9653719"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:11.733013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-74af51fd-c505-4b49-895a-56a42733f2dd req-88697980-f291-4055-bf23-1d220f758a51 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:11.733646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-74af51fd-c505-4b49-895a-56a42733f2dd req-88697980-f291-4055-bf23-1d220f758a51 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:11.734151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-74af51fd-c505-4b49-895a-56a42733f2dd req-88697980-f291-4055-bf23-1d220f758a51 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:11.746044 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-74af51fd-c505-4b49-895a-56a42733f2dd req-88697980-f291-4055-bf23-1d220f758a51 service nova] Creating event network-vif-plugged:e4661ac3-e27b-495f-ae9c-f8a6c9653719 for instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 on nch2-master-1 Oct 15 21:29:11.751660 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-74af51fd-c505-4b49-895a-56a42733f2dd req-88697980-f291-4055-bf23-1d220f758a51 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037112 Oct 15 21:29:11.752500 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1618/5479] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:29:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:29:11.843043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:11.843702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:11.853685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:11.855051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:11.855323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:11.856311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:11.856790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:11.857267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:11.968744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:11.969498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:11.975962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:11.976713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:11.977367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:11.978309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:11.978986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:11.979620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:12.093584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:12.101652 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ecf044d-f118-484b-bb69-9dc5fe46df5d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1309 microversion: 2.1 time: 0.137821 Oct 15 21:29:12.102352 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1619/5480] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:11 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1309 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:12.121692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:12.129920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2b9a2f24-5cb0-44d7-ae92-4bce95ce8e23 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.291332 Oct 15 21:29:12.130501 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3862/5481] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:11 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:13.125198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:13.125812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:13.134371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:13.134764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:13.135415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:13.136798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:13.137283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:13.137713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:13.155758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:13.156687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:13.164027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:13.170046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:13.170635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:13.171145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:13.172469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:13.173012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:13.173578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:13.471430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:13.479257 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a01b449d-b045-498a-8cbc-5cfcb50367b4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1375 microversion: 2.1 time: 0.358064 Oct 15 21:29:13.479824 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1620/5482] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:13 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1375 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:13.498615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:13.507524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-150a8def-9fb3-4500-9203-9f56040c015d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1315 microversion: 2.1 time: 0.358097 Oct 15 21:29:13.508498 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3863/5483] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:13 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1315 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:29:13.891898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b59549d6-10a6-4265-b3d8-3dbc09c7d66f req-b2b0edda-1a27-482a-a265-50b3c7ad6dc3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "799b855f-2148-4b2b-9042-9417967f70dd", "tag": "8c33f6e7-1d90-4145-b758-c02da49c8ef1"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:13.904667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b59549d6-10a6-4265-b3d8-3dbc09c7d66f req-b2b0edda-1a27-482a-a265-50b3c7ad6dc3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:13.905449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b59549d6-10a6-4265-b3d8-3dbc09c7d66f req-b2b0edda-1a27-482a-a265-50b3c7ad6dc3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:13.906036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b59549d6-10a6-4265-b3d8-3dbc09c7d66f req-b2b0edda-1a27-482a-a265-50b3c7ad6dc3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:13.918122 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b59549d6-10a6-4265-b3d8-3dbc09c7d66f req-b2b0edda-1a27-482a-a265-50b3c7ad6dc3 service nova] Creating event network-changed:8c33f6e7-1d90-4145-b758-c02da49c8ef1 for instance 799b855f-2148-4b2b-9042-9417967f70dd on nch1-master-1 Oct 15 21:29:13.922821 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b59549d6-10a6-4265-b3d8-3dbc09c7d66f req-b2b0edda-1a27-482a-a265-50b3c7ad6dc3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035201 Oct 15 21:29:13.923430 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1621/5484] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:29:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:29:14.502641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:14.503253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:14.510156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.511319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.511785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.512475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.512931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.513385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.533091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:14.533854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:14.540075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.540566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.541013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.541744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.542209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.542641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.699964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:14.707858 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b62debc-c41c-447d-abaf-a9d374e830d1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1375 microversion: 2.1 time: 0.209012 Oct 15 21:29:14.708444 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3864/5485] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:14 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1375 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:14.725209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:14.733485 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b489ad6-cfb5-4dc4-b0f8-e769d0fe2901 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1501 microversion: 2.1 time: 0.205388 Oct 15 21:29:14.734057 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1622/5486] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:14 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1501 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:29:14.756038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:14.756662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:14.763334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.763841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.764294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.764997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.765487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.765923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.894918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:14.902857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5f16bb3c-f5a5-4929-b7bc-4299df151026 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1501 microversion: 2.1 time: 0.150766 Oct 15 21:29:14.903435 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3865/5487] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:14 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1501 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:14.925663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:14.926328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:14.932976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.933501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.933944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.934644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:14.935105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:14.935538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:14.958354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lazy-loading 'flavor' on Instance uuid ce54bbef-d609-4599-8ae5-e46b0fa64f30 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:29:15.024285 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-00718831-027d-4359-9053-10cbd677e665 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action" status: 202 len: 0 microversion: 2.1 time: 0.102289 Oct 15 21:29:15.024919 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1623/5488] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:29:14 2022] POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:29:15.033620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:15.034247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:15.040511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:15.041016 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:15.041515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:15.043012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:15.043509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:15.043949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:15.264722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:15.272608 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c5bb0f06-0dc8-45b9-9ae9-5078e63b8360 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1509 microversion: 2.1 time: 0.242917 Oct 15 21:29:15.273208 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3866/5489] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:15 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1509 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:15.733578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:15.734202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:15.740662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:15.741179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:15.741623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:15.742317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:15.742775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:15.743205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:15.887548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:15.895314 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-db39c28c-1df5-4f3a-aeca-512f9345c7b5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1375 microversion: 2.1 time: 0.167548 Oct 15 21:29:15.895875 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1624/5490] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:15 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1375 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:16.295526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:16.296135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:16.303173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.303697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.304123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.304820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.305316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.305745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.436102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:16.444138 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-98b93b62-bda4-46ea-b9c4-1a0a09481bc0 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1492 microversion: 2.1 time: 0.153162 Oct 15 21:29:16.444722 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3867/5491] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:16 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1492 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:16.467912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:16.468381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:16.474523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.475014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.475461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.476158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.476618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.477084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.542735 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ddca49a2-3bd4-4570-abf4-cf551d7e97b2 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action" status: 202 len: 0 microversion: 2.1 time: 0.079154 Oct 15 21:29:16.543359 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1625/5492] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:29:16 2022] POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:29:16.552175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:16.552888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:16.560680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.561278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.561720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.563130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.563590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.564384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.800014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:16.809590 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6fcf9f8-629a-444a-bbe9-cb9034affb66 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1498 microversion: 2.1 time: 0.261341 Oct 15 21:29:16.810261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3868/5493] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:16 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1498 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:16.920716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:16.921452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:16.927685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.928206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.928644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:16.929393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:16.929831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:16.930256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:17.161138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:17.170326 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3aec7446-6f1f-4390-b4af-fd0d1ffa6020 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1375 microversion: 2.1 time: 0.255235 Oct 15 21:29:17.170898 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1626/5494] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:16 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1375 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:17.831911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:17.832598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:17.840224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:17.840725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:17.841216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:17.841931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:17.842404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:17.842839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:17.983863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:17.988369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:17.995640 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58971932-48e2-4849-be80-a0caceaa0584 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1498 microversion: 2.1 time: 0.167182 Oct 15 21:29:17.996253 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3869/5495] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:17 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1498 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:18.195410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:18.196848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:18.206491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:18.207073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:18.207557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:18.208300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:18.208766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:18.209259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:18.359218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:18.367370 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a899dae6-df33-4884-97d6-7ca680ffe89d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1375 microversion: 2.1 time: 0.176782 Oct 15 21:29:18.367967 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1627/5496] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:18 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1375 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:18.809358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-aea98712-8ec8-4c2f-90f1-0aa9c04d57bd req-67771713-3ba0-45e8-aa2e-cb4ce8d1028d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "799b855f-2148-4b2b-9042-9417967f70dd", "name": "network-vif-plugged", "status": "completed", "tag": "8c33f6e7-1d90-4145-b758-c02da49c8ef1"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:18.820502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aea98712-8ec8-4c2f-90f1-0aa9c04d57bd req-67771713-3ba0-45e8-aa2e-cb4ce8d1028d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:18.820998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aea98712-8ec8-4c2f-90f1-0aa9c04d57bd req-67771713-3ba0-45e8-aa2e-cb4ce8d1028d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:18.821488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-aea98712-8ec8-4c2f-90f1-0aa9c04d57bd req-67771713-3ba0-45e8-aa2e-cb4ce8d1028d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:18.833201 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-aea98712-8ec8-4c2f-90f1-0aa9c04d57bd req-67771713-3ba0-45e8-aa2e-cb4ce8d1028d service nova] Creating event network-vif-plugged:8c33f6e7-1d90-4145-b758-c02da49c8ef1 for instance 799b855f-2148-4b2b-9042-9417967f70dd on nch1-master-1 Oct 15 21:29:18.838115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-aea98712-8ec8-4c2f-90f1-0aa9c04d57bd req-67771713-3ba0-45e8-aa2e-cb4ce8d1028d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033095 Oct 15 21:29:18.838724 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3870/5497] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:29:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:29:19.022345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:19.023073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:19.029400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.029827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.030267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.030960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.031414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.031838 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.170659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:19.178460 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-30756b79-072c-486b-bb09-caa83c6a0585 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1501 microversion: 2.1 time: 0.160050 Oct 15 21:29:19.179025 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1628/5498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:19 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1501 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:19.203050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:19.203510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:19.209960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.210447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.210883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.211575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.212030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.212456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.239106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lazy-loading 'flavor' on Instance uuid ce54bbef-d609-4599-8ae5-e46b0fa64f30 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:29:19.307124 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-803292b7-342c-49ae-92e0-d99ff8e5dca7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action" status: 202 len: 0 microversion: 2.1 time: 0.109447 Oct 15 21:29:19.308431 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3871/5499] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:29:19 2022] POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:29:19.316691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:19.317352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:19.324942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.325484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.325919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.326652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.327360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.327794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.392676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:19.393549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:19.399982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.400469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.400925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.401983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:19.402110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:19.402522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:19.489733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:19.499280 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-055836a5-bbb1-4ef4-b487-f4531194c1ca tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1509 microversion: 2.1 time: 0.186376 Oct 15 21:29:19.501552 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1629/5500] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:19 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1509 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:19.554010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:19.562035 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb90f3dc-1e05-4fc1-a15c-60e95ec6152d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1375 microversion: 2.1 time: 0.173989 Oct 15 21:29:19.562544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3872/5501] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:19 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1375 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:29:20.522575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:20.523213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:20.530842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.531329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.531763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.532466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.532920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.533377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.585385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:20.586127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:20.593869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.594441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.594959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.595739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.596279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.596787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.729651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:20.763288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:20.825358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:20.828975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:20.834274 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-12b548d1-07a7-4834-83bf-0173c32975f6 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1492 microversion: 2.1 time: 0.315640 Oct 15 21:29:20.834872 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1630/5502] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:20 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1492 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:20.858656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:20.859209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:20.866513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.867022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.867480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.868252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.868739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.869323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.879759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:20.888310 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97e5b5ae-da02-48b6-8ffc-718f93b582e1 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1548 microversion: 2.1 time: 0.306853 Oct 15 21:29:20.889237 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3873/5503] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:20 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1548 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:20.914030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:20.915536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:20.923228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.923773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.924592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.925861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.926558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.927544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.945029 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cee1341a-9fdf-4063-91c9-333771a317ef tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action" status: 202 len: 0 microversion: 2.1 time: 0.090507 Oct 15 21:29:20.949080 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1631/5504] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:29:20 2022] POST /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30/action => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:29:20.949860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:20.955754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:20.956712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:20.963573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.964229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.964799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:20.965657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:20.966142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:20.966592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:21.236486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:21.244560 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2bc8f7e-6828-4407-9fec-4d7daa6f55d4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1548 microversion: 2.1 time: 0.335953 Oct 15 21:29:21.246510 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3874/5505] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:20 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1548 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:29:21.249114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:21.257225 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-462808e9-91dd-4cce-a975-9dbac4dee68d tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1498 microversion: 2.1 time: 0.305643 Oct 15 21:29:21.258079 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1632/5506] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:20 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1498 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:29:22.282189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:22.282829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:22.285649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:22.291064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:22.291558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:22.292638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:22.293405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:22.293871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:22.294298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:22.436302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:22.444847 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e29e6b14-037e-4ab0-bf44-0a3413c3c1ea tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1498 microversion: 2.1 time: 0.166461 Oct 15 21:29:22.445725 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3875/5507] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:22 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1498 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:22.961911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-618b0c31-2580-42b2-a663-caa077c4b75a req-3ae83d60-3b07-4be2-b39f-70f34a0b7948 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "799b855f-2148-4b2b-9042-9417967f70dd", "tag": "8c33f6e7-1d90-4145-b758-c02da49c8ef1"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:22.972014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-618b0c31-2580-42b2-a663-caa077c4b75a req-3ae83d60-3b07-4be2-b39f-70f34a0b7948 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:22.972439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-618b0c31-2580-42b2-a663-caa077c4b75a req-3ae83d60-3b07-4be2-b39f-70f34a0b7948 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:22.972879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-618b0c31-2580-42b2-a663-caa077c4b75a req-3ae83d60-3b07-4be2-b39f-70f34a0b7948 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:22.984546 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-618b0c31-2580-42b2-a663-caa077c4b75a req-3ae83d60-3b07-4be2-b39f-70f34a0b7948 service nova] Creating event network-changed:8c33f6e7-1d90-4145-b758-c02da49c8ef1 for instance 799b855f-2148-4b2b-9042-9417967f70dd on nch1-master-1 Oct 15 21:29:22.989684 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-618b0c31-2580-42b2-a663-caa077c4b75a req-3ae83d60-3b07-4be2-b39f-70f34a0b7948 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106361 Oct 15 21:29:22.990251 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1633/5508] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:29:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:29:23.470449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:23.471574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:23.478770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:23.479273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:23.480312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:23.481252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:23.481726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:23.482170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:23.695691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:23.705141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36be0835-24e8-4a37-a056-48d78af426e3 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1501 microversion: 2.1 time: 0.239088 Oct 15 21:29:23.722937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3876/5509] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:23 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1501 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:23.728877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:23.729604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:23.736277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:23.736782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:23.737264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:23.737983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:23.738621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:23.739063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:23.768125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:29:23.864001 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b36bb7cd-17b8-42b0-ad47-6f52dbf2a33c tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 204 len: 0 microversion: 2.1 time: 0.139034 Oct 15 21:29:23.864724 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1634/5510] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:29:23 2022] DELETE /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:29:23.873868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:23.874653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:23.880569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:23.881163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:23.881765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:23.882608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:23.883224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:23.883899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:24.057139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:24.065843 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-da223143-57a4-46fb-9002-4b5c67c8fee7 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1507 microversion: 2.1 time: 0.196334 Oct 15 21:29:24.066434 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3877/5511] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:23 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1507 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:25.089679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:25.090382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:25.097388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:25.097885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:25.098325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:25.099041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:25.099501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:25.099928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:25.233980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:25.242088 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7efedcec-c7a2-44c0-9acd-c65fe1f2159b tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1507 microversion: 2.1 time: 0.156808 Oct 15 21:29:25.242650 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1635/5512] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:25 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1507 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:26.266401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:26.267156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:26.281522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:26.282025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:26.282472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:26.283183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:26.283647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:26.284079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:26.341344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b26bc600-fec9-4f5f-9afd-fee63b7c73da req-2dbea4a9-09ee-45ca-911b-e0717571e2ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ce54bbef-d609-4599-8ae5-e46b0fa64f30", "name": "network-vif-deleted", "tag": "e4661ac3-e27b-495f-ae9c-f8a6c9653719"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:26.352453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b26bc600-fec9-4f5f-9afd-fee63b7c73da req-2dbea4a9-09ee-45ca-911b-e0717571e2ce service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:26.353290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b26bc600-fec9-4f5f-9afd-fee63b7c73da req-2dbea4a9-09ee-45ca-911b-e0717571e2ce service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:26.353752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b26bc600-fec9-4f5f-9afd-fee63b7c73da req-2dbea4a9-09ee-45ca-911b-e0717571e2ce service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:26.366593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3df1b449adeb42c58878a8344c9ddd66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82cb785a32d473db8e973a212715528', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:26.368702 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b26bc600-fec9-4f5f-9afd-fee63b7c73da req-2dbea4a9-09ee-45ca-911b-e0717571e2ce service nova] Creating event network-vif-deleted:e4661ac3-e27b-495f-ae9c-f8a6c9653719 for instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 on nch2-master-1 Oct 15 21:29:26.374922 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b26bc600-fec9-4f5f-9afd-fee63b7c73da req-2dbea4a9-09ee-45ca-911b-e0717571e2ce service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038342 Oct 15 21:29:26.375756 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1636/5513] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:29:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:29:26.376529 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-884e1b50-1211-48c1-9e29-7b937bda5c7f tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 200 len: 1465 microversion: 2.1 time: 0.114103 Oct 15 21:29:26.383318 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3878/5514] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:26 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 1465 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:27.402909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:27.403598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] [instance: ce54bbef-d609-4599-8ae5-e46b0fa64f30] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:27.409833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:27.410321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:27.410754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:27.411442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:27.411896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:27.412320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:27.425526 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] HTTP exception thrown: Instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 could not be found. Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Request method failure captured: Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 HTTP/1.1 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyYcdJzk0zyOFAKHuL_9BCaGam7djYUPOA1QCMewW8pFrRLHV-obx0UswpKlyRPwm7Nb_q4_P03nKU7iKZJZAgQmNallSWw3CSK7OfGVveWe9CeDU11CRvQJT4GOoqcw5sYwdw-VibiHiGlb7qFNqS0k0mqESgsWmNmPpu11mosESe0 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: a82cb785a32d473db8e973a212715528 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestServerAdvancedOps-289620579 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a82cb785a32d473db8e973a212715528"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a82cb785a32d473db8e973a212715528"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a82cb785a32d473db8e973a212715528"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a82cb785a32d473db8e973a212715528"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestServerAdvancedOps-289620579 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: a82cb785a32d473db8e973a212715528 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestServerAdvancedOps-289620579 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestServerAdvancedOps-289620579-project Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 3df1b449adeb42c58878a8344c9ddd66 Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestServerAdvancedOps-289620579-project Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 could not be found. Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ce54bbef-d609-4599-8ae5-e46b0fa64f30'} Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 could not be found. Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 could not be found. Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:29:27.427115 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 could not be found. Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] Returning 404 to user: Instance ce54bbef-d609-4599-8ae5-e46b0fa64f30 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c1bc13d2-7500-48d8-a0cf-56794ee16551 tempest-TestServerAdvancedOps-289620579 tempest-TestServerAdvancedOps-289620579-project] 10.222.0.16 "GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30" status: 404 len: 111 microversion: 2.1 time: 0.031087 Oct 15 21:29:27.431275 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1637/5515] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:27 2022] GET /compute/v2.1/servers/ce54bbef-d609-4599-8ae5-e46b0fa64f30 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:29:45.206021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:45.209405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:45.217617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:45.218815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:45.223619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:45.224232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:45.224684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:45.228764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:45.229792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:45.229912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:45.291141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4a5682a-aa03-4c5f-b50d-bab066702b21 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd/action" status: 200 len: 24570 microversion: 2.1 time: 0.090144 Oct 15 21:29:45.297112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3879/5516] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:29:45 2022] POST /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd/action => generated 24570 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:29:56.716677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a260a6d4-64cb-4340-97e4-5601a50ec57d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'create', calling method: .version_select at 0x7fe74f152280>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-1196937076", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJjYml9EvL04qMWFEwbuwOsW18NYYBVw5JukhVnMjf2jK9qkvQnSFvHf+gtEfBzl8t/KoFZiX22wswfexQolhqdvQguaBeq07jYoTJzL3+AhICjHPxG3KUu3n7MpJXYJZw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:56.725370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:56.726025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:56.738805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a260a6d4-64cb-4340-97e4-5601a50ec57d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:56.745370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a260a6d4-64cb-4340-97e4-5601a50ec57d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for user bd14dd0707a1423d82328ac3a54ccb28 and project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:56.785454 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a260a6d4-64cb-4340-97e4-5601a50ec57d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.072546 Oct 15 21:29:56.785992 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1638/5517] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:29:56 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:29:57.572112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f13c990-ce06-47e9-9b71-afe54a729e98 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Action: 'create', calling method: .version_select at 0x7fe74e1f5a60>, body: {"keypair": {"name": "tempest-TestVolumeBackupRestore-1285221332", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLiAh/7MqZo1MgC8ZzhCpdQ74xWyVnkcF05nRkDBAa4jfnDEBGJAFk25RPqV4XZ7IAKnALoJdkohSS6HlVLeIdd9URB+h+DFQk0HXAfJTWM+VQtdjtdgYzN4fmI5h5irDw=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:57.583247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:57.584338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:29:57.602150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0f13c990-ce06-47e9-9b71-afe54a729e98 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Getting quotas for project 6c1026ed03744eaaa999b2d54aac2250. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:57.608505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0f13c990-ce06-47e9-9b71-afe54a729e98 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Getting quotas for user 32a68a0c1d8f45149c42699b2367f108 and project 6c1026ed03744eaaa999b2d54aac2250. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:57.643467 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f13c990-ce06-47e9-9b71-afe54a729e98 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.077186 Oct 15 21:29:57.644273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3880/5518] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:29:57 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:29:58.051539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-994811612", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "536f394a-2e49-4fbf-9126-784a99aaa5b5"}], "key_name": "tempest-TestNetworkBasicOps-1196937076", "security_groups": [{"name": "tempest-secgroup-smoke-175236118"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:58.175865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] validate_networks() for [('536f394a-2e49-4fbf-9126-784a99aaa5b5', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:29:58.710763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:29:58.711442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:29:58.712700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:29:58.713603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:29:58.714240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:29:58.725907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:58.726982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:58.728082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:58.729253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:58.729942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:58.730716 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:58.753560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:58.759903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for user bd14dd0707a1423d82328ac3a54ccb28 and project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:58.774265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:29:58.778229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:29:58.838602 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-741efc56-3df6-499b-a913-918762317963 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.792706 Oct 15 21:29:58.851524 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1639/5519] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:29:58 2022] POST /compute/v2.1/servers => generated 405 bytes in 806 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:29:58.861094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82d1539e-e17c-4e4f-9068-e44831cbd4df tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:58.861094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82d1539e-e17c-4e4f-9068-e44831cbd4df tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:58.915413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBackupRestore-server-2066789415", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "59613ed5-fb16-47b8-811f-121321d64310", "source_type": "volume", "destination_type": "volume", "boot_index": 0}], "key_name": "tempest-TestVolumeBackupRestore-1285221332", "security_groups": [{"name": "tempest-secgroup-smoke-130339162"}], "networks": [{"uuid": "43f5555d-a1b0-4ab2-bed0-40a349ad9a6d"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:29:58.999383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82d1539e-e17c-4e4f-9068-e44831cbd4df tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:59.006703 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82d1539e-e17c-4e4f-9068-e44831cbd4df tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1248 microversion: 2.1 time: 0.159025 Oct 15 21:29:59.007284 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3881/5520] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:58 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1248 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:29:59.087474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] validate_networks() for [('43f5555d-a1b0-4ab2-bed0-40a349ad9a6d', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:29:59.532567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:29:59.533001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:29:59.533537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:29:59.533946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:29:59.534528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:29:59.545454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:59.546142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:59.546591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:59.547357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:29:59.547813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:29:59.548410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:29:59.566431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Getting quotas for project 6c1026ed03744eaaa999b2d54aac2250. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:29:59.571736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Getting quotas for user 32a68a0c1d8f45149c42699b2367f108 and project 6c1026ed03744eaaa999b2d54aac2250. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:29:59.633961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:29:59.636002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='59613ed5-fb16-47b8-811f-121321d64310',volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:29:59.778699 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-411945b4-8ad5-444a-aa92-3b47649b5f51 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.867918 Oct 15 21:29:59.786175 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1640/5521] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:29:58 2022] POST /compute/v2.1/servers => generated 405 bytes in 870 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:29:59.791674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21f1a5f5-7b57-485a-999f-719f06a75353 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:29:59.792545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21f1a5f5-7b57-485a-999f-719f06a75353 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:29:59.848892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-21f1a5f5-7b57-485a-999f-719f06a75353 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:29:59.857985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21f1a5f5-7b57-485a-999f-719f06a75353 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1099 microversion: 2.1 time: 0.070363 Oct 15 21:29:59.858572 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3882/5522] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:29:59 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1099 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:00.030594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:00.031864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:00.040061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:00.040641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:00.041601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:00.042993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:00.043911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:00.044943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:00.150867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:00.162052 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4dcf6626-b474-44d8-bb05-a3ef8a648db9 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1308 microversion: 2.1 time: 0.135388 Oct 15 21:30:00.166410 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1641/5523] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:00 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1308 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:00.887669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:00.888295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:00.899354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:00.901405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:00.901883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:00.902711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:00.903200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:00.905437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:01.041449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:01.051168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9e1692c-d7d5-46ad-ac66-cd137fc80397 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1215 microversion: 2.1 time: 0.168925 Oct 15 21:30:01.052061 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3883/5524] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:00 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1215 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:01.190874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:01.191503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:01.199791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:01.203154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:01.203737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:01.208576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:01.209166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:01.209772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:01.529189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:01.539075 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-705c95f8-c225-4e57-bf15-3a8d37d9fe81 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.353705 Oct 15 21:30:01.540224 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1642/5525] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:01 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:02.075404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:02.076660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:02.085535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:02.086202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:02.086736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:02.087693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:02.088342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:02.089026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:02.180383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:02.188848 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3afd3e0a-aeb9-4008-918e-8a81965e27fb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1215 microversion: 2.1 time: 0.117524 Oct 15 21:30:02.195281 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3884/5526] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:02 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1215 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:02.563911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:02.564564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:02.571416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:02.571906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:02.572375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:02.573111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:02.573574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:02.573996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:02.861988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:02.870591 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7a81d449-109f-4a28-80f2-0d853d9948aa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.310627 Oct 15 21:30:02.871256 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1643/5527] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:02 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:03.166728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f0d64fd2-2489-423a-97f1-d8aeff360803 req-784632e1-076b-4471-bc51-eef39932ba07 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0eafe785-7c8f-4e6f-a885-38105a181b8c", "tag": "135f84b6-a1f3-4fc2-b820-49b5c2c80ee7"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:03.176376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f0d64fd2-2489-423a-97f1-d8aeff360803 req-784632e1-076b-4471-bc51-eef39932ba07 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:03.176864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f0d64fd2-2489-423a-97f1-d8aeff360803 req-784632e1-076b-4471-bc51-eef39932ba07 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:03.177322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f0d64fd2-2489-423a-97f1-d8aeff360803 req-784632e1-076b-4471-bc51-eef39932ba07 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:03.188747 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f0d64fd2-2489-423a-97f1-d8aeff360803 req-784632e1-076b-4471-bc51-eef39932ba07 service nova] Creating event network-changed:135f84b6-a1f3-4fc2-b820-49b5c2c80ee7 for instance 0eafe785-7c8f-4e6f-a885-38105a181b8c on nch2-master-1 Oct 15 21:30:03.194972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f0d64fd2-2489-423a-97f1-d8aeff360803 req-784632e1-076b-4471-bc51-eef39932ba07 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.163031 Oct 15 21:30:03.195746 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3885/5528] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:03.212514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:03.213351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:03.219403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:03.219914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:03.220340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:03.221139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:03.221743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:03.222183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:03.516310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:03.524309 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2fac5c57-440b-49ff-a48b-1f8a1caecbfb tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.315770 Oct 15 21:30:03.524980 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1644/5529] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:03 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:03.894533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:03.895452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:03.901731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:03.902236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:03.902689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:03.903405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:03.903874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:03.904311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:04.139447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:04.148272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3acc08cb-84be-43f7-ba4e-f8e0f88a1896 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.258061 Oct 15 21:30:04.155885 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3886/5530] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:03 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:04.547813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:04.548432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:04.555960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:04.556584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:04.557061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:04.557749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:04.558205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:04.558633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:04.695336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:04.703138 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-986aa1ff-b101-4464-a6c6-bec3117f694c tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.159268 Oct 15 21:30:04.703703 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1645/5531] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:04 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:05.172029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:05.172744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:05.179680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:05.180165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:05.180603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:05.181318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:05.181776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:05.182202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:05.214609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5d6616ed-80ea-45b0-a92f-c90af6aa2198 req-f10f0317-6ea4-4eb9-8c0e-d1b452b828f1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:05.225096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5d6616ed-80ea-45b0-a92f-c90af6aa2198 req-f10f0317-6ea4-4eb9-8c0e-d1b452b828f1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:05.225614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5d6616ed-80ea-45b0-a92f-c90af6aa2198 req-f10f0317-6ea4-4eb9-8c0e-d1b452b828f1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:05.226139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5d6616ed-80ea-45b0-a92f-c90af6aa2198 req-f10f0317-6ea4-4eb9-8c0e-d1b452b828f1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:05.237429 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5d6616ed-80ea-45b0-a92f-c90af6aa2198 req-f10f0317-6ea4-4eb9-8c0e-d1b452b828f1 service nova] Creating event network-changed:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:30:05.242498 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5d6616ed-80ea-45b0-a92f-c90af6aa2198 req-f10f0317-6ea4-4eb9-8c0e-d1b452b828f1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031551 Oct 15 21:30:05.243350 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1646/5532] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:30:05.378131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:05.386115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1c653e3e-e4b7-418a-a048-7252f8984b3d tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.218216 Oct 15 21:30:05.386776 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3887/5533] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:05 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:30:05.727396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:05.728013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:05.734301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:05.734785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:05.735219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:05.735940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:05.736396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:05.737458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:05.976212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:05.984107 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65d42349-3ce0-456a-a160-09d8d43ccec7 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.261811 Oct 15 21:30:05.984618 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1647/5534] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:05 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:06.410601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:06.411252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:06.418591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:06.419089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:06.419527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:06.420238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:06.420699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:06.421157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:06.751627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:06.760072 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fd127d5c-1cf5-4301-ab86-0a037861c1be tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.353367 Oct 15 21:30:06.760578 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3888/5535] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:06 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:07.011184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:07.011710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:07.019142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:07.019639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:07.020088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:07.020785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:07.021278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:07.021715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:07.154640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:07.163194 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b7fc38bb-59ee-4a39-b6fe-ddc09b204b58 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.159610 Oct 15 21:30:07.163760 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1648/5536] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:07 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:07.783173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:07.783962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:07.791071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:07.791559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:07.791997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:07.792696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:07.793179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:07.793609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:07.929560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:07.938154 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3f08ed48-f7b8-48fb-aa3c-ea22340522a0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.158949 Oct 15 21:30:07.938856 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3889/5537] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:07 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:08.186889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:08.187506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:08.195576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:08.196134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:08.196761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:08.198311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:08.198935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:08.199374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:08.338487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:08.346706 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-22ede075-cd3e-4dc9-a3ca-140eb897d330 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.163404 Oct 15 21:30:08.347366 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1649/5538] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:08 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:08.963865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:08.963865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:08.970056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:08.970542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:08.971548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:08.972253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:08.972710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:08.973348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:09.120231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:09.128937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1f5be0e-354d-4d28-9f56-44726e81422e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.170191 Oct 15 21:30:09.129544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3890/5539] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:08 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:09.370159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:09.370739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:09.378786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:09.379312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:09.379755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:09.380466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:09.380926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:09.381388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:09.517176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:09.525070 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa3da833-d167-4fda-8493-5b332f6b3cb9 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.159139 Oct 15 21:30:09.525635 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1650/5540] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:09 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:09.873286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:09.938879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-26180540-b7d2-437f-848a-9052bbba3c23 req-f27837c4-de84-4324-a156-be2e39a81b19 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0eafe785-7c8f-4e6f-a885-38105a181b8c", "name": "network-vif-plugged", "status": "completed", "tag": "135f84b6-a1f3-4fc2-b820-49b5c2c80ee7"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:09.949475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-26180540-b7d2-437f-848a-9052bbba3c23 req-f27837c4-de84-4324-a156-be2e39a81b19 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:09.949957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-26180540-b7d2-437f-848a-9052bbba3c23 req-f27837c4-de84-4324-a156-be2e39a81b19 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:09.950387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-26180540-b7d2-437f-848a-9052bbba3c23 req-f27837c4-de84-4324-a156-be2e39a81b19 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:09.961344 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-26180540-b7d2-437f-848a-9052bbba3c23 req-f27837c4-de84-4324-a156-be2e39a81b19 service nova] Creating event network-vif-plugged:135f84b6-a1f3-4fc2-b820-49b5c2c80ee7 for instance 0eafe785-7c8f-4e6f-a885-38105a181b8c on nch2-master-1 Oct 15 21:30:09.966466 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-26180540-b7d2-437f-848a-9052bbba3c23 req-f27837c4-de84-4324-a156-be2e39a81b19 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097928 Oct 15 21:30:09.967041 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3891/5541] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:10.153144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:10.153761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:10.155720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:10.156381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:10.164280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:10.164632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:10.165387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:10.166114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:10.166575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:10.167007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:10.298761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:10.306564 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d1df0f41-05e2-4ec1-b9ff-aa36b7de26f4 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.157296 Oct 15 21:30:10.307134 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1651/5542] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:10 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:10.548233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:10.548947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:10.556984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:10.557499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:10.557938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:10.558633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:10.559098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:10.559528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:10.690243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:10.697916 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc3d40ae-64ee-4cf4-83c5-6bdd0b361984 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.153948 Oct 15 21:30:10.698484 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3892/5543] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:10 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:11.330136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:11.330859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:11.337759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:11.338256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:11.338705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:11.339403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:11.339866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:11.340300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:11.483346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:11.491685 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dbf65b98-f264-4719-9fd2-6aedba15f160 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1373 microversion: 2.1 time: 0.165498 Oct 15 21:30:11.492250 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1652/5544] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:11 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1373 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:11.720607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:11.721808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:11.728599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:11.729125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:11.729564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:11.730257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:11.730767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:11.731140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:11.863585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:11.871698 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e61a2004-b318-4d6f-a39d-6ea644405b99 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.154961 Oct 15 21:30:11.872267 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3893/5545] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:11 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:11.981445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-dfa90ec0-557c-49d9-b8bb-ed0c11773fc5 req-581b0778-1b7d-4578-98b6-ea2c26e2f0a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "name": "network-vif-plugged", "status": "completed", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:11.991460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dfa90ec0-557c-49d9-b8bb-ed0c11773fc5 req-581b0778-1b7d-4578-98b6-ea2c26e2f0a0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:11.991951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dfa90ec0-557c-49d9-b8bb-ed0c11773fc5 req-581b0778-1b7d-4578-98b6-ea2c26e2f0a0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:11.992400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dfa90ec0-557c-49d9-b8bb-ed0c11773fc5 req-581b0778-1b7d-4578-98b6-ea2c26e2f0a0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:12.005187 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-dfa90ec0-557c-49d9-b8bb-ed0c11773fc5 req-581b0778-1b7d-4578-98b6-ea2c26e2f0a0 service nova] Creating event network-vif-plugged:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:30:12.010197 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-dfa90ec0-557c-49d9-b8bb-ed0c11773fc5 req-581b0778-1b7d-4578-98b6-ea2c26e2f0a0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032459 Oct 15 21:30:12.010793 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1653/5546] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:12.515066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:12.515687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:12.522544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:12.523058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:12.523505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:12.524231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:12.524688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:12.525158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:12.657198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:12.665534 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-24f6b817-f119-4598-87da-d289e3eb7eb0 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1546 microversion: 2.1 time: 0.154436 Oct 15 21:30:12.666112 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3894/5547] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:12 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1546 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:12.689127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:12.689868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:12.696039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:12.696527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:12.696972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:12.697711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:12.698173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:12.698604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:12.835634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:12.843452 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d402d74b-65a8-4d06-ae4c-8ad8f32877fa tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1546 microversion: 2.1 time: 0.159354 Oct 15 21:30:12.844022 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1654/5548] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:12 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1546 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:12.895346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:12.896082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:12.903366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:12.903847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:12.904282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:12.905079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:12.905460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:12.905889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:13.052979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:13.060905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95625c74-1970-4d9a-9aad-904e7ccce10d tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1270 microversion: 2.1 time: 0.169645 Oct 15 21:30:13.061502 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3895/5549] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:12 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1270 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:14.086403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:14.087032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:14.093991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:14.094486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:14.094930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:14.095633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:14.096737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:14.097303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:14.226320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:14.234123 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3bce8579-b4e5-4ebb-bc00-d2c0c4d648d3 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1458 microversion: 2.1 time: 0.151639 Oct 15 21:30:14.234691 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1655/5550] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:14 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1458 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:14.256144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:14.256757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:14.263594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:14.264072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:14.264507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:14.265260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:14.265716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:14.266138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:14.402520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:14.410496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0b067cf-bbf7-45bb-92ed-ba07a164eb6f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1458 microversion: 2.1 time: 0.158165 Oct 15 21:30:14.411076 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3896/5551] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:14 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1458 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:16.287849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5e47954b-8291-4e56-a49a-d2806a335d9e req-166918c8-3ee1-4e66-90b1-7435297fe8ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:16.302340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5e47954b-8291-4e56-a49a-d2806a335d9e req-166918c8-3ee1-4e66-90b1-7435297fe8ef service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:16.303228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5e47954b-8291-4e56-a49a-d2806a335d9e req-166918c8-3ee1-4e66-90b1-7435297fe8ef service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:16.304022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5e47954b-8291-4e56-a49a-d2806a335d9e req-166918c8-3ee1-4e66-90b1-7435297fe8ef service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:16.322583 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5e47954b-8291-4e56-a49a-d2806a335d9e req-166918c8-3ee1-4e66-90b1-7435297fe8ef service nova] Creating event network-changed:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:30:16.329824 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5e47954b-8291-4e56-a49a-d2806a335d9e req-166918c8-3ee1-4e66-90b1-7435297fe8ef service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057351 Oct 15 21:30:16.331722 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1656/5552] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:18.393483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}, {"name": "network-changed", "server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:18.404861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:18.405478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:18.405920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:18.418699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] Creating event network-changed:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:30:18.419501 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] Creating event network-changed:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:30:18.425535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2eb68a26-d231-47f4-bb34-8337e03a469e req-f321a3db-49cb-4fbd-b0fa-36f8e5888e55 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.035750 Oct 15 21:30:18.426154 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3897/5553] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:18 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:30:47.414764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=7 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:47.415526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=88 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:47.420005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:47.423431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:47.432844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:47.433411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:47.433874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:47.434958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:47.435526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:47.436175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:47.472275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:30:47.571601 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-04f9a5a1-6a23-4637-a359-5e81dbc34bda tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 204 len: 0 microversion: 2.1 time: 0.159005 Oct 15 21:30:47.576154 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1657/5554] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:30:47 2022] DELETE /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:30:47.589468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:47.590946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:47.594458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:47.595247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=78 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:30:47.603852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:47.604402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:47.604880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:47.605668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:47.607293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:47.609695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:47.764920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:47.774190 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ad6982d-681b-46b2-896f-c9839bae13ee tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1552 microversion: 2.1 time: 0.195502 Oct 15 21:30:47.775148 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3898/5555] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:47 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1552 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:48.798704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:48.799380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:48.806847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:48.807420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:48.807863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:48.808598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:48.809074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:48.809508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:49.168257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:49.177776 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bc468b4b-39db-4f99-bcb5-520d05e77794 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 200 len: 1552 microversion: 2.1 time: 0.383609 Oct 15 21:30:49.187051 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1658/5556] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:48 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 1552 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:49.574110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-fbe82af8-1d6d-451c-b92b-28cf46906175 req-65e15f9b-7c40-4c16-ae92-f2dd66669492 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0eafe785-7c8f-4e6f-a885-38105a181b8c", "name": "network-vif-deleted", "tag": "135f84b6-a1f3-4fc2-b820-49b5c2c80ee7"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:49.588577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fbe82af8-1d6d-451c-b92b-28cf46906175 req-65e15f9b-7c40-4c16-ae92-f2dd66669492 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:49.589207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fbe82af8-1d6d-451c-b92b-28cf46906175 req-65e15f9b-7c40-4c16-ae92-f2dd66669492 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:49.589586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-fbe82af8-1d6d-451c-b92b-28cf46906175 req-65e15f9b-7c40-4c16-ae92-f2dd66669492 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:49.602676 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-fbe82af8-1d6d-451c-b92b-28cf46906175 req-65e15f9b-7c40-4c16-ae92-f2dd66669492 service nova] Creating event network-vif-deleted:135f84b6-a1f3-4fc2-b820-49b5c2c80ee7 for instance 0eafe785-7c8f-4e6f-a885-38105a181b8c on nch2-master-1 Oct 15 21:30:49.608206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-fbe82af8-1d6d-451c-b92b-28cf46906175 req-65e15f9b-7c40-4c16-ae92-f2dd66669492 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038699 Oct 15 21:30:49.608852 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3899/5557] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:50.200501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:50.201278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 0eafe785-7c8f-4e6f-a885-38105a181b8c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:50.208708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:50.209231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:50.209730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:50.210529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:50.211064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:50.211531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:50.226937 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] HTTP exception thrown: Instance 0eafe785-7c8f-4e6f-a885-38105a181b8c could not be found. Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Request method failure captured: Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c HTTP/1.1 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyYdyovFJVhQ-DTS2g0_kJPK9ucgBPejohisBcx8zlugdRuQhRrmo0Fhc5E1I7x_wwx-1WoMpg2pnOOMM0abzemR_uyhtjPvIN8FLH0dBhkKFUyQ-q-RRQSmLbXAayx6JUL6wVLgJrt-1Nbzk3kxzgPfU7q5W0S8C_zPfX_zYYU5rdo Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e2bfe1e4fbfd48998884d5e8ef8a3e57 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e2bfe1e4fbfd48998884d5e8ef8a3e57 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestNetworkBasicOps-1261870155-project Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: bd14dd0707a1423d82328ac3a54ccb28 Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestNetworkBasicOps-1261870155-project Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 0eafe785-7c8f-4e6f-a885-38105a181b8c could not be found. Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '0eafe785-7c8f-4e6f-a885-38105a181b8c'} Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0eafe785-7c8f-4e6f-a885-38105a181b8c could not be found. Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0eafe785-7c8f-4e6f-a885-38105a181b8c could not be found. Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:30:50.228934 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0eafe785-7c8f-4e6f-a885-38105a181b8c could not be found. Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Returning 404 to user: Instance 0eafe785-7c8f-4e6f-a885-38105a181b8c could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1523dff8-22ee-42b1-8f18-abfffb9c94b2 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c" status: 404 len: 111 microversion: 2.1 time: 0.034274 Oct 15 21:30:50.233328 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1659/5558] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:50 2022] GET /compute/v2.1/servers/0eafe785-7c8f-4e6f-a885-38105a181b8c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:30:50.415555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-87360474-9ac9-4c46-812f-48a679c818ba tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '.version_select at 0x7fe74f12d280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:50.438272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-87360474-9ac9-4c46-812f-48a679c818ba tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-1196937076" status: 202 len: 0 microversion: 2.1 time: 0.027174 Oct 15 21:30:50.439148 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3900/5559] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:30:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-1196937076 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:30:52.509792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:52.515948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0e6c72e6-fce6-49a4-87b0-b8adb58dbe2b req-165c5552-d1c8-4fb0-9bf2-b48fe6780531 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "799b855f-2148-4b2b-9042-9417967f70dd", "tag": "8c33f6e7-1d90-4145-b758-c02da49c8ef1"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:52.516659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:52.524808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:52.525565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:52.526084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:52.526905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:52.527190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e6c72e6-fce6-49a4-87b0-b8adb58dbe2b req-165c5552-d1c8-4fb0-9bf2-b48fe6780531 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:52.527512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:52.527807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e6c72e6-fce6-49a4-87b0-b8adb58dbe2b req-165c5552-d1c8-4fb0-9bf2-b48fe6780531 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:52.528089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:52.528529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0e6c72e6-fce6-49a4-87b0-b8adb58dbe2b req-165c5552-d1c8-4fb0-9bf2-b48fe6780531 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:52.540309 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0e6c72e6-fce6-49a4-87b0-b8adb58dbe2b req-165c5552-d1c8-4fb0-9bf2-b48fe6780531 service nova] Creating event network-changed:8c33f6e7-1d90-4145-b758-c02da49c8ef1 for instance 799b855f-2148-4b2b-9042-9417967f70dd on nch1-master-1 Oct 15 21:30:52.546394 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0e6c72e6-fce6-49a4-87b0-b8adb58dbe2b req-165c5552-d1c8-4fb0-9bf2-b48fe6780531 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034718 Oct 15 21:30:52.549452 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3901/5560] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:52.570712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:30:52.683255 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5afb6bb-e9a5-4f17-a22e-603fdda7fb88 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "DELETE /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 204 len: 0 microversion: 2.1 time: 0.177050 Oct 15 21:30:52.684501 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1660/5561] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:30:52 2022] DELETE /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:30:52.693819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:52.694542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:52.700856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:52.701417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:52.701868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:52.702582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:52.703055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:52.703485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:52.843991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:52.851893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a5c36837-a7ec-4bca-ad73-ca52267b8269 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1673 microversion: 2.1 time: 0.163012 Oct 15 21:30:52.852468 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3902/5562] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:52 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1673 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:53.876616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:53.877466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:53.884611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:53.885170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:53.885672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:53.886464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:53.886938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:53.887372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:54.203316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:30:54.211676 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e8d71a20-fbe8-41b0-aa17-cbf7add94c58 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 200 len: 1673 microversion: 2.1 time: 0.339440 Oct 15 21:30:54.212445 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1661/5563] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:53 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 1673 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:30:54.613487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-70996d1d-1756-4bc6-a3b1-a434b661ac20 req-af989070-d87d-4a25-9efc-d7083e2b5984 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "799b855f-2148-4b2b-9042-9417967f70dd", "name": "network-vif-deleted", "tag": "8c33f6e7-1d90-4145-b758-c02da49c8ef1"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:54.624803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-70996d1d-1756-4bc6-a3b1-a434b661ac20 req-af989070-d87d-4a25-9efc-d7083e2b5984 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:54.625403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-70996d1d-1756-4bc6-a3b1-a434b661ac20 req-af989070-d87d-4a25-9efc-d7083e2b5984 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:54.625918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-70996d1d-1756-4bc6-a3b1-a434b661ac20 req-af989070-d87d-4a25-9efc-d7083e2b5984 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:54.639083 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-70996d1d-1756-4bc6-a3b1-a434b661ac20 req-af989070-d87d-4a25-9efc-d7083e2b5984 service nova] Creating event network-vif-deleted:8c33f6e7-1d90-4145-b758-c02da49c8ef1 for instance 799b855f-2148-4b2b-9042-9417967f70dd on nch1-master-1 Oct 15 21:30:54.643998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-70996d1d-1756-4bc6-a3b1-a434b661ac20 req-af989070-d87d-4a25-9efc-d7083e2b5984 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034819 Oct 15 21:30:54.644603 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3903/5564] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:30:55.235014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:55.235946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 799b855f-2148-4b2b-9042-9417967f70dd] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:55.242878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:55.243372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:55.243812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:55.244511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:55.244971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:55.245428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:55.259276 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] HTTP exception thrown: Instance 799b855f-2148-4b2b-9042-9417967f70dd could not be found. Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Request method failure captured: Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd HTTP/1.1 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyYdyovFJVhQ-DTS2g0_kJPK9ucgBPejohisBcx8zlugdRuQhRrmo0Fhc5E1I7x_wwx-1WoMpg2pnOOMM0abzemR_uyhtjPvIN8FLH0dBhkKFUyQ-q-RRQSmLbXAayx6JUL6wVLgJrt-1Nbzk3kxzgPfU7q5W0S8C_zPfX_zYYU5rdo Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: e2bfe1e4fbfd48998884d5e8ef8a3e57 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: e2bfe1e4fbfd48998884d5e8ef8a3e57 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestNetworkBasicOps-1261870155-project Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: bd14dd0707a1423d82328ac3a54ccb28 Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestNetworkBasicOps-1261870155-project Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 799b855f-2148-4b2b-9042-9417967f70dd could not be found. Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '799b855f-2148-4b2b-9042-9417967f70dd'} Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 799b855f-2148-4b2b-9042-9417967f70dd could not be found. Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 799b855f-2148-4b2b-9042-9417967f70dd could not be found. Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:30:55.260905 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 799b855f-2148-4b2b-9042-9417967f70dd could not be found. Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Returning 404 to user: Instance 799b855f-2148-4b2b-9042-9417967f70dd could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-447a0fe5-cee7-4df5-a48c-2e0c798173e6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd" status: 404 len: 111 microversion: 2.1 time: 0.031255 Oct 15 21:30:55.265364 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1662/5565] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:55 2022] GET /compute/v2.1/servers/799b855f-2148-4b2b-9042-9417967f70dd => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:30:55.436675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45f68ec5-d697-43fe-bac5-7cc69c72795e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '.version_select at 0x7fe74e327e50>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:55.458442 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45f68ec5-d697-43fe-bac5-7cc69c72795e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-2086908361" status: 202 len: 0 microversion: 2.1 time: 0.025955 Oct 15 21:30:55.459062 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3904/5566] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:30:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-2086908361 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:30:59.618932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d72d3878-6534-48f3-be61-278f25d2d6bf req-bb3bcb91-29ca-4655-aabd-3b117a795889 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:30:59.620219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:59.621674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:59.630756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:59.631595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:59.632526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:59.633508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:59.633938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d72d3878-6534-48f3-be61-278f25d2d6bf req-bb3bcb91-29ca-4655-aabd-3b117a795889 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:59.634432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d72d3878-6534-48f3-be61-278f25d2d6bf req-bb3bcb91-29ca-4655-aabd-3b117a795889 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:59.634653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:59.634880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d72d3878-6534-48f3-be61-278f25d2d6bf req-bb3bcb91-29ca-4655-aabd-3b117a795889 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:59.635414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:59.650018 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d72d3878-6534-48f3-be61-278f25d2d6bf req-bb3bcb91-29ca-4655-aabd-3b117a795889 service nova] Creating event network-changed:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:30:59.660255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d72d3878-6534-48f3-be61-278f25d2d6bf req-bb3bcb91-29ca-4655-aabd-3b117a795889 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043548 Oct 15 21:30:59.662044 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3905/5567] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:30:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:30:59.687592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:30:59.815103 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aebdc841-c31e-4f79-ad99-6e8ed2cd047f tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 204 len: 0 microversion: 2.1 time: 0.203131 Oct 15 21:30:59.817960 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1663/5568] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:30:59 2022] DELETE /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:30:59.827864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:30:59.828514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:30:59.836728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:59.837528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:59.838139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:30:59.839080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:30:59.839777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:30:59.840903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:00.013487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:00.022547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b51bb0f-9565-4a89-8743-a1e425113043 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1582 microversion: 2.1 time: 0.199383 Oct 15 21:31:00.037434 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3906/5569] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:30:59 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1582 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:01.048526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:01.049569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:01.056927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:01.057656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:01.058278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:01.058995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:01.059585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:01.060173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:01.338480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:01.347745 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69f82274-79ad-470c-b266-c83a575f8e68 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1582 microversion: 2.1 time: 0.302963 Oct 15 21:31:01.348450 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1664/5570] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:01 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1582 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:02.372404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:02.372791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:02.379558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e299fde0-0333-48e5-9f68-57530d4656f3 req-9300d478-1e4f-45c8-b4e2-4302ea4461ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a832b7e1-4e38-423d-87c9-15ec291ed760", "name": "network-vif-deleted", "tag": "60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:02.380632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:02.381400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:02.382026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:02.383167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:02.383639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:02.384411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:02.397759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e299fde0-0333-48e5-9f68-57530d4656f3 req-9300d478-1e4f-45c8-b4e2-4302ea4461ca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:02.398570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e299fde0-0333-48e5-9f68-57530d4656f3 req-9300d478-1e4f-45c8-b4e2-4302ea4461ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:02.399715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e299fde0-0333-48e5-9f68-57530d4656f3 req-9300d478-1e4f-45c8-b4e2-4302ea4461ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:02.415242 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e299fde0-0333-48e5-9f68-57530d4656f3 req-9300d478-1e4f-45c8-b4e2-4302ea4461ca service nova] Creating event network-vif-deleted:60c7c6ea-c686-4c94-9d2d-3f01d6fbfc0b for instance a832b7e1-4e38-423d-87c9-15ec291ed760 on nch1-master-1 Oct 15 21:31:02.421424 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e299fde0-0333-48e5-9f68-57530d4656f3 req-9300d478-1e4f-45c8-b4e2-4302ea4461ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050046 Oct 15 21:31:02.422046 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1665/5571] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:31:02.493328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a68a0c1d8f45149c42699b2367f108', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c1026ed03744eaaa999b2d54aac2250', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:02.501789 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e5ff1f2-6e7e-413d-bf0b-354c6f6d1791 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 200 len: 1397 microversion: 2.1 time: 0.133659 Oct 15 21:31:02.502685 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3907/5572] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:02 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 1397 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:03.528550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:03.529587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] [instance: a832b7e1-4e38-423d-87c9-15ec291ed760] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:03.536498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:03.537197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:03.537966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:03.538929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:03.539654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:03.540328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:03.556018 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] HTTP exception thrown: Instance a832b7e1-4e38-423d-87c9-15ec291ed760 could not be found. Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Request method failure captured: Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 HTTP/1.1 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyY-8cvisdwG7_hJ1HE_4pY7pNDo569_2LVj2sJUD_kYSStRhX49Jh_jfy85w4mkcmrFatFlJs_PoHP1-D13LqT1S9511dEIr1xj6BkVMr-rHZOnNUMQA1Jvc2eKjyUqBSfUZ5ekt4jU8lBMqN8VnTspAplUlKM97FZ6pv7nHM_V-nY Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 6c1026ed03744eaaa999b2d54aac2250 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBackupRestore-611042947 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c1026ed03744eaaa999b2d54aac2250"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6c1026ed03744eaaa999b2d54aac2250"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c1026ed03744eaaa999b2d54aac2250"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6c1026ed03744eaaa999b2d54aac2250"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBackupRestore-611042947 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 6c1026ed03744eaaa999b2d54aac2250 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBackupRestore-611042947 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBackupRestore-611042947-project Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a68a0c1d8f45149c42699b2367f108 Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBackupRestore-611042947-project Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance a832b7e1-4e38-423d-87c9-15ec291ed760 could not be found. Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'a832b7e1-4e38-423d-87c9-15ec291ed760'} Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a832b7e1-4e38-423d-87c9-15ec291ed760 could not be found. Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a832b7e1-4e38-423d-87c9-15ec291ed760 could not be found. Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:03.557926 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a832b7e1-4e38-423d-87c9-15ec291ed760 could not be found. Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Returning 404 to user: Instance a832b7e1-4e38-423d-87c9-15ec291ed760 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a896a7f7-08e7-4f75-8618-ed90d7d86ecc tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760" status: 404 len: 111 microversion: 2.1 time: 0.037163 Oct 15 21:31:03.562032 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1666/5573] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:03 2022] GET /compute/v2.1/servers/a832b7e1-4e38-423d-87c9-15ec291ed760 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:31:03.757537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ac49c38-88b9-4d4f-8970-c13550d4ff48 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] Calling method '.version_select at 0x7fe74e08c700>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:03.789672 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ac49c38-88b9-4d4f-8970-c13550d4ff48 tempest-TestVolumeBackupRestore-611042947 tempest-TestVolumeBackupRestore-611042947-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBackupRestore-1285221332" status: 202 len: 0 microversion: 2.1 time: 0.036638 Oct 15 21:31:03.790989 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3908/5574] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:31:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBackupRestore-1285221332 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:31:04.926102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-34ebfd04-c93c-4b10-986f-2d715055f472 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'create', calling method: .version_select at 0x7fe74ec46af0>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-1702678073", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG5al7mvJwpe+K9jYHsYwPycyaPdnrbFSyGyt2lS7l/wFmRKvPfgg3zvrOMflMmBuQ09dHNdbcHaPAeTAqGgTci1nSy/4gn87hKgf2MrDcR3TT/ErfVVOweWMdRbm2j6nQ=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:04.996934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-34ebfd04-c93c-4b10-986f-2d715055f472 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:31:05.017958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-34ebfd04-c93c-4b10-986f-2d715055f472 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for user bd14dd0707a1423d82328ac3a54ccb28 and project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:31:05.083853 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-34ebfd04-c93c-4b10-986f-2d715055f472 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.171909 Oct 15 21:31:05.085805 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1667/5575] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:31:04 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:06.496577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-1028054709", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "595842de-9680-4d03-bd02-4712d82e670c"}], "key_name": "tempest-TestNetworkBasicOps-1702678073", "security_groups": [{"name": "tempest-secgroup-smoke-1427750816"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:06.615185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] validate_networks() for [('595842de-9680-4d03-bd02-4712d82e670c', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:31:07.075372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:31:07.075800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:31:07.076279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:31:07.076693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:31:07.077136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:31:07.088782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:07.089311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:07.089933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:07.090661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:07.091124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:07.093207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:07.115806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:31:07.122486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Getting quotas for user bd14dd0707a1423d82328ac3a54ccb28 and project e2bfe1e4fbfd48998884d5e8ef8a3e57. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:31:07.138912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:31:07.141605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:31:07.195287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-647a9be0-33e9-494b-843e-19ba49e42cd8 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.703917 Oct 15 21:31:07.196326 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3909/5576] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:31:06 2022] POST /compute/v2.1/servers => generated 406 bytes in 706 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:31:07.208593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-91353216-bf06-489a-908e-feef22f6e8f7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:07.209254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-91353216-bf06-489a-908e-feef22f6e8f7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:07.270399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-91353216-bf06-489a-908e-feef22f6e8f7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:07.278905 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-91353216-bf06-489a-908e-feef22f6e8f7 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1249 microversion: 2.1 time: 0.075045 Oct 15 21:31:07.279486 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1668/5577] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:07 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1249 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:08.305281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:08.306591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:08.314994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:08.315689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:08.316607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:08.317525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:08.318352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:08.318934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:08.432156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:08.440061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32c72244-25e3-4842-87b3-89ec6249b68e tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1309 microversion: 2.1 time: 0.141275 Oct 15 21:31:08.441328 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3910/5578] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:08 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1309 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:09.463789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:09.464413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:09.471770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:09.472928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:09.480842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:09.481657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:09.482144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:09.482586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:09.821525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:09.830167 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6280107a-c48f-4684-95df-42ba2058f9d6 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1375 microversion: 2.1 time: 0.370160 Oct 15 21:31:09.830784 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1669/5579] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:09 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1375 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:10.677060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-84ce0cc7-e856-4a3a-b782-a9280e0e6210 req-ecd72409-07c2-4182-b60a-e35649133e81 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "451f4372-4467-42ea-b0e1-f9126802c024", "tag": "19887850-ab68-49f0-81e0-79941afb7a0b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:10.690306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-84ce0cc7-e856-4a3a-b782-a9280e0e6210 req-ecd72409-07c2-4182-b60a-e35649133e81 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:10.691747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-84ce0cc7-e856-4a3a-b782-a9280e0e6210 req-ecd72409-07c2-4182-b60a-e35649133e81 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:10.692382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-84ce0cc7-e856-4a3a-b782-a9280e0e6210 req-ecd72409-07c2-4182-b60a-e35649133e81 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:10.705004 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-84ce0cc7-e856-4a3a-b782-a9280e0e6210 req-ecd72409-07c2-4182-b60a-e35649133e81 service nova] Creating event network-changed:19887850-ab68-49f0-81e0-79941afb7a0b for instance 451f4372-4467-42ea-b0e1-f9126802c024 on nch2-master-1 Oct 15 21:31:10.711645 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-84ce0cc7-e856-4a3a-b782-a9280e0e6210 req-ecd72409-07c2-4182-b60a-e35649133e81 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037476 Oct 15 21:31:10.711645 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3911/5580] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:10.854510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:10.855200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:10.864664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:10.865188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:10.865802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:10.866695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:10.867301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:10.867865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:11.156774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:11.166201 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e2e6e041-ae0d-4a3f-9878-4c898e7b6b8b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1375 microversion: 2.1 time: 0.315136 Oct 15 21:31:11.167352 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1670/5581] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:10 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1375 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:12.193002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:12.193926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:12.203342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:12.203847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:12.204289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:12.204998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:12.205487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:12.205914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:12.691133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:12.698919 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3755efa4-f2a4-4fb3-83db-9cc519e90d87 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1375 microversion: 2.1 time: 0.511770 Oct 15 21:31:12.699516 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3912/5582] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:12 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1375 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:13.722793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:13.724171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:13.731243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:13.731849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:13.732666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:13.734147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:13.734861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:13.735483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:14.082488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:14.097543 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-754facd7-04c5-47c7-84aa-862777773219 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1375 microversion: 2.1 time: 0.378957 Oct 15 21:31:14.099271 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1671/5583] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:13 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1375 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:15.123001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:15.123612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:15.131077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:15.131588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:15.132025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:15.132724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:15.133211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:15.133639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:15.305012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:15.312930 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31cc57d0-aea2-48b1-ac40-dbea0b1217cf tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1375 microversion: 2.1 time: 0.193882 Oct 15 21:31:15.313583 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3913/5584] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:15 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1375 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:16.177491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5b815cc9-aa90-4dd1-8692-c0cb5e0b3148 req-62c30c2d-b48b-4e3f-98c2-34c4d9733bb9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "451f4372-4467-42ea-b0e1-f9126802c024", "name": "network-vif-plugged", "status": "completed", "tag": "19887850-ab68-49f0-81e0-79941afb7a0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:16.205297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5b815cc9-aa90-4dd1-8692-c0cb5e0b3148 req-62c30c2d-b48b-4e3f-98c2-34c4d9733bb9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:16.205766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5b815cc9-aa90-4dd1-8692-c0cb5e0b3148 req-62c30c2d-b48b-4e3f-98c2-34c4d9733bb9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:16.206200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5b815cc9-aa90-4dd1-8692-c0cb5e0b3148 req-62c30c2d-b48b-4e3f-98c2-34c4d9733bb9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:16.221055 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5b815cc9-aa90-4dd1-8692-c0cb5e0b3148 req-62c30c2d-b48b-4e3f-98c2-34c4d9733bb9 service nova] Creating event network-vif-plugged:19887850-ab68-49f0-81e0-79941afb7a0b for instance 451f4372-4467-42ea-b0e1-f9126802c024 on nch2-master-1 Oct 15 21:31:16.227591 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5b815cc9-aa90-4dd1-8692-c0cb5e0b3148 req-62c30c2d-b48b-4e3f-98c2-34c4d9733bb9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054136 Oct 15 21:31:16.228258 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1672/5585] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:16.338079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:16.338865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:16.346190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:16.346681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:16.347227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:16.348427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:16.348957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:16.349404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:16.795461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:16.803723 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2836841-d9b0-456a-897a-a0edc0e92bb5 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1375 microversion: 2.1 time: 0.469994 Oct 15 21:31:16.804687 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3914/5586] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:16 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1375 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:17.828277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:17.829001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:17.832508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:17.833206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=59,nova_api:UPDATE=3,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:17.839276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:17.839772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:17.840214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:17.840921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:17.841416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:17.841844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:17.846739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:18.054522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:18.062412 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-004203ca-f221-4e7d-84fb-f030d49b1b20 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1546 microversion: 2.1 time: 0.238327 Oct 15 21:31:18.063189 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1673/5587] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:17 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1546 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:18.086175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:18.087695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:18.089834 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:18.091074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:18.092281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=51,nova_api:DELETE=3,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:18.099363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:18.100078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:18.100529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:18.101283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:18.101759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:18.102197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:18.107144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=61 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:18.289597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:18.298633 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aed88b61-c3f0-4dfe-89a2-afcff6b37bec tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1546 microversion: 2.1 time: 0.216722 Oct 15 21:31:18.299472 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3915/5588] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:18 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1546 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:20.255719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-dc8075f7-67e0-40a3-a71e-a936d18a9f30 req-f609af59-d122-42ca-9964-274b3b13e7b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "451f4372-4467-42ea-b0e1-f9126802c024", "tag": "19887850-ab68-49f0-81e0-79941afb7a0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:20.281975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dc8075f7-67e0-40a3-a71e-a936d18a9f30 req-f609af59-d122-42ca-9964-274b3b13e7b8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:20.282463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dc8075f7-67e0-40a3-a71e-a936d18a9f30 req-f609af59-d122-42ca-9964-274b3b13e7b8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:20.282987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-dc8075f7-67e0-40a3-a71e-a936d18a9f30 req-f609af59-d122-42ca-9964-274b3b13e7b8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:20.307478 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-dc8075f7-67e0-40a3-a71e-a936d18a9f30 req-f609af59-d122-42ca-9964-274b3b13e7b8 service nova] Creating event network-changed:19887850-ab68-49f0-81e0-79941afb7a0b for instance 451f4372-4467-42ea-b0e1-f9126802c024 on nch2-master-1 Oct 15 21:31:20.325162 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-dc8075f7-67e0-40a3-a71e-a936d18a9f30 req-f609af59-d122-42ca-9964-274b3b13e7b8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074257 Oct 15 21:31:20.325789 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1674/5589] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:32.213341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-916634272", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "1a8caae9-f26a-4bfa-9016-f45c40448b03", "source_type": "snapshot", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "5159a515-dfbe-44d6-b24b-a04195666fcf"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:32.224523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:32.225216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:32.313569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] validate_networks() for [('5159a515-dfbe-44d6-b24b-a04195666fcf', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:31:32.707989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:31:32.708440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:31:32.708919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:31:32.709518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:31:32.709941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:31:32.719448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:32.720123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:32.720566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:32.721335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:32.721798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:32.722220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:32.741071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:31:32.746673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:31:32.760199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:31:32.762372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='1a8caae9-f26a-4bfa-9016-f45c40448b03',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:31:32.843861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be3214f1-cd67-4318-bc96-af2659f56b65 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.634163 Oct 15 21:31:32.844925 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3916/5590] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:31:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 636 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:31:32.855911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:32.857025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=7 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:32.861929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1c0ee48e-d81a-45f0-8f60-51a9c206087c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:32.862647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1c0ee48e-d81a-45f0-8f60-51a9c206087c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:32.924456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1c0ee48e-d81a-45f0-8f60-51a9c206087c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:32.932979 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1c0ee48e-d81a-45f0-8f60-51a9c206087c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1056 microversion: 2.1 time: 0.079351 Oct 15 21:31:32.933776 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1675/5591] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:32 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1056 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:33.956180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:33.957383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:33.964586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:33.965115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:33.965567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:33.966277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:33.966732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:33.967158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:34.052863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:34.062606 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b2c423fe-2e03-48b2-b775-780097d96678 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1126 microversion: 2.1 time: 0.110480 Oct 15 21:31:34.063204 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3917/5592] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:33 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1126 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:31:35.086959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:35.087585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:35.095467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:35.095957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:35.096396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:35.097125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:35.097587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:35.098021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:35.310608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:35.319150 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-448057cd-d803-4673-b1ee-4815da6907c3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1168 microversion: 2.1 time: 0.235856 Oct 15 21:31:35.319247 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1676/5593] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:35 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1168 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:35.640547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-3143faa1-264b-4c44-803a-53430bc16c05 req-ea9ad114-8e44-4b10-96b8-200fe11571a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ecb09770-867d-4dc3-a208-f7fd51b82b43", "tag": "5b6b2f6a-f336-4584-b87f-ab509b9cdfbf"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:35.651672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3143faa1-264b-4c44-803a-53430bc16c05 req-ea9ad114-8e44-4b10-96b8-200fe11571a2 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:35.652419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3143faa1-264b-4c44-803a-53430bc16c05 req-ea9ad114-8e44-4b10-96b8-200fe11571a2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:35.653052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-3143faa1-264b-4c44-803a-53430bc16c05 req-ea9ad114-8e44-4b10-96b8-200fe11571a2 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:35.664458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-3143faa1-264b-4c44-803a-53430bc16c05 req-ea9ad114-8e44-4b10-96b8-200fe11571a2 service nova] Creating event network-changed:5b6b2f6a-f336-4584-b87f-ab509b9cdfbf for instance ecb09770-867d-4dc3-a208-f7fd51b82b43 on nch1-master-1 Oct 15 21:31:35.669365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-3143faa1-264b-4c44-803a-53430bc16c05 req-ea9ad114-8e44-4b10-96b8-200fe11571a2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033244 Oct 15 21:31:35.669980 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3918/5594] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:36.341715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:36.342403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:36.349427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:36.349910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:36.350349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:36.351040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:36.351492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:36.351920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:36.562310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:36.571032 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e266108-d29e-465b-b1fa-b065df76db24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1168 microversion: 2.1 time: 0.233107 Oct 15 21:31:36.571612 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1677/5595] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:36 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1168 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:37.595879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:37.596658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:37.603392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:37.604060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:37.604499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:37.605225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:37.605688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:37.606138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:37.753099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:37.761819 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e4bb9c4-eee9-4862-9b0f-78218af8a6cf tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1168 microversion: 2.1 time: 0.171125 Oct 15 21:31:37.762514 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3919/5596] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:37 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1168 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:38.786393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:38.787030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:38.793914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:38.794407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:38.794853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:38.795550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:38.796014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:38.796449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:38.929308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:38.937194 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-450b2461-9e98-4b7f-b112-4fc21f545e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1202 microversion: 2.1 time: 0.155484 Oct 15 21:31:38.937766 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1678/5597] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:38 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1202 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:39.960778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:39.961445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:39.968371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:39.968862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:39.969345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:39.970050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:39.970520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:39.970988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:40.102046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:40.109860 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fb47b40d-b79a-47dc-9d9d-d45f5f1f722d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1202 microversion: 2.1 time: 0.153786 Oct 15 21:31:40.110431 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3920/5598] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:39 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1202 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:41.133151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:41.133797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:41.140574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:41.141084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:41.141525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:41.142223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:41.142689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:41.143116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:41.275642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:41.283524 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ba434ec4-707b-4605-9102-a50ca7c09fe2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1202 microversion: 2.1 time: 0.154553 Oct 15 21:31:41.284095 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1679/5599] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:41 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1202 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:42.307845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:42.308453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:42.315726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:42.316911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:42.317395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:42.318082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:42.318540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:42.318968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:42.451840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:42.460079 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78fb4e0a-f42a-4cb0-8d8d-36c476e8441f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1202 microversion: 2.1 time: 0.156084 Oct 15 21:31:42.460650 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3921/5600] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:42 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1202 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:43.460640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-fce4704f-3a21-4a18-8f4b-acb1810c0d1c req-59bda0bf-0532-4353-8aff-35fc9dcb8678 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ecb09770-867d-4dc3-a208-f7fd51b82b43", "name": "network-vif-plugged", "status": "completed", "tag": "5b6b2f6a-f336-4584-b87f-ab509b9cdfbf"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:43.467318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:43.473540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-fce4704f-3a21-4a18-8f4b-acb1810c0d1c req-59bda0bf-0532-4353-8aff-35fc9dcb8678 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:43.474031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-fce4704f-3a21-4a18-8f4b-acb1810c0d1c req-59bda0bf-0532-4353-8aff-35fc9dcb8678 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:43.474472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-fce4704f-3a21-4a18-8f4b-acb1810c0d1c req-59bda0bf-0532-4353-8aff-35fc9dcb8678 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:43.484361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:43.485081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:43.486144 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-fce4704f-3a21-4a18-8f4b-acb1810c0d1c req-59bda0bf-0532-4353-8aff-35fc9dcb8678 service nova] Creating event network-vif-plugged:5b6b2f6a-f336-4584-b87f-ab509b9cdfbf for instance ecb09770-867d-4dc3-a208-f7fd51b82b43 on nch1-master-1 Oct 15 21:31:43.487207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:43.488642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:31:43.492315 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-fce4704f-3a21-4a18-8f4b-acb1810c0d1c req-59bda0bf-0532-4353-8aff-35fc9dcb8678 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035769 Oct 15 21:31:43.493132 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1680/5601] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:43.497726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:43.498595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:43.499043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:43.499751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:43.500867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:43.501331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:43.700125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:43.708181 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b623dbd0-51c6-402f-8f51-cd21479c0190 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1202 microversion: 2.1 time: 0.227868 Oct 15 21:31:43.708757 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3922/5602] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:43 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1202 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:44.732904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:44.733664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:44.739817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:44.740303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:44.740735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:44.741452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:44.741900 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:44.742322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:44.875415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:44.883876 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-309836ed-b36e-4227-b8b1-f526867510c2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1387 microversion: 2.1 time: 0.156004 Oct 15 21:31:44.884449 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1681/5603] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:44 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1387 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:44.906268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:44.906883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:44.913881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:44.914371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:44.914813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:44.915512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:44.915976 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:44.916405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.043910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:45.052288 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1214678-a9a4-4db5-b02f-690ec9302271 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1387 microversion: 2.1 time: 0.149804 Oct 15 21:31:45.052873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3923/5604] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:44 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1387 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:45.074012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:45.074676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:45.080597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:45.081142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:45.081579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.082313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:45.082769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:45.083190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.216278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:45.224471 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ccfd7bc-3f2a-4b44-a3d4-474bceece4d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1387 microversion: 2.1 time: 0.154289 Oct 15 21:31:45.225071 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1682/5605] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:45 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1387 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:31:45.304671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:45.305207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:45.311222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:45.311715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:45.312159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.312887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:45.313388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:45.313823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.339969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:31:45.420037 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2bffafd9-e946-400d-9c69-0e516486a6a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 204 len: 0 microversion: 2.1 time: 0.119020 Oct 15 21:31:45.420603 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3924/5606] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:31:45 2022] DELETE /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:31:45.429618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:45.430268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:45.436179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:45.436670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:45.437149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.437847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:45.438308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:45.438740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:45.569323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:45.577280 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-abc6e003-f27b-4989-b678-b4af106a8e7c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1393 microversion: 2.1 time: 0.152080 Oct 15 21:31:45.577882 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1683/5607] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:45 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1393 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:46.600447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:46.601183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:46.608025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:46.608511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:46.608954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:46.609687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:46.610151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:46.610581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:46.743887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:46.751989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccd9da11-471f-4c58-b0c9-6cc3cfccb3d9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1393 microversion: 2.1 time: 0.155862 Oct 15 21:31:46.752557 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3925/5608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:46 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1393 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:47.495501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d2902ea2-f8fb-4abc-89c1-cc04ca990642 req-3775f769-50cd-4046-8c94-a33dcdd618e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ecb09770-867d-4dc3-a208-f7fd51b82b43", "name": "network-vif-deleted", "tag": "5b6b2f6a-f336-4584-b87f-ab509b9cdfbf"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:47.508375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d2902ea2-f8fb-4abc-89c1-cc04ca990642 req-3775f769-50cd-4046-8c94-a33dcdd618e8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:47.508870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d2902ea2-f8fb-4abc-89c1-cc04ca990642 req-3775f769-50cd-4046-8c94-a33dcdd618e8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:47.509327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d2902ea2-f8fb-4abc-89c1-cc04ca990642 req-3775f769-50cd-4046-8c94-a33dcdd618e8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:47.522820 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d2902ea2-f8fb-4abc-89c1-cc04ca990642 req-3775f769-50cd-4046-8c94-a33dcdd618e8 service nova] Creating event network-vif-deleted:5b6b2f6a-f336-4584-b87f-ab509b9cdfbf for instance ecb09770-867d-4dc3-a208-f7fd51b82b43 on nch1-master-1 Oct 15 21:31:47.528002 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d2902ea2-f8fb-4abc-89c1-cc04ca990642 req-3775f769-50cd-4046-8c94-a33dcdd618e8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036093 Oct 15 21:31:47.528604 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1684/5609] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:47.775910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:47.777272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:47.791082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:47.791672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:47.792341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:47.793108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:47.793756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:47.794351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:47.829091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:47.829586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:47.830250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:47.830706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lazy-loading 'fault' on Instance uuid ecb09770-867d-4dc3-a208-f7fd51b82b43 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:31:47.879876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:47.889273 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d29e490a-9fad-4d02-b3bd-154a7299d9bc tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 200 len: 1192 microversion: 2.1 time: 0.117205 Oct 15 21:31:47.889889 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3926/5610] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:47 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 1192 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:48.913554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:48.914322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:48.921364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:48.921853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:48.922292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:48.922977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:48.923457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:48.923888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:48.937080 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 HTTP/1.1 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ecb09770-867d-4dc3-a208-f7fd51b82b43'} Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:48.938730 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d9422883-a863-4a83-a2f4-da26bade71a7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 404 len: 111 microversion: 2.1 time: 0.031080 Oct 15 21:31:48.944020 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1685/5611] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:48 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:31:48.971486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:48.971991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:48.978137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:48.978640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:48.979056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:48.979745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:48.980198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:48.980621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:48.993978 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 HTTP/1.1 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'ecb09770-867d-4dc3-a208-f7fd51b82b43'} Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:31:48.995658 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5e29299-69b1-400e-a591-9847d44a369d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 404 len: 111 microversion: 2.1 time: 0.029653 Oct 15 21:31:49.000588 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3927/5612] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:31:48 2022] DELETE /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:31:49.006872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:49.007494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: ecb09770-867d-4dc3-a208-f7fd51b82b43] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:49.013556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:49.014044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:49.014483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:49.015172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:49.015629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:49.016054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:49.029986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 HTTP/1.1 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'ecb09770-867d-4dc3-a208-f7fd51b82b43'} Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:31:49.031500 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance ecb09770-867d-4dc3-a208-f7fd51b82b43 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5f570a8-4d63-462d-9af1-b7f098487183 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43" status: 404 len: 111 microversion: 2.1 time: 0.030745 Oct 15 21:31:49.036057 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1686/5613] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:49 2022] GET /compute/v2.1/servers/ecb09770-867d-4dc3-a208-f7fd51b82b43 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:31:51.686339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:51.688879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:51.697175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:51.697661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:51.698098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:51.698799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:51.699253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:51.699734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:51.761723 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e28fd5d-7732-47e8-b2bc-6ecdea656dbe tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024/action" status: 200 len: 24571 microversion: 2.1 time: 0.078981 Oct 15 21:31:51.762486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3928/5614] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:31:51 2022] POST /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024/action => generated 24571 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:31:53.902429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-volume-backed-server-1014331117", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "c7d3300c-3164-4ded-8f73-802c79b5ec27", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "5159a515-dfbe-44d6-b24b-a04195666fcf"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:53.965498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] validate_networks() for [('5159a515-dfbe-44d6-b24b-a04195666fcf', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:31:54.343515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:31:54.343944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:31:54.344439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:31:54.344887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:31:54.345334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:31:54.354600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:54.355096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:54.355542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:54.356282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:54.356993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:54.357240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:54.374773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:31:54.380588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:31:54.440063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:31:54.442935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c7d3300c-3164-4ded-8f73-802c79b5ec27',volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:31:54.568705 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f16e27d-fcc1-4c1e-9795-a7149e27c745 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.670105 Oct 15 21:31:54.569386 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1687/5615] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:31:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:31:54.582158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1696b935-c15b-4428-9344-292f427275fa tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:54.583434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1696b935-c15b-4428-9344-292f427275fa tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:54.639831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1696b935-c15b-4428-9344-292f427275fa tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:54.648249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1696b935-c15b-4428-9344-292f427275fa tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1071 microversion: 2.1 time: 0.071333 Oct 15 21:31:54.648823 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3929/5616] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:54 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1071 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:55.672401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:55.673082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:55.679839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:55.680380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:55.680819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:55.681744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:55.682365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:55.682944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:55.755558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:55.763736 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d50ae1d1-edc3-447d-9008-a0a4f3eef64a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1187 microversion: 2.1 time: 0.096420 Oct 15 21:31:55.764515 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1688/5617] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:55 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1187 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:56.787543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:56.788200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:56.795482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:56.795978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:56.796427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:56.797165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:56.797633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:56.798070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:56.999064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:57.001113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-01ed5e00-8db6-46b7-b798-cccb55e9c4c9 req-8628d1ed-643a-4f6b-b5dd-992a53c38e6d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4", "tag": "3cd01562-bd65-41e3-8c19-ba0aa69de14c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:31:57.009445 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-16ef65de-39c7-4c70-90f7-bebf5d10653b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.226313 Oct 15 21:31:57.025758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3930/5618] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:56 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:57.025983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-01ed5e00-8db6-46b7-b798-cccb55e9c4c9 req-8628d1ed-643a-4f6b-b5dd-992a53c38e6d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:57.025983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-01ed5e00-8db6-46b7-b798-cccb55e9c4c9 req-8628d1ed-643a-4f6b-b5dd-992a53c38e6d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:57.025983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-01ed5e00-8db6-46b7-b798-cccb55e9c4c9 req-8628d1ed-643a-4f6b-b5dd-992a53c38e6d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:57.033013 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-01ed5e00-8db6-46b7-b798-cccb55e9c4c9 req-8628d1ed-643a-4f6b-b5dd-992a53c38e6d service nova] Creating event network-changed:3cd01562-bd65-41e3-8c19-ba0aa69de14c for instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 on nch1-master-1 Oct 15 21:31:57.038370 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-01ed5e00-8db6-46b7-b798-cccb55e9c4c9 req-8628d1ed-643a-4f6b-b5dd-992a53c38e6d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041182 Oct 15 21:31:57.039144 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1689/5619] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:31:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:31:58.033590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:58.034124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:58.041331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:58.041821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:58.042264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:58.042959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:58.043424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:58.043854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:58.268174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:58.276177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-37b00c16-c049-407b-9cfd-9a2f02bf2f67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.246595 Oct 15 21:31:58.276773 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3931/5620] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:58 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:31:59.300577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:31:59.301318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:31:59.307713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:59.308208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:59.308653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:59.309384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:31:59.309850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:31:59.310286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:31:59.445366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:31:59.453264 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-74d95a4d-3383-4a0e-b353-0056ee4aa298 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.157758 Oct 15 21:31:59.453777 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1690/5621] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:31:59 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:00.477766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:00.478315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:00.485432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:00.485927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:00.486371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:00.487135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:00.487630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:00.488065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:00.620529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:00.628507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c49679fe-b741-49d5-96a9-7e2f2a68f8d5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.155356 Oct 15 21:32:00.629087 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3932/5622] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:00 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:01.652030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:01.652647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:01.659634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:01.660119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:01.660597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:01.661332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:01.661786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:01.662214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:01.792417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:01.800375 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-33e7ca03-496e-4e4d-a8b6-51ff0af4da61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.152304 Oct 15 21:32:01.800945 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1691/5623] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:01 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:02.823183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:02.823796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:02.826610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:02.832094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:02.832521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:02.833530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:02.834253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:02.834702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:02.835130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:02.838645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:02.974130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:02.982115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6370c4dc-1aca-4e2e-9bfa-0968256f94ab tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.162791 Oct 15 21:32:02.982683 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3933/5624] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:02 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:03.570823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-57ff583c-8708-4e07-a12c-bc2036efe430 req-66ad6380-8929-48bd-9ec1-d39c1056e20b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4", "name": "network-vif-plugged", "status": "completed", "tag": "3cd01562-bd65-41e3-8c19-ba0aa69de14c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:03.577896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:03.583111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-57ff583c-8708-4e07-a12c-bc2036efe430 req-66ad6380-8929-48bd-9ec1-d39c1056e20b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:03.583794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-57ff583c-8708-4e07-a12c-bc2036efe430 req-66ad6380-8929-48bd-9ec1-d39c1056e20b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:03.584272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-57ff583c-8708-4e07-a12c-bc2036efe430 req-66ad6380-8929-48bd-9ec1-d39c1056e20b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:03.588712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=73 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:03.597378 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-57ff583c-8708-4e07-a12c-bc2036efe430 req-66ad6380-8929-48bd-9ec1-d39c1056e20b service nova] Creating event network-vif-plugged:3cd01562-bd65-41e3-8c19-ba0aa69de14c for instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 on nch1-master-1 Oct 15 21:32:03.602770 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-57ff583c-8708-4e07-a12c-bc2036efe430 req-66ad6380-8929-48bd-9ec1-d39c1056e20b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036728 Oct 15 21:32:03.603755 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1692/5625] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:32:04.004997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:04.005569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:04.012503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:04.012989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:04.013466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:04.014163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:04.014625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:04.015057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:04.144419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:04.153448 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-760c4348-5a84-4177-beec-f4a560a0508b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1217 microversion: 2.1 time: 0.152543 Oct 15 21:32:04.154050 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3934/5626] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:03 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1217 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:05.175461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:05.176219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:05.182223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:05.182926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:05.189881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:05.190365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:05.190804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:05.191497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:05.191950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:05.192376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:05.328016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:05.336384 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2fe24de7-2902-42e8-b577-56cb7dabf868 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1403 microversion: 2.1 time: 0.161841 Oct 15 21:32:05.336978 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1693/5627] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:05 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1403 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:05.359120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:05.359733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:05.361693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:05.368370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:05.368857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:05.369322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:05.370018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:05.370476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:05.370903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:05.502324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:05.510347 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e54ce441-0d85-4ea5-8814-cfce85f03802 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1403 microversion: 2.1 time: 0.155063 Oct 15 21:32:05.510932 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3935/5628] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:05 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1403 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:05.532631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestVolumeBootPatternsnapshot-608030667"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:05.536019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:05.542739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:05.543236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:05.543686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:05.544404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:05.544871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:05.545380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:05.641179 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.api [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Attempting to quiesce instance before volume snapshot. Oct 15 21:32:05.658261 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.api [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Skipping quiescing instance because the operation is not supported by the underlying compute driver. Oct 15 21:32:05.732339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Creating snapshot from volume c7d3300c-3164-4ded-8f73-802c79b5ec27. {{(pid=138689) snapshot_instance /opt/stack/nova/nova/compute/api.py:3509}} Oct 15 21:32:06.630732 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b91f0a1-7d4d-4938-bfb0-71180ed00905 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4/action" status: 202 len: 0 microversion: 2.1 time: 1.102013 Oct 15 21:32:06.632696 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1694/5629] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:32:05 2022] POST /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4/action => generated 0 bytes in 1103 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 21:32:07.786848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-image-snapshot-server-933052419", "imageRef": "b77da8c5-d5b1-491e-ac1e-73d2c8388df9", "flavorRef": "452", "networks": [{"uuid": "5159a515-dfbe-44d6-b24b-a04195666fcf"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:07.842899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] validate_networks() for [('5159a515-dfbe-44d6-b24b-a04195666fcf', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:32:08.308284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:32:08.308707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:32:08.309244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:32:08.309656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:32:08.310101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:32:08.318925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:08.319924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:08.320362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:08.321113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:08.321571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:08.321998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:08.339852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:32:08.345087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:32:08.357387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:32:08.359920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name='/dev/sda',device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='c8f07868-533d-4830-9aeb-53f08b64b288',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:32:08.436800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-404f4e24-2ddc-4572-9c98-40451cbb3568 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.653877 Oct 15 21:32:08.437487 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3936/5630] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:32:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 655 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:32:08.450782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-38ce41f7-78ee-4a86-a96f-0e0d7dcf6e3e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:08.451511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-38ce41f7-78ee-4a86-a96f-0e0d7dcf6e3e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:08.505824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38ce41f7-78ee-4a86-a96f-0e0d7dcf6e3e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:08.514687 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-38ce41f7-78ee-4a86-a96f-0e0d7dcf6e3e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1229 microversion: 2.1 time: 0.069240 Oct 15 21:32:08.515319 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1695/5631] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:08 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1229 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:09.538959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:09.539890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:09.546852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:09.547544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:09.548043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:09.548969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:09.549597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:09.550233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:09.656435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:09.665978 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6ba6eb3d-4bad-417c-93d4-0560e015363f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1299 microversion: 2.1 time: 0.131738 Oct 15 21:32:09.666581 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3937/5632] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:09 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1299 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:10.689885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:10.690518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:10.698360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:10.698871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:10.699331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:10.700059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:10.700538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:10.700985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:10.915276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:10.923181 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6aa7762e-0b56-4eb0-b515-5639f489d345 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1341 microversion: 2.1 time: 0.237254 Oct 15 21:32:10.923943 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1696/5633] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:10 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1341 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:11.007044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6533d545-2358-47e0-b483-290f45473624 req-25d8897c-cf8f-420c-997a-634ee9d63262 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8a38979-18bb-4ff9-82d9-b0652f29089e", "tag": "374870fe-7ff3-4e9d-be41-1e3de6475a5a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:11.018688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6533d545-2358-47e0-b483-290f45473624 req-25d8897c-cf8f-420c-997a-634ee9d63262 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:11.019653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6533d545-2358-47e0-b483-290f45473624 req-25d8897c-cf8f-420c-997a-634ee9d63262 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:11.020331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6533d545-2358-47e0-b483-290f45473624 req-25d8897c-cf8f-420c-997a-634ee9d63262 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:11.032469 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6533d545-2358-47e0-b483-290f45473624 req-25d8897c-cf8f-420c-997a-634ee9d63262 service nova] Creating event network-changed:374870fe-7ff3-4e9d-be41-1e3de6475a5a for instance b8a38979-18bb-4ff9-82d9-b0652f29089e on nch2-master-1 Oct 15 21:32:11.037459 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6533d545-2358-47e0-b483-290f45473624 req-25d8897c-cf8f-420c-997a-634ee9d63262 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034416 Oct 15 21:32:11.038071 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3938/5634] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:11.948215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:11.948771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:11.955839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:11.956343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:11.956799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:11.957566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:11.958038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:11.958485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:12.093497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:12.101120 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f8f907a9-725c-479a-819f-a98ff5d8ca34 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1341 microversion: 2.1 time: 0.157874 Oct 15 21:32:12.101766 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1697/5635] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:11 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1341 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:13.125763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:13.126475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:13.133968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:13.134466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:13.134906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:13.135611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:13.136075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:13.136509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:13.278056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:13.286086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cc5583b6-73b8-45b6-aa73-7f266692b5e2 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1341 microversion: 2.1 time: 0.165680 Oct 15 21:32:13.286839 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3939/5636] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:13 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1341 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:14.310244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:14.310778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:14.317723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:14.318225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:14.318677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:14.319383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:14.319854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:14.320832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:14.454003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:14.462361 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b9dc24f-b5de-4348-823a-ad15cf7386ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.156138 Oct 15 21:32:14.463106 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1698/5637] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:14 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:15.485970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:15.486529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:15.494096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:15.494594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:15.495027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:15.495750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:15.496214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:15.496653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:15.632356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:15.640972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a33f53b8-6af3-4c8a-8877-f25f0dc5cfd9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.159062 Oct 15 21:32:15.641602 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3940/5638] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:15 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:16.663651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:16.664277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:16.671442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:16.671944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:16.672399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:16.673133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:16.673606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:16.674046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:16.803143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:16.811399 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fc7fc78b-7170-40a6-b7c3-8743837d1aba tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.151603 Oct 15 21:32:16.811969 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1699/5639] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:16 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:17.834793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:17.835413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:17.843267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:17.843734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:17.844180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:17.844872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:17.845362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:17.845795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:17.981672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:17.989533 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0acba8ca-b3c0-4a06-ac82-7b803207ced4 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.158643 Oct 15 21:32:17.990110 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3941/5640] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:17 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:19.013645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:19.014316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:19.017194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:19.022878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:19.023388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:19.023850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:19.024570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:19.025550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:19.026263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:19.162865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:19.171249 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5afcb53d-355e-4030-a414-75f476a10fa6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.161573 Oct 15 21:32:19.171838 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1700/5641] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:19 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:20.193799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:20.194413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:20.196579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:20.198062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:20.204994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:20.205519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:20.205965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:20.206678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:20.207145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:20.207584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:20.342144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:20.350099 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5fd805a7-29d3-4c83-afa3-a2c0ddba5fd1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.160191 Oct 15 21:32:20.350686 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3942/5642] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:20 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:20.489427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8fe81994-062e-485d-a5fd-527e14a1ae30 req-5d170db7-be11-4244-b6b3-da9789fd787c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8a38979-18bb-4ff9-82d9-b0652f29089e", "name": "network-vif-plugged", "status": "completed", "tag": "374870fe-7ff3-4e9d-be41-1e3de6475a5a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:20.502549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8fe81994-062e-485d-a5fd-527e14a1ae30 req-5d170db7-be11-4244-b6b3-da9789fd787c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:20.503078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8fe81994-062e-485d-a5fd-527e14a1ae30 req-5d170db7-be11-4244-b6b3-da9789fd787c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:20.503530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8fe81994-062e-485d-a5fd-527e14a1ae30 req-5d170db7-be11-4244-b6b3-da9789fd787c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:20.516469 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8fe81994-062e-485d-a5fd-527e14a1ae30 req-5d170db7-be11-4244-b6b3-da9789fd787c service nova] Creating event network-vif-plugged:374870fe-7ff3-4e9d-be41-1e3de6475a5a for instance b8a38979-18bb-4ff9-82d9-b0652f29089e on nch2-master-1 Oct 15 21:32:20.521783 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8fe81994-062e-485d-a5fd-527e14a1ae30 req-5d170db7-be11-4244-b6b3-da9789fd787c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037215 Oct 15 21:32:20.522379 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1701/5643] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:21.373615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:21.374270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:21.381455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:21.381949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:21.382423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:21.383129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:21.383595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:21.384034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:21.514850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:21.523232 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eab3f2d4-89b1-4858-a249-1209921b80f0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1375 microversion: 2.1 time: 0.153450 Oct 15 21:32:21.523821 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3943/5644] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:21 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1375 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:22.143153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:22.146005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:22.153893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:22.154478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:22.154935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:22.155665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:22.156190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:22.156635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:22.213189 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dca2d831-a7de-40ac-b876-0a5f67130dc3 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "POST /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024/action" status: 200 len: 32638 microversion: 2.1 time: 0.075563 Oct 15 21:32:22.213923 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1702/5645] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:32:22 2022] POST /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024/action => generated 32638 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 21:32:22.550854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:22.551480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:22.558511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:22.559006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:22.559454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:22.560164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:22.560632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:22.561088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:22.768396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:22.777403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b06e906e-4be6-4602-bd24-49be36474b9d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1560 microversion: 2.1 time: 0.230329 Oct 15 21:32:22.778306 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3944/5646] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:22 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1560 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:22.803065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:22.803846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:22.811821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:22.812333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:22.812786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:22.813745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:22.814221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:22.814664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.064279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:23.074057 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-45ca0e93-d2c8-4229-806b-bf549a1f7e91 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1560 microversion: 2.1 time: 0.275771 Oct 15 21:32:23.074751 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1703/5647] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:22 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1560 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:23.158302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:23.158857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:23.165184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.166190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.167322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.168031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.168539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.168913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.201104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:32:23.290655 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae41c99c-b6ff-4105-bae6-7d9bd5e55989 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 204 len: 0 microversion: 2.1 time: 0.135447 Oct 15 21:32:23.291758 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3945/5648] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:32:23 2022] DELETE /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:32:23.300634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:23.301406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:23.309526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.310037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.310515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.311239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.311709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.312155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.614033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:23.622673 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1b9a5b7b-51a0-490d-a12a-a60454f6e785 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1566 microversion: 2.1 time: 0.326226 Oct 15 21:32:23.623239 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1704/5649] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:23 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1566 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:23.759574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:23.761729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:23.762824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a1991f6c-44fc-4811-acfc-e876b75ffc24 req-7f1d6ba0-d3e7-4c9e-a3bb-f882993e25ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "451f4372-4467-42ea-b0e1-f9126802c024", "tag": "19887850-ab68-49f0-81e0-79941afb7a0b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:23.772930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a1991f6c-44fc-4811-acfc-e876b75ffc24 req-7f1d6ba0-d3e7-4c9e-a3bb-f882993e25ff service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.773464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a1991f6c-44fc-4811-acfc-e876b75ffc24 req-7f1d6ba0-d3e7-4c9e-a3bb-f882993e25ff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.773917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a1991f6c-44fc-4811-acfc-e876b75ffc24 req-7f1d6ba0-d3e7-4c9e-a3bb-f882993e25ff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.777427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.777902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.778545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.779447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.780033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.780465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.787445 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a1991f6c-44fc-4811-acfc-e876b75ffc24 req-7f1d6ba0-d3e7-4c9e-a3bb-f882993e25ff service nova] Creating event network-changed:19887850-ab68-49f0-81e0-79941afb7a0b for instance 451f4372-4467-42ea-b0e1-f9126802c024 on nch2-master-1 Oct 15 21:32:23.792838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a1991f6c-44fc-4811-acfc-e876b75ffc24 req-7f1d6ba0-d3e7-4c9e-a3bb-f882993e25ff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034249 Oct 15 21:32:23.793616 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1705/5650] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:23.809256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:32:23.888554 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5752799f-e49a-4fc8-b725-4275f432c33b tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "DELETE /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 204 len: 0 microversion: 2.1 time: 0.133000 Oct 15 21:32:23.889937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3946/5651] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:32:23 2022] DELETE /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:32:23.899413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:23.900561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:23.907147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.907656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.908110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:23.908858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:23.909411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:23.909859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:24.051710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:24.060521 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-193784ec-278d-41e0-bf6f-546d1c238043 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1671 microversion: 2.1 time: 0.165975 Oct 15 21:32:24.061112 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1706/5652] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:23 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1671 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:24.648188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:24.649024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:24.656749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:24.657603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:24.658307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:24.659054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:24.659512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:24.659947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:24.819908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:24.823061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:24.830479 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5af12da1-830a-44df-85d7-d4178f12a656 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1566 microversion: 2.1 time: 0.186578 Oct 15 21:32:24.831072 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3947/5653] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:24 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1566 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:25.085749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:25.086488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:25.093594 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:25.094096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:25.094550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:25.095261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:25.095738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:25.096225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:25.268952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:25.277991 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9297f85b-51c6-4289-9152-5c1203230d59 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1671 microversion: 2.1 time: 0.196001 Oct 15 21:32:25.278739 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1707/5654] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:25 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1671 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:25.857069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:25.858485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:25.868690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:25.869231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:25.870284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:25.871514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:25.872292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:25.873080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:26.117533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:26.126542 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c2cd90a1-2252-4833-9eae-d4f356c8ddf5 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1566 microversion: 2.1 time: 0.274583 Oct 15 21:32:26.127132 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3948/5655] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:25 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1566 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:32:26.302691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:26.304047 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:26.313369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:26.313941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:26.314503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:26.315335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:26.316185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:26.316874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:26.541102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d805f7c6-1dc8-47c4-b991-ac3fe929a8d4 req-b17d29c4-39c3-45df-ae55-82343b44c2c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8a38979-18bb-4ff9-82d9-b0652f29089e", "name": "network-vif-deleted", "tag": "374870fe-7ff3-4e9d-be41-1e3de6475a5a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:26.565020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d805f7c6-1dc8-47c4-b991-ac3fe929a8d4 req-b17d29c4-39c3-45df-ae55-82343b44c2c8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:26.565835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d805f7c6-1dc8-47c4-b991-ac3fe929a8d4 req-b17d29c4-39c3-45df-ae55-82343b44c2c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:26.566627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d805f7c6-1dc8-47c4-b991-ac3fe929a8d4 req-b17d29c4-39c3-45df-ae55-82343b44c2c8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:26.585135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d805f7c6-1dc8-47c4-b991-ac3fe929a8d4 req-b17d29c4-39c3-45df-ae55-82343b44c2c8 service nova] Creating event network-vif-deleted:374870fe-7ff3-4e9d-be41-1e3de6475a5a for instance b8a38979-18bb-4ff9-82d9-b0652f29089e on nch2-master-1 Oct 15 21:32:26.603621 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d805f7c6-1dc8-47c4-b991-ac3fe929a8d4 req-b17d29c4-39c3-45df-ae55-82343b44c2c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066111 Oct 15 21:32:26.608333 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3949/5656] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:32:26.693226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd14dd0707a1423d82328ac3a54ccb28', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2bfe1e4fbfd48998884d5e8ef8a3e57', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:26.722456 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dfb37482-2a9d-476d-ac68-ddf2e7f71116 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 200 len: 1671 microversion: 2.1 time: 0.423213 Oct 15 21:32:26.741344 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1708/5657] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:26 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 1671 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:26.908573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-eb537363-eec7-4141-9e31-277c0a713d92 req-b0ad825a-dcd9-4beb-9a41-8ae4c448bd06 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "451f4372-4467-42ea-b0e1-f9126802c024", "name": "network-vif-deleted", "tag": "19887850-ab68-49f0-81e0-79941afb7a0b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:26.925169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb537363-eec7-4141-9e31-277c0a713d92 req-b0ad825a-dcd9-4beb-9a41-8ae4c448bd06 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:26.926289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb537363-eec7-4141-9e31-277c0a713d92 req-b0ad825a-dcd9-4beb-9a41-8ae4c448bd06 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:26.926973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-eb537363-eec7-4141-9e31-277c0a713d92 req-b0ad825a-dcd9-4beb-9a41-8ae4c448bd06 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:26.942516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-eb537363-eec7-4141-9e31-277c0a713d92 req-b0ad825a-dcd9-4beb-9a41-8ae4c448bd06 service nova] Creating event network-vif-deleted:19887850-ab68-49f0-81e0-79941afb7a0b for instance 451f4372-4467-42ea-b0e1-f9126802c024 on nch2-master-1 Oct 15 21:32:26.953507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-eb537363-eec7-4141-9e31-277c0a713d92 req-b0ad825a-dcd9-4beb-9a41-8ae4c448bd06 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052426 Oct 15 21:32:26.960702 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3950/5658] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:27.151547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:27.152295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:27.160415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:27.161000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:27.161491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:27.162295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:27.162805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:27.163254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:27.199526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:27.200262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:27.200921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:27.201688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lazy-loading 'fault' on Instance uuid b8a38979-18bb-4ff9-82d9-b0652f29089e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:32:27.211210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=2,nova_cell1:UPDATE=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:27.291013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:27.304088 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f314cac1-5678-4e10-adc6-ffa7dc22486c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 200 len: 1365 microversion: 2.1 time: 0.156596 Oct 15 21:32:27.329067 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1709/5659] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:27 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 1365 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:27.747671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:27.748289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] [instance: 451f4372-4467-42ea-b0e1-f9126802c024] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:27.755422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:27.755906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:27.756341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:27.757028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:27.757505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:27.757929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:27.771950 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] HTTP exception thrown: Instance 451f4372-4467-42ea-b0e1-f9126802c024 could not be found. Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Request method failure captured: Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 HTTP/1.1 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyYdyovFJVhQ-DTS2g0_kJPK9ucgBPejohisBcx8zlugdRuQhRrmo0Fhc5E1I7x_wwx-1WoMpg2pnOOMM0abzemR_uyhtjPvIN8FLH0dBhkKFUyQ-q-RRQSmLbXAayx6JUL6wVLgJrt-1Nbzk3kxzgPfU7q5W0S8C_zPfX_zYYU5rdo Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e2bfe1e4fbfd48998884d5e8ef8a3e57 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e2bfe1e4fbfd48998884d5e8ef8a3e57"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e2bfe1e4fbfd48998884d5e8ef8a3e57 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkBasicOps-1261870155 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkBasicOps-1261870155-project Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: bd14dd0707a1423d82328ac3a54ccb28 Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkBasicOps-1261870155-project Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 451f4372-4467-42ea-b0e1-f9126802c024 could not be found. Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '451f4372-4467-42ea-b0e1-f9126802c024'} Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 451f4372-4467-42ea-b0e1-f9126802c024 could not be found. Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 451f4372-4467-42ea-b0e1-f9126802c024 could not be found. Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:32:27.773686 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 451f4372-4467-42ea-b0e1-f9126802c024 could not be found. Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Returning 404 to user: Instance 451f4372-4467-42ea-b0e1-f9126802c024 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f71bad02-f4f1-4758-ba36-5bb4d11a0b31 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024" status: 404 len: 111 microversion: 2.1 time: 0.031936 Oct 15 21:32:27.781027 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3951/5660] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:27 2022] GET /compute/v2.1/servers/451f4372-4467-42ea-b0e1-f9126802c024 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:32:27.953216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-56d0a8fe-9f6c-40c0-b8a1-5b55fba21c11 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] Calling method '.version_select at 0x7fe74eb950d0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:27.981428 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-56d0a8fe-9f6c-40c0-b8a1-5b55fba21c11 tempest-TestNetworkBasicOps-1261870155 tempest-TestNetworkBasicOps-1261870155-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-1702678073" status: 202 len: 0 microversion: 2.1 time: 0.032822 Oct 15 21:32:27.982381 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1710/5661] 10.222.0.16 () {58 vars in 1257 bytes} [Sat Oct 15 21:32:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-1702678073 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:32:28.339070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:28.339919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:28.347289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:28.347774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:28.348210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:28.348905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:28.349385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:28.349929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:28.365094 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e HTTP/1.1 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b8a38979-18bb-4ff9-82d9-b0652f29089e'} Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:32:28.366871 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-931753f7-d1be-4b47-a342-75877ba5c21b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 404 len: 111 microversion: 2.1 time: 0.033536 Oct 15 21:32:28.373908 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3952/5662] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:28 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:32:44.258633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:44.259431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:44.470555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d7a51ce-65ad-4ad7-b19b-eecb75625451 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Action: 'create', calling method: .version_select at 0x7fe74ec36430>, body: {"keypair": {"name": "tempest-TestServerBasicOps-1556940914", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOpldDtOVhbkIHCpRmGXsKXKbX8W38XyU/eDBChBJL6sS790S7FfawhH19uXSnJVlYZEWv0s9bGXFGmk68MwG+hfql/YiqtpoRPuS7gGRyk63M5BWE2hoodgWxbZPAeOnQ=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:44.489954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0d7a51ce-65ad-4ad7-b19b-eecb75625451 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Getting quotas for project 84ddbf82d0764067ad376f9626516cb9. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:32:44.496786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-0d7a51ce-65ad-4ad7-b19b-eecb75625451 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Getting quotas for user 98a36b3c55b1462ea9cc65bfad1b967b and project 84ddbf82d0764067ad376f9626516cb9. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:32:44.526077 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d7a51ce-65ad-4ad7-b19b-eecb75625451 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.269621 Oct 15 21:32:44.527571 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1711/5663] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:32:44 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:45.788455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerBasicOps-server-513561940", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "key_name": "tempest-TestServerBasicOps-1556940914", "security_groups": [{"name": "tempest-secgroup-smoke-1997299569"}], "config_drive": true, "metadata": {"meta1": "data1", "meta2": "data2", "metaN": "dataN"}, "networks": [{"uuid": "d92914d6-6222-4602-ae9f-deacdbcae62d"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:45.799849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:45.800572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=15 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:45.914045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] validate_networks() for [('d92914d6-6222-4602-ae9f-deacdbcae62d', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:32:46.360302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Getting quotas for project 84ddbf82d0764067ad376f9626516cb9. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:32:46.366623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Getting quotas for user 98a36b3c55b1462ea9cc65bfad1b967b and project 84ddbf82d0764067ad376f9626516cb9. Resources: dict_keys(['metadata_items']) {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:32:46.382301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:32:46.382766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:32:46.383487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:32:46.383925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:32:46.384372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:32:46.395007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:46.396065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:46.396510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:46.397351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:46.397862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:46.398534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:46.419456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Getting quotas for project 84ddbf82d0764067ad376f9626516cb9. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:32:46.425556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Getting quotas for user 98a36b3c55b1462ea9cc65bfad1b967b and project 84ddbf82d0764067ad376f9626516cb9. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:32:46.440444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:32:46.443280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:32:46.494997 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45ecd728-9210-4270-af4d-4af18f65d073 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.710617 Oct 15 21:32:46.496384 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3953/5664] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:32:45 2022] POST /compute/v2.1/servers => generated 406 bytes in 713 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:32:46.514855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1169046e-58fa-413c-9882-caf973077b76 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:46.515480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1169046e-58fa-413c-9882-caf973077b76 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:46.599246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1169046e-58fa-413c-9882-caf973077b76 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:46.608754 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1169046e-58fa-413c-9882-caf973077b76 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1302 microversion: 2.1 time: 0.104721 Oct 15 21:32:46.609441 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1712/5665] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:46 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1302 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:47.636965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:47.638412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:47.648382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:47.649432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:47.650048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:47.650948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:47.652308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:47.653111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:47.771074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:47.786429 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9efb4ef1-1461-4d8e-8175-7cc8a53f89b0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1362 microversion: 2.1 time: 0.155790 Oct 15 21:32:47.788309 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3954/5666] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:47 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1362 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:32:48.813325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:48.814520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:48.822129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:48.822638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:48.823643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:48.824948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:48.825455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:48.825906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:49.033895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:49.043736 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ca5c2fd7-96ad-4a85-b0d8-4a14bde0716a tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.234203 Oct 15 21:32:49.044831 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1713/5667] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:48 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:49.341609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-96bfaa72-d75c-4aca-b395-6595a57c72db req-45b5e7fb-5233-4002-ab7c-773405f9903c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6504681-bc85-4aaf-b4c0-175378b6e772", "tag": "9f075495-6657-4ffd-8a76-849d4b785e49"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:49.354484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-96bfaa72-d75c-4aca-b395-6595a57c72db req-45b5e7fb-5233-4002-ab7c-773405f9903c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:49.355460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-96bfaa72-d75c-4aca-b395-6595a57c72db req-45b5e7fb-5233-4002-ab7c-773405f9903c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:49.355895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-96bfaa72-d75c-4aca-b395-6595a57c72db req-45b5e7fb-5233-4002-ab7c-773405f9903c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:49.368955 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-96bfaa72-d75c-4aca-b395-6595a57c72db req-45b5e7fb-5233-4002-ab7c-773405f9903c service nova] Creating event network-changed:9f075495-6657-4ffd-8a76-849d4b785e49 for instance e6504681-bc85-4aaf-b4c0-175378b6e772 on nch2-master-1 Oct 15 21:32:49.374348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-96bfaa72-d75c-4aca-b395-6595a57c72db req-45b5e7fb-5233-4002-ab7c-773405f9903c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037584 Oct 15 21:32:49.375348 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3955/5668] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:50.067878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:50.068484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:50.076077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:50.076573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:50.077022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:50.077764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:50.078232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:50.078671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:50.210100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:50.217755 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5d49a33d-6bfb-4ec2-9600-070f52bea22e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.153863 Oct 15 21:32:50.218320 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1714/5669] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:50 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:51.241344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:51.242041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:51.249844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:51.250338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:51.250786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:51.251754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:51.252222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:51.252658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:51.384427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:51.392980 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0642ca43-2df8-4772-b2bd-3f7fde2c331e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.155859 Oct 15 21:32:51.393577 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3956/5670] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:51 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:52.415536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:52.416194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:52.423708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:52.424407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:52.424887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:52.425624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:52.426093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:52.426531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:52.563697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:52.571976 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8fda06c4-7e80-43ee-a1c0-b9bd8de6e36d tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.160138 Oct 15 21:32:52.572573 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1715/5671] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:52 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:53.595915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:53.596807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:53.604735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:53.605287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:53.605720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:53.606427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:53.606893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:53.607321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:53.756471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:53.765379 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48751bcb-c7f8-445b-ad92-44560e54563f tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.173949 Oct 15 21:32:53.765951 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3957/5672] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:53 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:54.661660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0f5c9c2a-903b-4c78-9a3f-9f4f0e036417 req-15c86ad8-9e93-4faa-a17a-c4caf1f56a73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6504681-bc85-4aaf-b4c0-175378b6e772", "name": "network-vif-plugged", "status": "completed", "tag": "9f075495-6657-4ffd-8a76-849d4b785e49"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:54.673002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0f5c9c2a-903b-4c78-9a3f-9f4f0e036417 req-15c86ad8-9e93-4faa-a17a-c4caf1f56a73 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:54.673532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0f5c9c2a-903b-4c78-9a3f-9f4f0e036417 req-15c86ad8-9e93-4faa-a17a-c4caf1f56a73 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:54.673985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0f5c9c2a-903b-4c78-9a3f-9f4f0e036417 req-15c86ad8-9e93-4faa-a17a-c4caf1f56a73 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:54.688229 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0f5c9c2a-903b-4c78-9a3f-9f4f0e036417 req-15c86ad8-9e93-4faa-a17a-c4caf1f56a73 service nova] Creating event network-vif-plugged:9f075495-6657-4ffd-8a76-849d4b785e49 for instance e6504681-bc85-4aaf-b4c0-175378b6e772 on nch2-master-1 Oct 15 21:32:54.693259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0f5c9c2a-903b-4c78-9a3f-9f4f0e036417 req-15c86ad8-9e93-4faa-a17a-c4caf1f56a73 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035295 Oct 15 21:32:54.693877 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1716/5673] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:32:54.790350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:54.791779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:54.799001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:54.799594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:54.800159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:54.800909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:54.801405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:54.801904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:54.960912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:54.970541 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36d832e6-fcd9-482c-9a23-75bb42d6190e tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.185345 Oct 15 21:32:54.971260 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3958/5674] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:54 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:56.001837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:56.002471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:56.009364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:56.009872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:56.010462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:56.011175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:56.011648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:56.012333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:56.169435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:56.178678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0c5645a5-b88f-4be4-9d39-80906ffaf9c0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1428 microversion: 2.1 time: 0.182471 Oct 15 21:32:56.179641 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1717/5675] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:55 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1428 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:57.204425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:57.205401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:57.208461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:57.209521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=10 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:57.215578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:57.216561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:57.216999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:57.217719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:57.218173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:57.218602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:57.371647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:57.380136 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b74a9d39-6f0e-4d76-9b23-fd9ae6b03a82 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1608 microversion: 2.1 time: 0.179747 Oct 15 21:32:57.380755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3959/5676] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:57 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1608 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:57.402316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:57.402930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:57.405437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:32:57.411651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:57.412153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:57.412600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:57.413785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:57.413785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:57.414232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:57.557727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:57.567421 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6556629a-1925-4828-8259-9296e4eba3ba tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1608 microversion: 2.1 time: 0.168885 Oct 15 21:32:57.568070 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1718/5677] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:57 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1608 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:57.590684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:57.591324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:57.597981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:57.598502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:57.599004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:57.599765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:57.600216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:57.600696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:57.764691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:57.773348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c6571fef-ed8a-462f-abe8-ff91900d10d3 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1608 microversion: 2.1 time: 0.186534 Oct 15 21:32:57.774040 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3960/5678] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:57 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1608 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:32:59.623879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:59.624543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:59.631491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:59.632064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:59.632641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:59.633384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:59.633852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:59.634294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:59.661472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:32:59.737541 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e4b7fdfb-cf60-4337-a55b-81d75612ba29 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 204 len: 0 microversion: 2.1 time: 0.118072 Oct 15 21:32:59.741302 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1719/5679] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:32:59 2022] DELETE /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:32:59.746565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:32:59.747822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:32:59.755485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:59.756379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:59.757018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:59.757977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:59.758638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:59.759180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:59.813730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a649631b-f7c2-4908-9b0d-a2f360993b43 req-34609699-9842-445f-a227-910793d2b220 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6504681-bc85-4aaf-b4c0-175378b6e772", "tag": "9f075495-6657-4ffd-8a76-849d4b785e49"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:32:59.824189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a649631b-f7c2-4908-9b0d-a2f360993b43 req-34609699-9842-445f-a227-910793d2b220 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:32:59.824696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a649631b-f7c2-4908-9b0d-a2f360993b43 req-34609699-9842-445f-a227-910793d2b220 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:32:59.825170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a649631b-f7c2-4908-9b0d-a2f360993b43 req-34609699-9842-445f-a227-910793d2b220 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:32:59.838695 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a649631b-f7c2-4908-9b0d-a2f360993b43 req-34609699-9842-445f-a227-910793d2b220 service nova] Creating event network-changed:9f075495-6657-4ffd-8a76-849d4b785e49 for instance e6504681-bc85-4aaf-b4c0-175378b6e772 on nch2-master-1 Oct 15 21:32:59.843807 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a649631b-f7c2-4908-9b0d-a2f360993b43 req-34609699-9842-445f-a227-910793d2b220 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035108 Oct 15 21:32:59.844941 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1720/5680] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:32:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:32:59.942756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:32:59.951071 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f98bbc09-b0e2-4c66-82d0-58779f3b2f45 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1409 microversion: 2.1 time: 0.208482 Oct 15 21:32:59.961540 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3961/5681] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:32:59 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1409 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:00.975596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:00.976243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:00.982572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:00.983077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:00.983539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:00.984251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:00.984729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:00.985227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:01.120863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:01.129459 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c8435859-fbc7-4ae6-9ede-4adbc4993292 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1409 microversion: 2.1 time: 0.159073 Oct 15 21:33:01.130041 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1721/5682] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:00 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1409 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:01.706059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-c64fdd66-1c70-4317-9223-b633b92faee7 req-a88671ab-f0b2-48ce-a53d-5e342f2fecff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4", "name": "network-vif-deleted", "tag": "3cd01562-bd65-41e3-8c19-ba0aa69de14c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:33:01.718392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c64fdd66-1c70-4317-9223-b633b92faee7 req-a88671ab-f0b2-48ce-a53d-5e342f2fecff service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:01.718992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c64fdd66-1c70-4317-9223-b633b92faee7 req-a88671ab-f0b2-48ce-a53d-5e342f2fecff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:01.719503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-c64fdd66-1c70-4317-9223-b633b92faee7 req-a88671ab-f0b2-48ce-a53d-5e342f2fecff service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:01.731036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-c64fdd66-1c70-4317-9223-b633b92faee7 req-a88671ab-f0b2-48ce-a53d-5e342f2fecff service nova] Creating event network-vif-deleted:3cd01562-bd65-41e3-8c19-ba0aa69de14c for instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 on nch1-master-1 Oct 15 21:33:01.737154 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-c64fdd66-1c70-4317-9223-b633b92faee7 req-a88671ab-f0b2-48ce-a53d-5e342f2fecff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034729 Oct 15 21:33:01.737770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3962/5683] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:33:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:33:02.155130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:02.155745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:02.163003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:02.163514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:02.164155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:02.165857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:02.166349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:02.166805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:02.205107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:02.205747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:02.206264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:02.206762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lazy-loading 'fault' on Instance uuid c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:33:02.254475 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:02.263053 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1e1a4bd5-cfb9-4291-84ed-b391df87bc3f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 200 len: 1207 microversion: 2.1 time: 0.113359 Oct 15 21:33:02.264435 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1722/5684] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:02 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 1207 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:03.288848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:03.289572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:03.296457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.296939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.297417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.298111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.298565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.298991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.312303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 HTTP/1.1 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4'} Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.313957 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9ca730b-19bb-4513-ad91-43eecbf9b5ff tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 404 len: 111 microversion: 2.1 time: 0.031670 Oct 15 21:33:03.318482 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3963/5685] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:03 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:33:03.346721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:03.347228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:03.353880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.354378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.354875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.355559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.356050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.356474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.370309 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e HTTP/1.1 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'b8a38979-18bb-4ff9-82d9-b0652f29089e'} Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:33:03.371974 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1e16dc2d-e0d3-476e-905e-7edd627c0055 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 404 len: 111 microversion: 2.1 time: 0.030645 Oct 15 21:33:03.376808 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1723/5686] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:33:03 2022] DELETE /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:33:03.382734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:03.383347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: b8a38979-18bb-4ff9-82d9-b0652f29089e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:03.389512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.389992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.390430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.391115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.391567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.391991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.405900 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e HTTP/1.1 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b8a38979-18bb-4ff9-82d9-b0652f29089e'} Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.407982 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance b8a38979-18bb-4ff9-82d9-b0652f29089e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c27d348-a49d-4067-9342-96317e21595e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e" status: 404 len: 111 microversion: 2.1 time: 0.030613 Oct 15 21:33:03.412996 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3964/5687] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:03 2022] GET /compute/v2.1/servers/b8a38979-18bb-4ff9-82d9-b0652f29089e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:33:03.878898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:03.879449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:03.885487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.885987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.886443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.887172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.887641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.888621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.905410 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 HTTP/1.1 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4'} Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:33:03.907685 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-33715631-9747-4840-9c36-a4d0a090e838 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 404 len: 111 microversion: 2.1 time: 0.034585 Oct 15 21:33:03.911723 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1724/5688] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:33:03 2022] DELETE /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:33:03.917837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:03.918480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:03.924661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.925183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.925625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.926313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:03.926772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:03.927197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:03.941617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 HTTP/1.1 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4'} Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:33:03.943133 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:33:03.947718 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-702bd6a4-d3c7-4158-8cba-78fd54cc760f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4" status: 404 len: 111 microversion: 2.1 time: 0.030462 Oct 15 21:33:03.948353 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3965/5689] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:03 2022] GET /compute/v2.1/servers/c4531a3f-dca0-455e-bcc5-1f1fbd1bddf4 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:33:04.015710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-297c56b2-b308-47d7-a654-a3bfcffb409b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: .version_select at 0x7fe74eb95af0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1386124229", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKwHX//7R/PI+HGaJmP3M1lKamBRFtnI6HzbZcx5Sui3Lvv9e9bnQO0OoKvUqzIDJ9oaA/q5MlVVhfhHzA1BFP46Fd4mrP0yEqgQPogGap5aFIykxs5w/bQCjJ2wGRqLMw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:33:04.030996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-297c56b2-b308-47d7-a654-a3bfcffb409b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:33:04.036044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-297c56b2-b308-47d7-a654-a3bfcffb409b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:33:04.058685 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-297c56b2-b308-47d7-a654-a3bfcffb409b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.047500 Oct 15 21:33:04.059268 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1725/5690] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:33:04 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:33:06.585054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1010794424", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1386124229", "security_groups": [{"name": "tempest-secgroup-smoke-1001796539"}], "block_device_mapping_v2": [{"uuid": "ff3c68c4-6b40-423b-96f4-9c16db4ba6a6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "5159a515-dfbe-44d6-b24b-a04195666fcf"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:33:06.705004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] validate_networks() for [('5159a515-dfbe-44d6-b24b-a04195666fcf', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:33:07.346919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:33:07.347344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:33:07.347840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:33:07.348247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:33:07.348656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:33:07.357651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:07.358134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:07.358571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:07.359298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:07.359761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:07.360187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:07.377011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:33:07.382232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:33:07.440362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:33:07.442654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ff3c68c4-6b40-423b-96f4-9c16db4ba6a6',volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:33:07.570750 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7c46d2a4-5b82-4554-b4fe-4632a502271c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.989468 Oct 15 21:33:07.571396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3966/5691] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:33:06 2022] POST /compute/v2.1/servers => generated 406 bytes in 991 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:33:07.582575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d210f1d1-2ff9-4a95-9af8-f9e5e1bc2670 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:07.583362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d210f1d1-2ff9-4a95-9af8-f9e5e1bc2670 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:07.641515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d210f1d1-2ff9-4a95-9af8-f9e5e1bc2670 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:07.648624 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d210f1d1-2ff9-4a95-9af8-f9e5e1bc2670 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1095 microversion: 2.1 time: 0.069973 Oct 15 21:33:07.649262 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1726/5692] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:07 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1095 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:33:08.673367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:08.674046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:08.681420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:08.682252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:08.682701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:08.684062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:08.684536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:08.685353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:08.767603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:08.775944 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d1ac776-8645-424c-96cf-a2eca01e2e0d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1211 microversion: 2.1 time: 0.108356 Oct 15 21:33:08.776733 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3967/5693] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:08 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1211 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:09.800011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:09.800705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:09.807478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:09.808066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:09.808533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:09.809262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:09.809736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:09.810207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:10.017240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:10.025153 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8505faa1-8db0-4676-966d-e8e90b57af4b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1267 microversion: 2.1 time: 0.229025 Oct 15 21:33:10.025730 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1727/5694] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:09 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1267 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:10.108830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b24e72ac-d2e1-4ff4-8270-5f06afaa5bff req-3f773241-b530-4f46-97d6-6ef4bce124a0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6a73b62c-d3dd-492d-9483-bb6bf1760fe3", "tag": "d1875697-6588-4362-afc5-70d5af65cd2c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:33:10.118877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b24e72ac-d2e1-4ff4-8270-5f06afaa5bff req-3f773241-b530-4f46-97d6-6ef4bce124a0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:10.119370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b24e72ac-d2e1-4ff4-8270-5f06afaa5bff req-3f773241-b530-4f46-97d6-6ef4bce124a0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:10.120081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b24e72ac-d2e1-4ff4-8270-5f06afaa5bff req-3f773241-b530-4f46-97d6-6ef4bce124a0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:10.132189 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b24e72ac-d2e1-4ff4-8270-5f06afaa5bff req-3f773241-b530-4f46-97d6-6ef4bce124a0 service nova] Creating event network-changed:d1875697-6588-4362-afc5-70d5af65cd2c for instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 on nch1-master-1 Oct 15 21:33:10.137052 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b24e72ac-d2e1-4ff4-8270-5f06afaa5bff req-3f773241-b530-4f46-97d6-6ef4bce124a0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033598 Oct 15 21:33:10.137841 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3968/5695] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:33:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:33:11.048833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:11.049598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:11.056474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:11.056994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:11.057465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:11.058184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:11.059313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:11.059812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:11.296124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:11.303934 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ee6ed18c-32b0-402b-84e2-230a12648881 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1267 microversion: 2.1 time: 0.259618 Oct 15 21:33:11.304862 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1728/5696] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:11 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1267 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:12.329511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:12.330230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:12.336466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:12.336955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:12.337436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:12.338135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:12.338599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:12.339034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:12.478283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:12.486618 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ba1c0b26-1d49-4a5d-92fc-4b64cd0732d1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1267 microversion: 2.1 time: 0.162527 Oct 15 21:33:12.487195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3969/5697] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:12 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1267 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:13.509865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:13.510443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:13.517533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:13.518031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:13.518479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:13.519180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:13.519647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:13.520083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:13.650076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:13.657949 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c46aedba-5626-4f6f-87c9-756fa2c2eb2e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1267 microversion: 2.1 time: 0.152071 Oct 15 21:33:13.658510 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1729/5698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:13 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:14.681837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:14.682490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:14.690028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:14.690521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:14.690966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:14.691669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:14.692132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:14.692565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:14.828111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:14.835933 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8edd3de-db11-4b57-a3bf-409674df8ede tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1267 microversion: 2.1 time: 0.158006 Oct 15 21:33:14.836506 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3970/5699] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:14 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1267 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:15.859459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:15.860204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:15.866469 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ec9a7a55-e774-4f3f-bc7f-e763090b1e50 req-96e02d17-3966-4d2c-a285-9b1e9f82c4e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6a73b62c-d3dd-492d-9483-bb6bf1760fe3", "name": "network-vif-plugged", "status": "completed", "tag": "d1875697-6588-4362-afc5-70d5af65cd2c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:33:15.867029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=2,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:15.868507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:15.869286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:15.869719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:15.870525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:15.871188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:15.871652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:15.873185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:15.877284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:15.882160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec9a7a55-e774-4f3f-bc7f-e763090b1e50 req-96e02d17-3966-4d2c-a285-9b1e9f82c4e4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:15.882742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec9a7a55-e774-4f3f-bc7f-e763090b1e50 req-96e02d17-3966-4d2c-a285-9b1e9f82c4e4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:15.883119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec9a7a55-e774-4f3f-bc7f-e763090b1e50 req-96e02d17-3966-4d2c-a285-9b1e9f82c4e4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:15.895617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=77 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:15.898282 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ec9a7a55-e774-4f3f-bc7f-e763090b1e50 req-96e02d17-3966-4d2c-a285-9b1e9f82c4e4 service nova] Creating event network-vif-plugged:d1875697-6588-4362-afc5-70d5af65cd2c for instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 on nch1-master-1 Oct 15 21:33:15.904929 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ec9a7a55-e774-4f3f-bc7f-e763090b1e50 req-96e02d17-3966-4d2c-a285-9b1e9f82c4e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042373 Oct 15 21:33:15.906240 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3971/5700] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:33:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:33:16.061522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:16.069484 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65bddec0-8ea2-442b-acb1-6281cffd106f tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1267 microversion: 2.1 time: 0.214184 Oct 15 21:33:16.070058 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1730/5701] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:15 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1267 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:17.094110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:17.094811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:17.102638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:17.103122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:17.103561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:17.104249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:17.104707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:17.105158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:17.243443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:17.252281 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9b611487-b5d3-4255-8781-e55c101ff5df tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1452 microversion: 2.1 time: 0.162824 Oct 15 21:33:17.252857 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3972/5702] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:17 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1452 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:17.274816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:33:17.275404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:33:17.282474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:17.282974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:17.283432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:17.284136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:17.284609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:17.285060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:17.418612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:33:17.426948 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bf88f935-fafd-4259-9a9a-6ae10611fd61 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1452 microversion: 2.1 time: 0.156038 Oct 15 21:33:17.427524 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1731/5703] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:33:17 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1452 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:33:19.133858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f7dd94c4-c2af-4fea-8487-0ea895a4faba req-15ea1308-13fc-4398-b45d-be49b185eac1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6a73b62c-d3dd-492d-9483-bb6bf1760fe3", "tag": "d1875697-6588-4362-afc5-70d5af65cd2c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:33:19.138731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:19.139554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:33:19.146479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f7dd94c4-c2af-4fea-8487-0ea895a4faba req-15ea1308-13fc-4398-b45d-be49b185eac1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:33:19.146967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f7dd94c4-c2af-4fea-8487-0ea895a4faba req-15ea1308-13fc-4398-b45d-be49b185eac1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:33:19.147410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f7dd94c4-c2af-4fea-8487-0ea895a4faba req-15ea1308-13fc-4398-b45d-be49b185eac1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:33:19.159541 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f7dd94c4-c2af-4fea-8487-0ea895a4faba req-15ea1308-13fc-4398-b45d-be49b185eac1 service nova] Creating event network-changed:d1875697-6588-4362-afc5-70d5af65cd2c for instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 on nch1-master-1 Oct 15 21:33:19.164418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f7dd94c4-c2af-4fea-8487-0ea895a4faba req-15ea1308-13fc-4398-b45d-be49b185eac1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034470 Oct 15 21:33:19.165289 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3973/5704] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:33:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:34:06.617005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:34:06.617625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:34:06.619856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:34:06.620609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:34:06.621339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=11 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:34:06.627671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:34:06.628171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:34:06.628876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:34:06.629619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:34:06.630091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:34:06.631062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:34:06.655784 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-26507470-4b01-47c2-a0d6-5262229c94d0 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772/os-server-password" status: 200 len: 116 microversion: 2.1 time: 0.043018 Oct 15 21:34:06.656365 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1732/5705] 10.222.0.16 () {58 vars in 1293 bytes} [Sat Oct 15 21:34:06 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772/os-server-password => generated 116 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:35:24.046346 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:35:24.047428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:24.048355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=9 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:24.049278 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:35:24.049493 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:35:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 (ip 10.222.0.16) !!! Oct 15 21:35:24.055518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:24.056133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:24.068492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:24.068977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:24.069493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:24.070209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:24.070678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:24.071107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:24.098108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:35:24.172123 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bea8385a-a3a2-4ed8-885d-df9080da5337 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 204 len: 0 microversion: 2.1 time: 0.127601 Oct 15 21:35:24.172726 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3974/5706] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:35:24 2022] DELETE /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:35:24.179744 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 21:35:24.180526 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:35:24.180688 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:35:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 (ip 10.222.0.16) !!! Oct 15 21:35:24.182739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:24.183519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:24.188785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:24.189509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:24.197339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:24.197842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:24.198290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:24.199005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:24.199469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:24.199908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:24.341261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:24.349421 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52f5b472-05f5-4c68-9bac-5f91b7fdda60 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1577 microversion: 2.1 time: 0.172091 Oct 15 21:35:24.350032 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1733/5707] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:24 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1577 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:35:25.375126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:25.375825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:25.384123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:25.384594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:25.385025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:25.385744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:25.386197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:25.386620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:25.393940 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 41900. Oct 15 21:35:25.578588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:25.586550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-311918d2-e172-4586-b835-a073611bb4b3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1577 microversion: 2.1 time: 0.216488 Oct 15 21:35:25.587129 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3975/5708] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:25 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1577 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:26.571072 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 41908. Oct 15 21:35:26.610587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:26.611225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:26.619863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:26.625873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:26.626346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:26.627098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:26.627742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:26.628358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:26.677283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c010b8b1-7b1e-45a2-bcee-9c0358430478 req-392f558d-4d97-48be-8ebb-9388fa85e39a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6a73b62c-d3dd-492d-9483-bb6bf1760fe3", "name": "network-vif-deleted", "tag": "d1875697-6588-4362-afc5-70d5af65cd2c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:35:26.689150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c010b8b1-7b1e-45a2-bcee-9c0358430478 req-392f558d-4d97-48be-8ebb-9388fa85e39a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:26.690017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c010b8b1-7b1e-45a2-bcee-9c0358430478 req-392f558d-4d97-48be-8ebb-9388fa85e39a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:26.690654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c010b8b1-7b1e-45a2-bcee-9c0358430478 req-392f558d-4d97-48be-8ebb-9388fa85e39a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:26.718859 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c010b8b1-7b1e-45a2-bcee-9c0358430478 req-392f558d-4d97-48be-8ebb-9388fa85e39a service nova] Creating event network-vif-deleted:d1875697-6588-4362-afc5-70d5af65cd2c for instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 on nch1-master-1 Oct 15 21:35:26.724844 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c010b8b1-7b1e-45a2-bcee-9c0358430478 req-392f558d-4d97-48be-8ebb-9388fa85e39a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.166684 Oct 15 21:35:26.726023 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1734/5709] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:35:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:35:26.751427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:26.764696 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-61c3555a-15bb-4e65-b9c5-b46b849feeef tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 200 len: 1226 microversion: 2.1 time: 0.157638 Oct 15 21:35:26.765563 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3976/5710] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:26 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 1226 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:27.790333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:27.791054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:27.798919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:27.799426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:27.799884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:27.800597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:27.801100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:27.801551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:27.814969 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 HTTP/1.1 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '6a73b62c-d3dd-492d-9483-bb6bf1760fe3'} Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:35:27.816592 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35e529b7-e6dc-4f04-b5b0-3d984c2c586a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 404 len: 111 microversion: 2.1 time: 0.032007 Oct 15 21:35:27.820910 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1735/5711] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:27 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:35:27.866638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1846987435", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1386124229", "security_groups": [{"name": "tempest-secgroup-smoke-1001796539"}], "block_device_mapping_v2": [{"uuid": "ff3c68c4-6b40-423b-96f4-9c16db4ba6a6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "5159a515-dfbe-44d6-b24b-a04195666fcf"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:35:27.985022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] validate_networks() for [('5159a515-dfbe-44d6-b24b-a04195666fcf', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:35:28.353873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:35:28.354299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:35:28.354776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:35:28.355187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:35:28.355600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:35:28.364944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:28.365491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:28.365925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:28.366650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:28.367105 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:28.367526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:28.386578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:35:28.392671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:35:28.453225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:35:28.455392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ff3c68c4-6b40-423b-96f4-9c16db4ba6a6',volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:35:28.583153 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf51947e-2d47-4e15-9104-ee96896c73b1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.720382 Oct 15 21:35:28.583808 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3977/5712] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:35:27 2022] POST /compute/v2.1/servers => generated 406 bytes in 722 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:35:28.595368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-023aa92a-d068-4ced-8acc-bfe1a7f13d1a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:28.596008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-023aa92a-d068-4ced-8acc-bfe1a7f13d1a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:28.656850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-023aa92a-d068-4ced-8acc-bfe1a7f13d1a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:28.664079 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-023aa92a-d068-4ced-8acc-bfe1a7f13d1a tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1095 microversion: 2.1 time: 0.073023 Oct 15 21:35:28.664640 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1736/5713] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:28 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1095 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:29.690127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:29.690758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:29.698049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:29.698677 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:29.699124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:29.699833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:29.700292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:29.700720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:29.773825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:29.782757 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08799c29-e179-43dc-87b6-fda29f34ab42 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1201 microversion: 2.1 time: 0.098845 Oct 15 21:35:29.783986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3978/5714] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:29 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1201 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:30.806181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:30.808970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:30.821514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:30.821931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:30.822399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:30.826342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:30.826921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:30.827402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:31.078321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:31.086346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d955e58b-a289-430c-876a-e2b2658373a0 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1279 microversion: 2.1 time: 0.284254 Oct 15 21:35:31.086930 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1737/5715] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:30 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1279 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:31.471875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-0ec0547f-dc85-4048-b578-03783a506382 req-3fcb326b-827b-48ef-9fbc-3a26a4ca81ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bf05dbdf-1502-4a02-87a0-ecaca5e57410", "tag": "ac232161-b824-4b06-8735-729a92e6dc93"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:35:31.486204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ec0547f-dc85-4048-b578-03783a506382 req-3fcb326b-827b-48ef-9fbc-3a26a4ca81ad service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:31.486690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ec0547f-dc85-4048-b578-03783a506382 req-3fcb326b-827b-48ef-9fbc-3a26a4ca81ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:31.487128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-0ec0547f-dc85-4048-b578-03783a506382 req-3fcb326b-827b-48ef-9fbc-3a26a4ca81ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:31.499745 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-0ec0547f-dc85-4048-b578-03783a506382 req-3fcb326b-827b-48ef-9fbc-3a26a4ca81ad service nova] Creating event network-changed:ac232161-b824-4b06-8735-729a92e6dc93 for instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 on nch1-master-1 Oct 15 21:35:31.504590 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-0ec0547f-dc85-4048-b578-03783a506382 req-3fcb326b-827b-48ef-9fbc-3a26a4ca81ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037171 Oct 15 21:35:31.505886 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3979/5716] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:35:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:35:32.110754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:32.111381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:32.118346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:32.118849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:32.119302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:32.120011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:32.120486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:32.120926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:32.252832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:32.260955 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de9e86f6-bd45-47fa-88b2-8fcc7eed3ed8 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1267 microversion: 2.1 time: 0.155263 Oct 15 21:35:32.261556 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1738/5717] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:32 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:33.285578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:33.286290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:33.292822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:33.293352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:33.293827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:33.294528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:33.294996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:33.295935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:33.435445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:33.444115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c49f694a-a41f-48ee-adf0-e1439fc9f031 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1267 microversion: 2.1 time: 0.163761 Oct 15 21:35:33.444689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3980/5718] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:33 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1267 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:35:34.466992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:34.467719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:34.475526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:34.476042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:34.476502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:34.477271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:34.477750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:34.478225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:34.620653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:34.629587 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1065ccdf-6bca-4f53-8c20-d56a2e0c43a1 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1267 microversion: 2.1 time: 0.166545 Oct 15 21:35:34.630381 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1739/5719] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:34 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1267 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:35.653346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:35.653958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:35.660859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:35.661383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:35.661816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:35.662528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:35.662980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:35.663405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:35.802741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:35.810552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-08582130-a9c9-4d3b-972b-f086043e8320 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1267 microversion: 2.1 time: 0.161157 Oct 15 21:35:35.811283 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3981/5720] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:35 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1267 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:36.588527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a733ad70-a81c-4c01-b397-89de9280dc96 req-87cc330d-d791-4c41-ae74-9747074da23c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bf05dbdf-1502-4a02-87a0-ecaca5e57410", "name": "network-vif-plugged", "status": "completed", "tag": "ac232161-b824-4b06-8735-729a92e6dc93"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:35:36.599515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a733ad70-a81c-4c01-b397-89de9280dc96 req-87cc330d-d791-4c41-ae74-9747074da23c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:36.600010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a733ad70-a81c-4c01-b397-89de9280dc96 req-87cc330d-d791-4c41-ae74-9747074da23c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:36.600491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a733ad70-a81c-4c01-b397-89de9280dc96 req-87cc330d-d791-4c41-ae74-9747074da23c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:36.612807 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a733ad70-a81c-4c01-b397-89de9280dc96 req-87cc330d-d791-4c41-ae74-9747074da23c service nova] Creating event network-vif-plugged:ac232161-b824-4b06-8735-729a92e6dc93 for instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 on nch1-master-1 Oct 15 21:35:36.617822 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a733ad70-a81c-4c01-b397-89de9280dc96 req-87cc330d-d791-4c41-ae74-9747074da23c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097025 Oct 15 21:35:36.618404 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1740/5721] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:35:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:35:36.833406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:36.834023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:36.840888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:36.841412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:36.841856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:36.843058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:36.843517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:36.843946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:36.980710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:36.988644 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df5f20df-7e28-4cbc-8135-1d811afb3ceb tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1267 microversion: 2.1 time: 0.159266 Oct 15 21:35:36.989256 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3982/5722] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:36 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:38.013099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:38.013869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:38.020303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:38.020797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:38.021272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:38.021975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:38.022446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:38.022885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:38.154919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:38.163691 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f316c2a3-0176-4154-9e86-c668a8449f88 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1453 microversion: 2.1 time: 0.155678 Oct 15 21:35:38.164264 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1741/5723] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:38 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1453 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:38.187296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:35:38.187922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:35:38.195448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:38.195987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:38.196415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:38.197222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:38.198076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:38.198594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:38.330207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:35:38.337997 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-319e7e29-062f-4e69-9100-1044ca5d66c7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1453 microversion: 2.1 time: 0.154825 Oct 15 21:35:38.338572 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3983/5724] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:35:38 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1453 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:35:39.910448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5a317359-da64-4d0a-b086-ac16bc7f2c21 req-b2fe59d5-bda6-4bc0-ac70-7dbc4d3740bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bf05dbdf-1502-4a02-87a0-ecaca5e57410", "tag": "ac232161-b824-4b06-8735-729a92e6dc93"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:35:39.915547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:35:39.923103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a317359-da64-4d0a-b086-ac16bc7f2c21 req-b2fe59d5-bda6-4bc0-ac70-7dbc4d3740bc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:35:39.923608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a317359-da64-4d0a-b086-ac16bc7f2c21 req-b2fe59d5-bda6-4bc0-ac70-7dbc4d3740bc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:35:39.924066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a317359-da64-4d0a-b086-ac16bc7f2c21 req-b2fe59d5-bda6-4bc0-ac70-7dbc4d3740bc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:35:39.935701 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5a317359-da64-4d0a-b086-ac16bc7f2c21 req-b2fe59d5-bda6-4bc0-ac70-7dbc4d3740bc service nova] Creating event network-changed:ac232161-b824-4b06-8735-729a92e6dc93 for instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 on nch1-master-1 Oct 15 21:35:39.940884 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5a317359-da64-4d0a-b086-ac16bc7f2c21 req-b2fe59d5-bda6-4bc0-ac70-7dbc4d3740bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034160 Oct 15 21:35:39.941510 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1742/5725] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:35:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:37:31.160822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:31.161611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:31.162605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:31.163350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:31.169201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-610964332", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1386124229", "security_groups": [{"name": "tempest-secgroup-smoke-1001796539"}], "block_device_mapping_v2": [{"uuid": "b0e4b7e7-1909-4976-971e-dcd2d5087f23", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "5159a515-dfbe-44d6-b24b-a04195666fcf"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:37:31.290841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] validate_networks() for [('5159a515-dfbe-44d6-b24b-a04195666fcf', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:37:31.782007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:37:31.782446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:37:31.782924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:37:31.783345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:37:31.783768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:37:31.792781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:31.793859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:31.794317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:31.795058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:31.795523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:31.795956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:31.813630 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:37:31.818943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Getting quotas for user 32a6ad0f3d3b44d48df46e91d80abc95 and project b11f869e7a3e49e59e3236788206474b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:37:31.877753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:37:31.879994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b0e4b7e7-1909-4976-971e-dcd2d5087f23',volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:37:32.009546 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-faec9971-632b-4947-be5a-1a9d482ffd9b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.850374 Oct 15 21:37:32.010202 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3984/5726] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:37:31 2022] POST /compute/v2.1/servers => generated 406 bytes in 852 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:37:32.021002 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] AMQP server on 10.222.0.16:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Oct 15 21:37:32.022691 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] AMQP server on 10.222.0.16:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Oct 15 21:37:32.023262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=22 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:32.024061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=38 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:32.024970 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:37:32.025162 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:37:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 (ip 10.222.0.16) !!! Oct 15 21:37:32.034216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-35a3de76-247d-42f3-a29a-9af04912ff8b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:32.034982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-35a3de76-247d-42f3-a29a-9af04912ff8b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:32.148270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-35a3de76-247d-42f3-a29a-9af04912ff8b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:32.155905 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-35a3de76-247d-42f3-a29a-9af04912ff8b tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1094 microversion: 2.1 time: 0.137682 Oct 15 21:37:32.156459 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1743/5727] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:32 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1094 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:33.179644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3bfb58c2-4c4c-4984-b144-7c9fbd2d30e6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:33.181101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3bfb58c2-4c4c-4984-b144-7c9fbd2d30e6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:33.276415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3bfb58c2-4c4c-4984-b144-7c9fbd2d30e6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:33.299174 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3bfb58c2-4c4c-4984-b144-7c9fbd2d30e6 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1094 microversion: 2.1 time: 0.124342 Oct 15 21:37:33.300436 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3985/5728] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:33 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1094 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:34.323754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:34.324426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:34.333849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:34.334355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:34.334812 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:34.335527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:34.336004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:34.336449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:34.341918 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 42166. Oct 15 21:37:34.345573 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 42164. Oct 15 21:37:34.401739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:34.410240 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dbe92654-e953-4fbc-ac1f-5491c29b5379 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1144 microversion: 2.1 time: 0.090452 Oct 15 21:37:34.410806 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1744/5729] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:34 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1144 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:35.433179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:35.434799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:35.442403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:35.442903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:35.443534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:35.444459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:35.444935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:35.445401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:35.515752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:35.524432 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2d3da53-0a34-4e7d-a81d-17c677395d22 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1210 microversion: 2.1 time: 0.095159 Oct 15 21:37:35.536113 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3986/5730] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:35 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1210 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:36.548078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:36.548819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:36.555674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:36.556114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:36.556561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:36.557326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:36.557800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:36.558237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:36.754794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:36.764174 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-315fa0ad-a4ff-4f9a-b2af-6269f0a4bb06 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1266 microversion: 2.1 time: 0.220018 Oct 15 21:37:36.764760 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1745/5731] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:36 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1266 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:36.810578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f3c991e0-dd34-4c6e-ba08-da471ba5f347 req-0ce9dcc9-b17b-4af6-8dab-5fd902f000c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16c13143-ae26-4c34-968e-3755b7056425", "tag": "a09cc48e-da69-4caa-9073-d8fe0450522c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:37:36.825543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f3c991e0-dd34-4c6e-ba08-da471ba5f347 req-0ce9dcc9-b17b-4af6-8dab-5fd902f000c3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:36.826153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f3c991e0-dd34-4c6e-ba08-da471ba5f347 req-0ce9dcc9-b17b-4af6-8dab-5fd902f000c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:36.826658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f3c991e0-dd34-4c6e-ba08-da471ba5f347 req-0ce9dcc9-b17b-4af6-8dab-5fd902f000c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:36.843833 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f3c991e0-dd34-4c6e-ba08-da471ba5f347 req-0ce9dcc9-b17b-4af6-8dab-5fd902f000c3 service nova] Creating event network-changed:a09cc48e-da69-4caa-9073-d8fe0450522c for instance 16c13143-ae26-4c34-968e-3755b7056425 on nch2-master-1 Oct 15 21:37:36.849210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f3c991e0-dd34-4c6e-ba08-da471ba5f347 req-0ce9dcc9-b17b-4af6-8dab-5fd902f000c3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044117 Oct 15 21:37:36.849917 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3987/5732] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:37:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:37:37.789531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:37.790162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:37.797222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:37.797731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:37.798186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:37.798895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:37.799369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:37.799815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:37.933251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:37.941101 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-01916bce-119e-46f0-9a46-246e69b6d8ee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1266 microversion: 2.1 time: 0.155533 Oct 15 21:37:37.941667 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1746/5733] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:37 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:38.964283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:38.965246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:38.972208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:38.972699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:38.973179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:38.973887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:38.974358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:38.974793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:39.105992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:39.113911 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0fe87e0a-e152-4388-8a7f-9efd6e899401 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1266 microversion: 2.1 time: 0.153893 Oct 15 21:37:39.114484 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3988/5734] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:38 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:40.137683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:40.138313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:40.145504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:40.146010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:40.146460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:40.147275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:40.147754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:40.148197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:40.291785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:40.300919 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-38377365-cf9e-4eff-abe3-eaf5f38f1f8e tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1266 microversion: 2.1 time: 0.167905 Oct 15 21:37:40.301549 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1747/5735] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:40 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:41.324668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:41.325551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:41.332494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:41.333064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:41.333508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:41.334210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:41.334668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:41.335092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:41.480681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:41.488434 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36679fa4-2355-4a30-b870-9f909f0001f9 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1266 microversion: 2.1 time: 0.167978 Oct 15 21:37:41.489000 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3989/5736] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:41 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:42.279764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5428f109-e76f-41b4-aef3-1b1e521c7414 req-0de5d7d3-fbb0-44e2-8a66-e7ca780eacab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16c13143-ae26-4c34-968e-3755b7056425", "name": "network-vif-plugged", "status": "completed", "tag": "a09cc48e-da69-4caa-9073-d8fe0450522c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:37:42.286868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:42.290995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5428f109-e76f-41b4-aef3-1b1e521c7414 req-0de5d7d3-fbb0-44e2-8a66-e7ca780eacab service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:42.291540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5428f109-e76f-41b4-aef3-1b1e521c7414 req-0de5d7d3-fbb0-44e2-8a66-e7ca780eacab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:42.292194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5428f109-e76f-41b4-aef3-1b1e521c7414 req-0de5d7d3-fbb0-44e2-8a66-e7ca780eacab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:42.305322 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5428f109-e76f-41b4-aef3-1b1e521c7414 req-0de5d7d3-fbb0-44e2-8a66-e7ca780eacab service nova] Creating event network-vif-plugged:a09cc48e-da69-4caa-9073-d8fe0450522c for instance 16c13143-ae26-4c34-968e-3755b7056425 on nch2-master-1 Oct 15 21:37:42.311350 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5428f109-e76f-41b4-aef3-1b1e521c7414 req-0de5d7d3-fbb0-44e2-8a66-e7ca780eacab service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035564 Oct 15 21:37:42.311946 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1748/5737] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:37:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:37:42.512463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:42.513099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:42.515057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:42.521416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:42.521900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:42.522333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:42.523022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:42.523476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:42.523916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:42.931939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:42.940255 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c067cc76-abfb-48da-9faf-843067a31906 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1266 microversion: 2.1 time: 0.432507 Oct 15 21:37:42.940828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3990/5738] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:42 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1266 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:43.964344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:43.965258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:43.971597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:43.972095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:43.972550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:43.973285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:43.973756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:43.974193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:44.104387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:44.113408 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-39297d1e-816d-4048-b760-e3a7f69b1a24 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1451 microversion: 2.1 time: 0.154137 Oct 15 21:37:44.113982 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1749/5739] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:43 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1451 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:44.136216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:37:44.136833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:37:44.138705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:37:44.144910 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:44.145452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:44.145891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:44.146584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:44.147041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:44.147467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:44.282516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:37:44.290561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0688f34e-d9d1-4ada-aabe-6f4c3566370c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1451 microversion: 2.1 time: 0.158433 Oct 15 21:37:44.291129 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3991/5740] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:37:44 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1451 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:37:46.006897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b82f4e2a-fef5-4277-b967-0ac6bb5a9ce1 req-127cb81e-5daa-4af1-a6d8-30c40f48324d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16c13143-ae26-4c34-968e-3755b7056425", "tag": "a09cc48e-da69-4caa-9073-d8fe0450522c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:37:46.020272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b82f4e2a-fef5-4277-b967-0ac6bb5a9ce1 req-127cb81e-5daa-4af1-a6d8-30c40f48324d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:37:46.020911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b82f4e2a-fef5-4277-b967-0ac6bb5a9ce1 req-127cb81e-5daa-4af1-a6d8-30c40f48324d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:37:46.021388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b82f4e2a-fef5-4277-b967-0ac6bb5a9ce1 req-127cb81e-5daa-4af1-a6d8-30c40f48324d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:37:46.034877 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b82f4e2a-fef5-4277-b967-0ac6bb5a9ce1 req-127cb81e-5daa-4af1-a6d8-30c40f48324d service nova] Creating event network-changed:a09cc48e-da69-4caa-9073-d8fe0450522c for instance 16c13143-ae26-4c34-968e-3755b7056425 on nch2-master-1 Oct 15 21:37:46.040016 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b82f4e2a-fef5-4277-b967-0ac6bb5a9ce1 req-127cb81e-5daa-4af1-a6d8-30c40f48324d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037444 Oct 15 21:37:46.040918 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1750/5741] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:37:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:38:10.222706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:10.223920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=34 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:10.285716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:10.286220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:10.293372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:10.293932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:10.294411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:10.295307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:10.295801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:10.296231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:10.325892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:38:10.421518 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf8589ef-b23f-4735-ab7e-e7c8369d72a5 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 204 len: 0 microversion: 2.1 time: 0.205525 Oct 15 21:38:10.422140 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3992/5742] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:38:10 2022] DELETE /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:38:10.432782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:10.433788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:10.436946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=18 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:10.437695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=34 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:10.445921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:10.446491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:10.446956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:10.447688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:10.448175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:10.448623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:10.607994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:38:10.616386 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f3dbcca-a21a-4eed-8139-cd5e02e2a12b tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1733 microversion: 2.1 time: 0.189092 Oct 15 21:38:10.616987 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1751/5743] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:10 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1733 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:38:11.643479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:11.644498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:11.652842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:11.653372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:11.653795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:11.654807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:11.655266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:11.655687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:11.745645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '98a36b3c55b1462ea9cc65bfad1b967b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '84ddbf82d0764067ad376f9626516cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:38:11.753696 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7c1d874-b26c-438d-a262-62c3165149f6 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 200 len: 1665 microversion: 2.1 time: 0.115238 Oct 15 21:38:11.754324 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3993/5744] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:11 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 1665 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:11.971108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0725e015-7106-4049-81b4-420c87f14ca0 req-22961e95-e3df-4486-9815-bbee25ffbcf0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6504681-bc85-4aaf-b4c0-175378b6e772", "name": "network-vif-deleted", "tag": "9f075495-6657-4ffd-8a76-849d4b785e49"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:11.986394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0725e015-7106-4049-81b4-420c87f14ca0 req-22961e95-e3df-4486-9815-bbee25ffbcf0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:11.987282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0725e015-7106-4049-81b4-420c87f14ca0 req-22961e95-e3df-4486-9815-bbee25ffbcf0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:11.987994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0725e015-7106-4049-81b4-420c87f14ca0 req-22961e95-e3df-4486-9815-bbee25ffbcf0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:12.007095 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0725e015-7106-4049-81b4-420c87f14ca0 req-22961e95-e3df-4486-9815-bbee25ffbcf0 service nova] Creating event network-vif-deleted:9f075495-6657-4ffd-8a76-849d4b785e49 for instance e6504681-bc85-4aaf-b4c0-175378b6e772 on nch2-master-1 Oct 15 21:38:12.014567 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0725e015-7106-4049-81b4-420c87f14ca0 req-22961e95-e3df-4486-9815-bbee25ffbcf0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047039 Oct 15 21:38:12.020145 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1752/5745] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:38:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:38:12.779578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:12.780326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:12.787905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:12.788410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:12.788878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:12.789627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:12.790099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:12.790537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:12.804472 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] HTTP exception thrown: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Request method failure captured: Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 HTTP/1.1 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyb2UOQbwXl0Pa8r9cK3R8uTq6WM-3gugZMiFsRvLTPc1cbOfbWHxBBZfHplt0RZsG1bEBc8Ch09nHhAsyP9o8NQ3g3z9vHB8_n2WbeVhAMjsZoy_OOugqpXmshpxEX2FpYbSMS-amnlT-rCo7MV3eNOsdDB7RTBNtyJheuj3HzQ2qc Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 84ddbf82d0764067ad376f9626516cb9 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 84ddbf82d0764067ad376f9626516cb9 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestServerBasicOps-2119829556-project Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 98a36b3c55b1462ea9cc65bfad1b967b Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestServerBasicOps-2119829556-project Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e6504681-bc85-4aaf-b4c0-175378b6e772'} Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:12.806759 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Returning 404 to user: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-65631407-787b-43f3-9e47-1cee01ee47d9 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 404 len: 111 microversion: 2.1 time: 0.032662 Oct 15 21:38:12.811682 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3994/5746] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:12 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:38:13.300911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:13.302142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:13.309247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:13.309751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:13.310201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:13.310909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:13.311383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:13.311827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:13.325266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] HTTP exception thrown: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Request method failure captured: Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 HTTP/1.1 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyb2UOQbwXl0Pa8r9cK3R8uTq6WM-3gugZMiFsRvLTPc1cbOfbWHxBBZfHplt0RZsG1bEBc8Ch09nHhAsyP9o8NQ3g3z9vHB8_n2WbeVhAMjsZoy_OOugqpXmshpxEX2FpYbSMS-amnlT-rCo7MV3eNOsdDB7RTBNtyJheuj3HzQ2qc Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 84ddbf82d0764067ad376f9626516cb9 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 84ddbf82d0764067ad376f9626516cb9 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestServerBasicOps-2119829556-project Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 98a36b3c55b1462ea9cc65bfad1b967b Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestServerBasicOps-2119829556-project Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'e6504681-bc85-4aaf-b4c0-175378b6e772'} Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:38:13.326945 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Returning 404 to user: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f0b03ee-7aa9-454a-89ff-081d36c765c2 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 404 len: 111 microversion: 2.1 time: 0.032682 Oct 15 21:38:13.331136 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1753/5747] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:38:13 2022] DELETE /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:38:13.338207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:13.338949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] [instance: e6504681-bc85-4aaf-b4c0-175378b6e772] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:13.346247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:13.346740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:13.347183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:13.347885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:13.348352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:13.348783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:13.362841 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] HTTP exception thrown: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Request method failure captured: Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 HTTP/1.1 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyb2UOQbwXl0Pa8r9cK3R8uTq6WM-3gugZMiFsRvLTPc1cbOfbWHxBBZfHplt0RZsG1bEBc8Ch09nHhAsyP9o8NQ3g3z9vHB8_n2WbeVhAMjsZoy_OOugqpXmshpxEX2FpYbSMS-amnlT-rCo7MV3eNOsdDB7RTBNtyJheuj3HzQ2qc Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 84ddbf82d0764067ad376f9626516cb9 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/84ddbf82d0764067ad376f9626516cb9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 84ddbf82d0764067ad376f9626516cb9 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestServerBasicOps-2119829556 Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestServerBasicOps-2119829556-project Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 98a36b3c55b1462ea9cc65bfad1b967b Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestServerBasicOps-2119829556-project Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e6504681-bc85-4aaf-b4c0-175378b6e772'} Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:13.364387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Returning 404 to user: Instance e6504681-bc85-4aaf-b4c0-175378b6e772 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-49004491-5c7a-4fb3-9d6a-7828fa757f93 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772" status: 404 len: 111 microversion: 2.1 time: 0.031735 Oct 15 21:38:13.369694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3995/5748] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:13 2022] GET /compute/v2.1/servers/e6504681-bc85-4aaf-b4c0-175378b6e772 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:38:13.534093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1361b5ce-066b-4deb-b1bc-4de68dfd29b7 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] Calling method '.version_select at 0x7fe74f239310>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:13.559184 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1361b5ce-066b-4deb-b1bc-4de68dfd29b7 tempest-TestServerBasicOps-2119829556 tempest-TestServerBasicOps-2119829556-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestServerBasicOps-1556940914" status: 202 len: 0 microversion: 2.1 time: 0.029006 Oct 15 21:38:13.560308 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1754/5749] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 21:38:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestServerBasicOps-1556940914 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:38:31.251793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:31.253396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:31.398589 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:31.400431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:31.401012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:31.401682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:31.402649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:31.403358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:31.404015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:31.434351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a844e6b0-8765-4c92-8f73-3557188831cb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/os-availability-zone/detail" status: 200 len: 613 microversion: 2.1 time: 0.187534 Oct 15 21:38:31.435243 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3996/5750] 10.222.0.16 () {58 vars in 1185 bytes} [Sat Oct 15 21:38:31 2022] GET /compute/v2.1/os-availability-zone/detail => generated 613 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:38:31.548431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=10 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:31.549237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=9,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:31.554073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerMultinode-server-716018606", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "availability_zone": "nova:nch1-master-1", "networks": [{"uuid": "52648a48-b7fa-47e0-b923-f2d001c24598"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:31.621401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] validate_networks() for [('52648a48-b7fa-47e0-b923-f2d001c24598', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:38:32.105334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:38:32.105773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:38:32.106261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:38:32.106687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:38:32.107111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:38:32.116036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:32.116528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:32.116978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:32.117748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:32.118782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:32.119219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:32.135998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Getting quotas for project 8ae615766ce94a5e87b5e65a40075deb. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:38:32.141218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Getting quotas for user a0b6605213214338a02e96aee7ce7cec and project 8ae615766ce94a5e87b5e65a40075deb. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:38:32.153934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:38:32.156241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:38:32.203324 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [7545fcb5-d814-488b-b74f-e0013db97cfc] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:38:33.213080 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [7545fcb5-d814-488b-b74f-e0013db97cfc] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 42414. Oct 15 21:38:33.217720 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1779e638-40e3-4522-a8cf-4ecc521c3a0c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.671036 Oct 15 21:38:33.218878 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1755/5751] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:38:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1673 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:38:33.230649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ee27738-28c5-42c8-b718-bfd9a978cbe2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:33.231285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ee27738-28c5-42c8-b718-bfd9a978cbe2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:33.372150 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ee27738-28c5-42c8-b718-bfd9a978cbe2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1328 microversion: 2.1 time: 0.145513 Oct 15 21:38:33.378961 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3997/5752] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:33 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1328 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:34.404113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:34.404751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:34.410880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:34.411384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:34.411844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:34.413451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:34.413451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:34.413651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:34.483100 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5d4de516-d039-455a-9b6f-4fc26af5ef9c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1415 microversion: 2.1 time: 0.084422 Oct 15 21:38:34.483687 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1756/5753] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:34 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1415 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:35.507030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:35.507597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:35.517952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:35.518785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:35.519305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:35.520154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:35.520728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:35.521297 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:35.734408 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e480bb6-fbe7-418a-b677-281e87a79d0e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1463 microversion: 2.1 time: 0.231244 Oct 15 21:38:35.734987 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3998/5754] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:35 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1463 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:36.336851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-879b104a-7941-4b23-a005-f4bcd07f81a6 req-e09a7307-fec0-4b72-bf14-3fa4d43300ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "093114e0-2c49-4e62-ac31-5f71f6494c22", "tag": "ae8f4a70-5daa-4be7-bc1a-4c5a434da4d0"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:36.348856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-879b104a-7941-4b23-a005-f4bcd07f81a6 req-e09a7307-fec0-4b72-bf14-3fa4d43300ae service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:36.349530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-879b104a-7941-4b23-a005-f4bcd07f81a6 req-e09a7307-fec0-4b72-bf14-3fa4d43300ae service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:36.350690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-879b104a-7941-4b23-a005-f4bcd07f81a6 req-e09a7307-fec0-4b72-bf14-3fa4d43300ae service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:36.365341 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-879b104a-7941-4b23-a005-f4bcd07f81a6 req-e09a7307-fec0-4b72-bf14-3fa4d43300ae service nova] Creating event network-changed:ae8f4a70-5daa-4be7-bc1a-4c5a434da4d0 for instance 093114e0-2c49-4e62-ac31-5f71f6494c22 on nch1-master-1 Oct 15 21:38:36.371311 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-879b104a-7941-4b23-a005-f4bcd07f81a6 req-e09a7307-fec0-4b72-bf14-3fa4d43300ae service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038280 Oct 15 21:38:36.372018 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1757/5755] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:38:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:38:36.758414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:36.759039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:36.767451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:36.768248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:36.768693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:36.769447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:36.769915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:36.770348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:36.920350 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-099a90ec-ae25-4c25-8177-b36f992b03ea tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1463 microversion: 2.1 time: 0.166115 Oct 15 21:38:36.936937 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 3999/5756] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:36 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1463 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:37.945630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:37.947525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:37.954418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:37.955089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:37.955545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:37.956341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:37.957284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:37.957925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:38.285674 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-44202473-a73a-4fb1-b1bb-e52cf3c4b72e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1463 microversion: 2.1 time: 0.345242 Oct 15 21:38:38.286404 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1758/5757] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:37 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1463 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:38:39.310096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:39.310762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:39.319867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:39.320641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:39.321767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:39.322500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:39.323009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:39.323467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:39.474573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e8cac46-7857-4e91-bef3-d4b641831533 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1463 microversion: 2.1 time: 0.168684 Oct 15 21:38:39.484755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4000/5758] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:39 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1463 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:40.499487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:40.500036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:40.506901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:40.507408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:40.507860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:40.508577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:40.509072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:40.509524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:40.652638 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3cf884ef-23e6-4ee8-aea7-1336a828fff1 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1463 microversion: 2.1 time: 0.157554 Oct 15 21:38:40.653324 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1759/5759] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:40 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1463 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:41.450363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2e6a8d30-1151-42e8-a3e8-8b83b4682344 req-210788ca-4f87-4bfa-b02d-a04fa80e105a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "093114e0-2c49-4e62-ac31-5f71f6494c22", "name": "network-vif-plugged", "status": "completed", "tag": "ae8f4a70-5daa-4be7-bc1a-4c5a434da4d0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:41.456325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:41.462783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2e6a8d30-1151-42e8-a3e8-8b83b4682344 req-210788ca-4f87-4bfa-b02d-a04fa80e105a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:41.463287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2e6a8d30-1151-42e8-a3e8-8b83b4682344 req-210788ca-4f87-4bfa-b02d-a04fa80e105a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:41.463723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2e6a8d30-1151-42e8-a3e8-8b83b4682344 req-210788ca-4f87-4bfa-b02d-a04fa80e105a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:41.475699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2e6a8d30-1151-42e8-a3e8-8b83b4682344 req-210788ca-4f87-4bfa-b02d-a04fa80e105a service nova] Creating event network-vif-plugged:ae8f4a70-5daa-4be7-bc1a-4c5a434da4d0 for instance 093114e0-2c49-4e62-ac31-5f71f6494c22 on nch1-master-1 Oct 15 21:38:41.481711 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2e6a8d30-1151-42e8-a3e8-8b83b4682344 req-210788ca-4f87-4bfa-b02d-a04fa80e105a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035320 Oct 15 21:38:41.482311 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4001/5760] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:38:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:38:41.679514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:41.680149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:41.687262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:41.687753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:41.688204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:41.688901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:41.689411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:41.689852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:41.826389 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e9fa6104-ac33-4b12-ab25-323b6485904f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1463 microversion: 2.1 time: 0.150399 Oct 15 21:38:41.827036 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1760/5761] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:41 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1463 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:42.855475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:42.856325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:42.864701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:42.865285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:42.865725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:42.866526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:42.867280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:42.867983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.021061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7eec712-c65c-4f00-a955-b15efdab52d0 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1648 microversion: 2.1 time: 0.172655 Oct 15 21:38:43.021675 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4002/5762] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:42 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1648 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:43.047939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:43.048529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:43.050969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:43.051927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:43.059518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:43.060014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:43.060626 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.062087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:43.062595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:43.063168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.210625 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e58b4f57-14ca-4285-9411-05d7952fe4c3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1648 microversion: 2.1 time: 0.167910 Oct 15 21:38:43.211464 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1761/5763] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:43 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1648 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:43.235744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:43.236519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:43.243486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:43.244061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:43.244643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.245395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:43.245864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:43.246315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.370220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:43.398976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a7b8393-21d3-4547-bb48-a3a57a75e25e tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1648 microversion: 2.1 time: 0.168186 Oct 15 21:38:43.399576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4003/5764] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:43 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1648 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:43.465822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerMultinode-server-1295712947", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "availability_zone": "nova:nch2-master-1", "networks": [{"uuid": "52648a48-b7fa-47e0-b923-f2d001c24598"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:43.533837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] validate_networks() for [('52648a48-b7fa-47e0-b923-f2d001c24598', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:38:43.926053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:38:43.926496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:38:43.927152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:38:43.927870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:38:43.928427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:38:43.940043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:43.940684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:43.941348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.942121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:43.942603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:43.943203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:43.964066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Getting quotas for project 8ae615766ce94a5e87b5e65a40075deb. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:38:43.970773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Getting quotas for user a0b6605213214338a02e96aee7ce7cec and project 8ae615766ce94a5e87b5e65a40075deb. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:38:43.987636 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:38:43.990384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:38:44.042839 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2d5c767d-b348-4e92-b38f-a86c2799434a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.581040 Oct 15 21:38:44.043529 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1762/5765] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:38:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 583 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:38:44.056701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0cf36e9e-327a-4b34-b78d-51095f6880ff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:44.057418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0cf36e9e-327a-4b34-b78d-51095f6880ff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:44.139108 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0cf36e9e-327a-4b34-b78d-51095f6880ff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1329 microversion: 2.1 time: 0.086708 Oct 15 21:38:44.139702 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4004/5766] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:44 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1329 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:45.166079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:45.166723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:45.174423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:45.175059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:45.175588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:45.176588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:45.177097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:45.177554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:45.277788 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0aaff3b9-0e66-4fb3-909f-4066e759eac3 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1434 microversion: 2.1 time: 0.117175 Oct 15 21:38:45.278568 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1763/5767] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:45 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1434 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:46.303284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:46.304213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:46.314581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:46.315084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:46.315619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:46.316259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:46.316725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:46.317176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:46.329911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ffac7f9c-f301-4e9b-a6d2-f84458648226 req-8911c152-7dc7-4537-b5a9-d5166b2e26fc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d301df30-a0d8-457e-b5ff-344a091d4465", "tag": "bd1d4644-0b88-4f3b-ac71-a098ce58ac04"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:46.342679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ffac7f9c-f301-4e9b-a6d2-f84458648226 req-8911c152-7dc7-4537-b5a9-d5166b2e26fc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:46.343574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ffac7f9c-f301-4e9b-a6d2-f84458648226 req-8911c152-7dc7-4537-b5a9-d5166b2e26fc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:46.344356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ffac7f9c-f301-4e9b-a6d2-f84458648226 req-8911c152-7dc7-4537-b5a9-d5166b2e26fc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:46.355760 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ffac7f9c-f301-4e9b-a6d2-f84458648226 req-8911c152-7dc7-4537-b5a9-d5166b2e26fc service nova] Creating event network-changed:bd1d4644-0b88-4f3b-ac71-a098ce58ac04 for instance d301df30-a0d8-457e-b5ff-344a091d4465 on nch2-master-1 Oct 15 21:38:46.361642 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ffac7f9c-f301-4e9b-a6d2-f84458648226 req-8911c152-7dc7-4537-b5a9-d5166b2e26fc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035604 Oct 15 21:38:46.362257 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1764/5768] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:38:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:38:46.566073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe5725a6-d457-4a65-be1c-e9482fd67dff tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.267042 Oct 15 21:38:46.566744 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4005/5769] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:46 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:47.591037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:47.591779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:47.598646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:47.599143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:47.599619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:47.600318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:47.600782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:47.601259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:47.730142 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8d97ff6a-2b96-4b42-971a-de2a6c4e22bf tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.143953 Oct 15 21:38:47.730689 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1765/5770] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:47 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:48.753853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:48.754453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:48.761175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:48.761660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:48.762093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:48.762782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:48.763230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:48.763658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:48.924773 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e491d488-5803-42d4-9a81-045b6c10659d tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.174815 Oct 15 21:38:48.926303 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4006/5771] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:48 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:49.947748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:49.948377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:49.964697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:49.965259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:49.965722 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:49.966448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:49.966956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:49.967440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:50.092106 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-729f559e-d1c7-4d00-9cc4-2ded31b93c82 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.148061 Oct 15 21:38:50.093182 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1766/5772] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:49 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:51.115948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:51.116519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:51.123848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:51.124363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:51.124836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:51.125652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:51.126289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:51.126662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:51.260213 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69e27d96-726d-4672-b21a-eebf3cfe68a2 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.148189 Oct 15 21:38:51.261143 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4007/5773] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:51 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:52.284733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:52.285408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:52.292357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:52.292855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:52.293340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:52.294046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:52.294511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:52.294950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:52.435295 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-60f64ee4-4a0d-485e-bbac-77546a26907c tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.154629 Oct 15 21:38:52.435938 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1767/5774] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:52 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:52.544062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7639969b-0063-40ea-a7a9-b3b0bdc7bdd4 req-16e0b56a-3e39-4274-9db8-01c44a3ca545 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d301df30-a0d8-457e-b5ff-344a091d4465", "name": "network-vif-plugged", "status": "completed", "tag": "bd1d4644-0b88-4f3b-ac71-a098ce58ac04"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:52.557286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7639969b-0063-40ea-a7a9-b3b0bdc7bdd4 req-16e0b56a-3e39-4274-9db8-01c44a3ca545 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:52.558315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7639969b-0063-40ea-a7a9-b3b0bdc7bdd4 req-16e0b56a-3e39-4274-9db8-01c44a3ca545 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:52.558966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7639969b-0063-40ea-a7a9-b3b0bdc7bdd4 req-16e0b56a-3e39-4274-9db8-01c44a3ca545 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:52.572652 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7639969b-0063-40ea-a7a9-b3b0bdc7bdd4 req-16e0b56a-3e39-4274-9db8-01c44a3ca545 service nova] Creating event network-vif-plugged:bd1d4644-0b88-4f3b-ac71-a098ce58ac04 for instance d301df30-a0d8-457e-b5ff-344a091d4465 on nch2-master-1 Oct 15 21:38:52.580059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7639969b-0063-40ea-a7a9-b3b0bdc7bdd4 req-16e0b56a-3e39-4274-9db8-01c44a3ca545 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039559 Oct 15 21:38:52.581128 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4008/5775] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:38:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:38:53.460273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:53.460947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:53.467913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:53.468470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:53.468930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:53.469792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:53.470281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:53.470834 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:53.622540 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ea0f768-95e9-415d-97df-40c82a7f53bb tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1464 microversion: 2.1 time: 0.167131 Oct 15 21:38:53.623171 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1768/5776] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:53 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1464 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:54.648145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:54.653432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:54.654179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:54.662963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:54.663657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:54.664161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:54.665111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:54.665691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:54.666263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:54.802277 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-57800371-1eaf-4835-b58d-c3d9648ece28 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1649 microversion: 2.1 time: 0.156194 Oct 15 21:38:54.802901 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4009/5777] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:54 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1649 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:54.825890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:54.826482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:54.828917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:54.829676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:54.836496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:54.837068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:54.837726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:54.838523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:54.839018 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:54.839461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:54.987613 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-87a1dc13-a315-497f-8d1c-2804c9b50f12 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1649 microversion: 2.1 time: 0.165642 Oct 15 21:38:54.988422 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1769/5778] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:54 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1649 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:55.011953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:55.012569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:55.019757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:55.020365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:55.020808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:55.021538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:55.022037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:55.022434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:55.160762 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-92348c2f-b97b-4494-ae16-77af9de59205 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1649 microversion: 2.1 time: 0.153111 Oct 15 21:38:55.161384 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4010/5779] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:55 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1649 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:55.188537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:55.189059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:55.200704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:55.201328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:55.201921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:55.203160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:55.203160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:55.204231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:55.231893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:38:55.312445 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-57812283-6198-4d82-9ce8-a7c61d234241 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "DELETE /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 204 len: 0 microversion: 2.1 time: 0.128025 Oct 15 21:38:55.313984 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1770/5780] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:38:55 2022] DELETE /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:38:55.321783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:55.322398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:55.329460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:55.329959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:55.330400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:55.331099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:55.331561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:55.331981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:55.464872 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d035f8a6-e0fe-40be-a1db-282436e6ea07 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1655 microversion: 2.1 time: 0.146744 Oct 15 21:38:55.465694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4011/5781] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:55 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1655 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:56.491607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:56.492325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:56.498290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:56.498792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:56.499235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:56.499932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:56.500409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:56.500826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:56.633651 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-59709f1b-a578-4f7b-80d9-18a1742ed705 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1655 microversion: 2.1 time: 0.147015 Oct 15 21:38:56.634218 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1771/5782] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:56 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1655 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:57.186489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d2a7888e-8daa-4273-a428-648396cb6e10 req-dcfb5e58-0383-4de0-b615-341e0921adb7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d301df30-a0d8-457e-b5ff-344a091d4465", "name": "network-vif-deleted", "tag": "bd1d4644-0b88-4f3b-ac71-a098ce58ac04"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:38:57.198699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2a7888e-8daa-4273-a428-648396cb6e10 req-dcfb5e58-0383-4de0-b615-341e0921adb7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:57.199287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2a7888e-8daa-4273-a428-648396cb6e10 req-dcfb5e58-0383-4de0-b615-341e0921adb7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:57.199857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2a7888e-8daa-4273-a428-648396cb6e10 req-dcfb5e58-0383-4de0-b615-341e0921adb7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:57.212609 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d2a7888e-8daa-4273-a428-648396cb6e10 req-dcfb5e58-0383-4de0-b615-341e0921adb7 service nova] Creating event network-vif-deleted:bd1d4644-0b88-4f3b-ac71-a098ce58ac04 for instance d301df30-a0d8-457e-b5ff-344a091d4465 on nch2-master-1 Oct 15 21:38:57.218053 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d2a7888e-8daa-4273-a428-648396cb6e10 req-dcfb5e58-0383-4de0-b615-341e0921adb7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035581 Oct 15 21:38:57.218997 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4012/5783] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:38:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:38:57.657231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:57.657869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:57.665211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:57.665721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:57.666380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:57.667109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:57.667588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:57.668039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:57.688006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:38:57.706141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:57.706792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:57.707248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:57.707713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lazy-loading 'fault' on Instance uuid d301df30-a0d8-457e-b5ff-344a091d4465 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:38:57.761252 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1853458d-4829-4f8f-8938-d6de4044035f tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 200 len: 1454 microversion: 2.1 time: 0.107488 Oct 15 21:38:57.761849 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1772/5784] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:57 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 1454 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:38:58.786823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:58.787554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: d301df30-a0d8-457e-b5ff-344a091d4465] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:58.793900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:58.794390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:58.794884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:58.795580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:58.796043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:58.796604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:58.810014 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] HTTP exception thrown: Instance d301df30-a0d8-457e-b5ff-344a091d4465 could not be found. Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Request method failure captured: Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 HTTP/1.1 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyhXCgoZDPTEvpUb94nt8YYrADSlUdzhqN_dbJNarQeJ41C4-5ZkqCwGM8awk650LV7ViCJEXhpeI7tDWI4kPYn-woZO7ED34ozHl_mMwV71MQHzLHPl_ZCsrS43bfk4pDK2VzpltXGimUKnq8Tugx2lACWdnb4QEHpQJvc7VVD_fbM Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 8ae615766ce94a5e87b5e65a40075deb Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestServerMultinode-1150381984 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestServerMultinode-1150381984 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 8ae615766ce94a5e87b5e65a40075deb Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestServerMultinode-1150381984 Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestServerMultinode-1150381984-project-admin Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: a0b6605213214338a02e96aee7ce7cec Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestServerMultinode-1150381984-project-admin Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d301df30-a0d8-457e-b5ff-344a091d4465 could not be found. Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd301df30-a0d8-457e-b5ff-344a091d4465'} Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d301df30-a0d8-457e-b5ff-344a091d4465 could not be found. Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d301df30-a0d8-457e-b5ff-344a091d4465 could not be found. Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:38:58.811655 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d301df30-a0d8-457e-b5ff-344a091d4465 could not be found. Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Returning 404 to user: Instance d301df30-a0d8-457e-b5ff-344a091d4465 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03e5a4b6-d84e-44bc-9e02-bccd2394896b tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465" status: 404 len: 111 microversion: 2.1 time: 0.032167 Oct 15 21:38:58.816981 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4013/5785] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:58 2022] GET /compute/v2.1/servers/d301df30-a0d8-457e-b5ff-344a091d4465 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:38:58.821551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:58.822069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:58.829023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:58.829558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:58.830011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:58.831243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:58.831850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:58.832299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:58.859110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:38:58.934785 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bb0a5b61-5edc-48c8-b61a-27ca8aafb659 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "DELETE /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 204 len: 0 microversion: 2.1 time: 0.116498 Oct 15 21:38:58.935462 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1773/5786] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:38:58 2022] DELETE /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:38:58.944128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:38:58.944762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:38:58.950734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:58.951210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:58.951638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:58.952319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:38:58.952772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:38:58.953232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:38:59.078342 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f525d023-b16b-443b-9c04-0eda48104891 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1654 microversion: 2.1 time: 0.138402 Oct 15 21:38:59.078916 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4014/5787] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:38:58 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1654 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:39:00.102908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:00.103634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:00.110533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:00.111038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:00.111497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:00.112725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:00.113235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:00.113682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:00.257327 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ffc8c27-93be-43b4-844b-501f58b9d418 tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 200 len: 1654 microversion: 2.1 time: 0.158596 Oct 15 21:39:00.258290 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1774/5788] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:39:00 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 1654 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:39:00.693972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1330f41f-329d-4236-abdf-4390e7f66276 req-1a9c268f-1c48-4c87-899b-97c4fcb8fb34 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "093114e0-2c49-4e62-ac31-5f71f6494c22", "name": "network-vif-deleted", "tag": "ae8f4a70-5daa-4be7-bc1a-4c5a434da4d0"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:39:00.705925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1330f41f-329d-4236-abdf-4390e7f66276 req-1a9c268f-1c48-4c87-899b-97c4fcb8fb34 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:00.706431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1330f41f-329d-4236-abdf-4390e7f66276 req-1a9c268f-1c48-4c87-899b-97c4fcb8fb34 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:00.707060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1330f41f-329d-4236-abdf-4390e7f66276 req-1a9c268f-1c48-4c87-899b-97c4fcb8fb34 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:00.719817 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1330f41f-329d-4236-abdf-4390e7f66276 req-1a9c268f-1c48-4c87-899b-97c4fcb8fb34 service nova] Creating event network-vif-deleted:ae8f4a70-5daa-4be7-bc1a-4c5a434da4d0 for instance 093114e0-2c49-4e62-ac31-5f71f6494c22 on nch1-master-1 Oct 15 21:39:00.725849 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1330f41f-329d-4236-abdf-4390e7f66276 req-1a9c268f-1c48-4c87-899b-97c4fcb8fb34 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039316 Oct 15 21:39:00.726517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4015/5789] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:39:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:39:01.281693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:01.282329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] [instance: 093114e0-2c49-4e62-ac31-5f71f6494c22] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:01.289339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:01.289842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:01.290291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:01.291084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:01.291558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:01.292005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:01.305434 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] HTTP exception thrown: Instance 093114e0-2c49-4e62-ac31-5f71f6494c22 could not be found. Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Request method failure captured: Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 HTTP/1.1 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyhXCgoZDPTEvpUb94nt8YYrADSlUdzhqN_dbJNarQeJ41C4-5ZkqCwGM8awk650LV7ViCJEXhpeI7tDWI4kPYn-woZO7ED34ozHl_mMwV71MQHzLHPl_ZCsrS43bfk4pDK2VzpltXGimUKnq8Tugx2lACWdnb4QEHpQJvc7VVD_fbM Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 8ae615766ce94a5e87b5e65a40075deb Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestServerMultinode-1150381984 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8ae615766ce94a5e87b5e65a40075deb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestServerMultinode-1150381984 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 8ae615766ce94a5e87b5e65a40075deb Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestServerMultinode-1150381984 Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestServerMultinode-1150381984-project-admin Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: a0b6605213214338a02e96aee7ce7cec Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestServerMultinode-1150381984-project-admin Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 093114e0-2c49-4e62-ac31-5f71f6494c22 could not be found. Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '093114e0-2c49-4e62-ac31-5f71f6494c22'} Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 093114e0-2c49-4e62-ac31-5f71f6494c22 could not be found. Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 093114e0-2c49-4e62-ac31-5f71f6494c22 could not be found. Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:39:01.307082 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 093114e0-2c49-4e62-ac31-5f71f6494c22 could not be found. Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] Returning 404 to user: Instance 093114e0-2c49-4e62-ac31-5f71f6494c22 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cf4a9868-0300-4649-84c7-c1f94707363a tempest-TestServerMultinode-1150381984 tempest-TestServerMultinode-1150381984-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22" status: 404 len: 111 microversion: 2.1 time: 0.030595 Oct 15 21:39:01.311599 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1775/5790] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:39:01 2022] GET /compute/v2.1/servers/093114e0-2c49-4e62-ac31-5f71f6494c22 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:39:36.457300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=46 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:36.459289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=84 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:36.460859 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:39:36.460990 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:39:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 (ip 10.222.0.16) !!! Oct 15 21:39:36.463751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=6,nova_api:UPDATE=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:36.465009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:39:36.465839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:36.466841 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:36.475730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:36.476517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:36.477243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:36.478241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:36.479074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:36.479848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:36.508468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:39:36.593739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0f876f0-22d7-49ba-8008-a0901e3e9266 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 204 len: 0 microversion: 2.1 time: 0.138392 Oct 15 21:39:36.596525 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4016/5791] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:39:36 2022] DELETE /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:39:36.605172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:36.606033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:36.612388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:36.612945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:36.613477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:36.614314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:36.615772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:36.615945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:36.650732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-671f950b-24d3-4882-8158-02e2b3b4d929 req-d7dce00c-5c0f-4bba-a793-41a8d3cebcc5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16c13143-ae26-4c34-968e-3755b7056425", "tag": "a09cc48e-da69-4caa-9073-d8fe0450522c"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:39:36.660927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-671f950b-24d3-4882-8158-02e2b3b4d929 req-d7dce00c-5c0f-4bba-a793-41a8d3cebcc5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:36.662681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-671f950b-24d3-4882-8158-02e2b3b4d929 req-d7dce00c-5c0f-4bba-a793-41a8d3cebcc5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:36.662849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-671f950b-24d3-4882-8158-02e2b3b4d929 req-d7dce00c-5c0f-4bba-a793-41a8d3cebcc5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:36.674949 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-671f950b-24d3-4882-8158-02e2b3b4d929 req-d7dce00c-5c0f-4bba-a793-41a8d3cebcc5 service nova] Creating event network-changed:a09cc48e-da69-4caa-9073-d8fe0450522c for instance 16c13143-ae26-4c34-968e-3755b7056425 on nch2-master-1 Oct 15 21:39:36.680907 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-671f950b-24d3-4882-8158-02e2b3b4d929 req-d7dce00c-5c0f-4bba-a793-41a8d3cebcc5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.223992 Oct 15 21:39:36.681915 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1776/5792] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:39:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:39:36.758503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:39:36.766888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f50ad36f-17db-45ea-ae3a-25771250afb7 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1574 microversion: 2.1 time: 0.166312 Oct 15 21:39:36.767544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4017/5793] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:39:36 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1574 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:39:37.792996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:37.793835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:37.801058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:37.801574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:37.802072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:37.802917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:37.803411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:37.803851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:38.012239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:39:38.020033 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5fcad525-89d9-4417-aed1-b9468e4f1a67 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1574 microversion: 2.1 time: 0.231338 Oct 15 21:39:38.020592 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1777/5794] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:39:37 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1574 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:39:38.690929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-57904ebc-a501-4482-91b0-92652ce6b5c9 req-8cf7acc0-191a-4258-868d-8b3e10054041 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16c13143-ae26-4c34-968e-3755b7056425", "name": "network-vif-deleted", "tag": "a09cc48e-da69-4caa-9073-d8fe0450522c"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:39:38.702623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-57904ebc-a501-4482-91b0-92652ce6b5c9 req-8cf7acc0-191a-4258-868d-8b3e10054041 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:38.703207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-57904ebc-a501-4482-91b0-92652ce6b5c9 req-8cf7acc0-191a-4258-868d-8b3e10054041 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:38.703710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-57904ebc-a501-4482-91b0-92652ce6b5c9 req-8cf7acc0-191a-4258-868d-8b3e10054041 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:38.718031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-57904ebc-a501-4482-91b0-92652ce6b5c9 req-8cf7acc0-191a-4258-868d-8b3e10054041 service nova] Creating event network-vif-deleted:a09cc48e-da69-4caa-9073-d8fe0450522c for instance 16c13143-ae26-4c34-968e-3755b7056425 on nch2-master-1 Oct 15 21:39:38.722942 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-57904ebc-a501-4482-91b0-92652ce6b5c9 req-8cf7acc0-191a-4258-868d-8b3e10054041 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035762 Oct 15 21:39:38.723755 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4018/5795] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:39:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:39:39.043819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:39.044448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:39.051614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:39.052235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:39.052964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:39.053592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:39.054070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:39.054717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:39.087854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:39.088360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:39.088816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:39.089323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lazy-loading 'fault' on Instance uuid 16c13143-ae26-4c34-968e-3755b7056425 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:39:39.133609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:39:39.141802 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3347d933-6a62-4db5-ad6a-eacb74dcea57 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 200 len: 1230 microversion: 2.1 time: 0.101925 Oct 15 21:39:39.142592 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1778/5796] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:39:39 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 1230 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:39:40.166427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:39:40.167154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 16c13143-ae26-4c34-968e-3755b7056425] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:39:40.173400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:40.173887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:40.174325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:40.175021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:39:40.175480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:39:40.175909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:39:40.479455 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance 16c13143-ae26-4c34-968e-3755b7056425 could not be found. Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 HTTP/1.1 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 16c13143-ae26-4c34-968e-3755b7056425 could not be found. Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '16c13143-ae26-4c34-968e-3755b7056425'} Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 16c13143-ae26-4c34-968e-3755b7056425 could not be found. Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 16c13143-ae26-4c34-968e-3755b7056425 could not be found. Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:39:40.481026 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 16c13143-ae26-4c34-968e-3755b7056425 could not be found. Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance 16c13143-ae26-4c34-968e-3755b7056425 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-210e77ee-d8cc-4c22-b8b1-f4bb818ac2a3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425" status: 404 len: 111 microversion: 2.1 time: 0.321240 Oct 15 21:39:40.485918 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4019/5797] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:39:40 2022] GET /compute/v2.1/servers/16c13143-ae26-4c34-968e-3755b7056425 => generated 111 bytes in 322 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:13.439710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:13.440256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:13.443142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-249c7267-9607-419a-a079-efa05be00626 req-3d65c1cc-0c1e-4329-a4f9-0ce6e5a89ec1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bf05dbdf-1502-4a02-87a0-ecaca5e57410", "tag": "ac232161-b824-4b06-8735-729a92e6dc93"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:13.444529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=8 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:13.446105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:13.451022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:13.451511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:13.452160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:13.452642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:13.453096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:13.453476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:13.454062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:13.454507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:13.458801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-249c7267-9607-419a-a079-efa05be00626 req-3d65c1cc-0c1e-4329-a4f9-0ce6e5a89ec1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:13.459358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-249c7267-9607-419a-a079-efa05be00626 req-3d65c1cc-0c1e-4329-a4f9-0ce6e5a89ec1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:13.460027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-249c7267-9607-419a-a079-efa05be00626 req-3d65c1cc-0c1e-4329-a4f9-0ce6e5a89ec1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:13.473979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-249c7267-9607-419a-a079-efa05be00626 req-3d65c1cc-0c1e-4329-a4f9-0ce6e5a89ec1 service nova] Creating event network-changed:ac232161-b824-4b06-8735-729a92e6dc93 for instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 on nch1-master-1 Oct 15 21:40:13.480155 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-249c7267-9607-419a-a079-efa05be00626 req-3d65c1cc-0c1e-4329-a4f9-0ce6e5a89ec1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041486 Oct 15 21:40:13.481339 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4020/5798] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:40:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:40:13.483452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:40:13.573398 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-12bc6586-3b4b-4f16-83ae-02275ee4094d tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 204 len: 0 microversion: 2.1 time: 0.137684 Oct 15 21:40:13.574359 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1779/5799] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:40:13 2022] DELETE /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:40:13.591559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:13.594933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:13.607114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:13.607874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:13.608482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:13.609493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:13.610151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:13.610838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:13.763343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:13.771229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ba4bfa5-4a14-4128-ad86-7d40bf827e83 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1578 microversion: 2.1 time: 0.190907 Oct 15 21:40:13.771810 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4021/5800] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:13 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1578 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:14.795745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:14.796600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:14.803407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:14.803910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:14.804363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:14.805099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:14.805577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:14.806692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:14.998862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:15.006844 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a94cec4-a44d-4f8d-970e-70118e7569dd tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1578 microversion: 2.1 time: 0.215395 Oct 15 21:40:15.007418 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1780/5801] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:14 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1578 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:15.532781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-411af27d-38f7-42a6-a04a-0cc1f40f8fd2 req-5ca2f756-b75e-4540-85a5-6d168938b781 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bf05dbdf-1502-4a02-87a0-ecaca5e57410", "name": "network-vif-deleted", "tag": "ac232161-b824-4b06-8735-729a92e6dc93"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:15.543508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-411af27d-38f7-42a6-a04a-0cc1f40f8fd2 req-5ca2f756-b75e-4540-85a5-6d168938b781 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:15.543989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-411af27d-38f7-42a6-a04a-0cc1f40f8fd2 req-5ca2f756-b75e-4540-85a5-6d168938b781 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:15.544424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-411af27d-38f7-42a6-a04a-0cc1f40f8fd2 req-5ca2f756-b75e-4540-85a5-6d168938b781 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:15.557098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-411af27d-38f7-42a6-a04a-0cc1f40f8fd2 req-5ca2f756-b75e-4540-85a5-6d168938b781 service nova] Creating event network-vif-deleted:ac232161-b824-4b06-8735-729a92e6dc93 for instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 on nch1-master-1 Oct 15 21:40:15.563216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-411af27d-38f7-42a6-a04a-0cc1f40f8fd2 req-5ca2f756-b75e-4540-85a5-6d168938b781 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034021 Oct 15 21:40:15.563822 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4022/5802] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:40:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:16.031703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:16.032338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:16.038992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:16.039517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:16.039975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:16.041499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:16.041979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:16.042429 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:16.077990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:16.078415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:16.078874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:16.079342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lazy-loading 'fault' on Instance uuid bf05dbdf-1502-4a02-87a0-ecaca5e57410 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:40:16.122222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '32a6ad0f3d3b44d48df46e91d80abc95', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b11f869e7a3e49e59e3236788206474b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:16.129507 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bd12709e-b8a3-4095-9ac2-8a21e34c3dc3 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 200 len: 1185 microversion: 2.1 time: 0.101533 Oct 15 21:40:16.130090 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1781/5803] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:16 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 1185 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:17.154316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:17.155048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: bf05dbdf-1502-4a02-87a0-ecaca5e57410] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:17.161307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:17.161791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:17.162230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:17.162924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:17.163396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:17.163828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:17.177073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 could not be found. Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 HTTP/1.1 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 could not be found. Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'bf05dbdf-1502-4a02-87a0-ecaca5e57410'} Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 could not be found. Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 could not be found. Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:17.178714 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 could not be found. Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance bf05dbdf-1502-4a02-87a0-ecaca5e57410 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c06c0c79-a9b1-45f1-8052-201ff2b7f715 tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410" status: 404 len: 111 microversion: 2.1 time: 0.030952 Oct 15 21:40:17.185464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4023/5804] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:17 2022] GET /compute/v2.1/servers/bf05dbdf-1502-4a02-87a0-ecaca5e57410 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:17.898877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:17.899750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:17.906627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:17.907163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:17.907597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:17.908295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:17.908760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:17.909233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:17.922361 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 HTTP/1.1 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '6a73b62c-d3dd-492d-9483-bb6bf1760fe3'} Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:40:17.924020 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-863442a1-321f-46c2-b621-97e8ce6c17ae tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 404 len: 111 microversion: 2.1 time: 0.031881 Oct 15 21:40:17.928695 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1782/5805] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:40:17 2022] DELETE /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:17.934883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:17.935572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] [instance: 6a73b62c-d3dd-492d-9483-bb6bf1760fe3] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:17.941645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:17.942143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:17.942592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:17.943294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:17.943760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:17.944199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:17.958270 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] HTTP exception thrown: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Request method failure captured: Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 HTTP/1.1 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyaqpsgk7ZNylGyOU3lL1Bri-wDjgutj8CSWeWasu81LpgOqUCetnLyHQ_byEcrClZMWMjijFqKk_Vc3GeUymFLYYw_H0pgwU5kLjstUd6lBB23m3kyz1-qCq7x2sVgjviqDsylINSLYLTIjkxq5MhOJ6DBmPawkcyTsGB2XLKwPCz8 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b11f869e7a3e49e59e3236788206474b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b11f869e7a3e49e59e3236788206474b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b11f869e7a3e49e59e3236788206474b Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestVolumeBootPattern-921280471 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 32a6ad0f3d3b44d48df46e91d80abc95 Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestVolumeBootPattern-921280471-project Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '6a73b62c-d3dd-492d-9483-bb6bf1760fe3'} Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:17.959813 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Returning 404 to user: Instance 6a73b62c-d3dd-492d-9483-bb6bf1760fe3 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8917203c-fd51-458c-aa1f-834e7acf457c tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3" status: 404 len: 111 microversion: 2.1 time: 0.031557 Oct 15 21:40:17.964994 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4024/5806] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:17 2022] GET /compute/v2.1/servers/6a73b62c-d3dd-492d-9483-bb6bf1760fe3 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:19.269775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5cf78984-c6b5-4616-99e3-943f8bc4efee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] Calling method '.version_select at 0x7fe74c300790>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:19.295227 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5cf78984-c6b5-4616-99e3-943f8bc4efee tempest-TestVolumeBootPattern-921280471 tempest-TestVolumeBootPattern-921280471-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1386124229" status: 202 len: 0 microversion: 2.1 time: 0.029377 Oct 15 21:40:19.295750 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1783/5807] 10.222.0.16 () {58 vars in 1263 bytes} [Sat Oct 15 21:40:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1386124229 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:40:37.919053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=10 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:37.919802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=14 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:38.031818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-1565767531", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:38.263487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] validate_networks() for None {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:40:38.538549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:40:38.538968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:40:38.539442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:40:38.539849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:40:38.540269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:40:38.549685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:38.550168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:38.550608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:38.551331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:38.551782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:38.552205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:38.569248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Getting quotas for project e41304ad412e441eb0941973df577f98. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:40:38.574403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Getting quotas for user 11e52af42d814aed8ea14a256678bce9 and project e41304ad412e441eb0941973df577f98. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:40:38.586684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:40:38.588986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:40:38.632059 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [e8f8a3fb-35ec-40da-86d5-324eb17c0caa] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:40:39.640899 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [e8f8a3fb-35ec-40da-86d5-324eb17c0caa] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 42902. Oct 15 21:40:39.644810 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-215169e7-abe3-4a0e-94f9-1b0f268b2e6b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.730976 Oct 15 21:40:39.645661 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4025/5808] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:40:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 1732 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:40:39.657182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:39.657974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:39.664171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c714047d-907a-4010-9524-7c2220f4a1d6 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:39.665212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c714047d-907a-4010-9524-7c2220f4a1d6 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:39.722551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c714047d-907a-4010-9524-7c2220f4a1d6 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:39.729546 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c714047d-907a-4010-9524-7c2220f4a1d6 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1222 microversion: 2.1 time: 0.074519 Oct 15 21:40:39.730116 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1784/5809] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:39 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1222 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:40.754515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:40.755139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:40.761421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:40.761914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:40.762612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:40.763894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:40.764380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:40.764804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:40.844408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:40.852148 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-077ca29d-c3f0-4bf0-86a0-013c9590f74e tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1280 microversion: 2.1 time: 0.102705 Oct 15 21:40:40.852720 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4026/5810] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:40 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1280 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:41.877016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:41.877739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:41.885749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:41.886281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:41.887741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:41.888763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:41.890535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:41.891136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:42.118428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:42.127502 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa9d65fe-12be-4bf7-8fa6-bd40977450e8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1322 microversion: 2.1 time: 0.255486 Oct 15 21:40:42.128244 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1785/5811] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:41 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1322 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:42.397763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e2c964a1-2841-45ce-af50-289e173953f5 req-19468551-d640-466b-938a-6404c9add826 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "05e08192-67d5-45ae-8bd0-4744c66b4291", "tag": "a9fa12fa-1a88-457a-8107-2acfca54362d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:42.407606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2c964a1-2841-45ce-af50-289e173953f5 req-19468551-d640-466b-938a-6404c9add826 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:42.408177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2c964a1-2841-45ce-af50-289e173953f5 req-19468551-d640-466b-938a-6404c9add826 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:42.408728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e2c964a1-2841-45ce-af50-289e173953f5 req-19468551-d640-466b-938a-6404c9add826 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:42.420822 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e2c964a1-2841-45ce-af50-289e173953f5 req-19468551-d640-466b-938a-6404c9add826 service nova] Creating event network-changed:a9fa12fa-1a88-457a-8107-2acfca54362d for instance 05e08192-67d5-45ae-8bd0-4744c66b4291 on nch2-master-1 Oct 15 21:40:42.426092 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e2c964a1-2841-45ce-af50-289e173953f5 req-19468551-d640-466b-938a-6404c9add826 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101522 Oct 15 21:40:42.426703 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4027/5812] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:40:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:43.150699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:43.151358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:43.158384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:43.158890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:43.159347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:43.160067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:43.160550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:43.160996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:43.291949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:43.301011 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ac5a00e4-6409-47b1-be24-635d11e51d4b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1322 microversion: 2.1 time: 0.154138 Oct 15 21:40:43.301624 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1786/5813] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:43 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1322 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:44.326312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:44.327029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:44.333309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:44.333801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:44.334251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:44.334954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:44.335420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:44.335855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:44.471238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:44.479009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77763f98-632d-4e62-8848-8955b66a00f2 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1322 microversion: 2.1 time: 0.158434 Oct 15 21:40:44.479597 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4028/5814] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:44 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1322 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:45.502901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:45.503520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:45.510805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:45.511315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:45.511807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:45.512822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:45.513337 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:45.513785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:45.647140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:45.656030 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28295991-c798-49d4-9b2a-d20cd61e742b tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1322 microversion: 2.1 time: 0.156183 Oct 15 21:40:45.656544 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1787/5815] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:45 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1322 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:46.679868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:46.680526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:46.688362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:46.688886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:46.689373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:46.690215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:46.690787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:46.691291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:46.820010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:46.827854 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4de3276-1265-4b27-bb83-3a2896ba8010 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1322 microversion: 2.1 time: 0.151954 Oct 15 21:40:46.828418 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4029/5816] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:46 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1322 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:47.148619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0fcb1ec3-76cd-48e2-aee5-1302cee9f4be req-ed63bd7f-b391-4d64-a54e-2bd2f5e35b84 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05e08192-67d5-45ae-8bd0-4744c66b4291", "name": "network-vif-plugged", "status": "completed", "tag": "a9fa12fa-1a88-457a-8107-2acfca54362d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:47.158796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0fcb1ec3-76cd-48e2-aee5-1302cee9f4be req-ed63bd7f-b391-4d64-a54e-2bd2f5e35b84 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:47.159296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0fcb1ec3-76cd-48e2-aee5-1302cee9f4be req-ed63bd7f-b391-4d64-a54e-2bd2f5e35b84 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:47.159750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0fcb1ec3-76cd-48e2-aee5-1302cee9f4be req-ed63bd7f-b391-4d64-a54e-2bd2f5e35b84 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:47.171291 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0fcb1ec3-76cd-48e2-aee5-1302cee9f4be req-ed63bd7f-b391-4d64-a54e-2bd2f5e35b84 service nova] Creating event network-vif-plugged:a9fa12fa-1a88-457a-8107-2acfca54362d for instance 05e08192-67d5-45ae-8bd0-4744c66b4291 on nch2-master-1 Oct 15 21:40:47.176397 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0fcb1ec3-76cd-48e2-aee5-1302cee9f4be req-ed63bd7f-b391-4d64-a54e-2bd2f5e35b84 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106330 Oct 15 21:40:47.176978 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1788/5817] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:40:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:40:47.850504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:47.851123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:47.858512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:47.859009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:47.859457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:47.860160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:47.860656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:47.861073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:47.989357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:47.997407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bacfdb26-17a9-46b7-a2ed-f183769aeb06 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1322 microversion: 2.1 time: 0.150728 Oct 15 21:40:47.998005 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4030/5818] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:47 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:49.025258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.025962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:49.033178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.033681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.034134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.034850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.035317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.035758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.170704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:49.178635 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-64055938-4391-4bbd-a3ac-761fab6d6524 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1472 microversion: 2.1 time: 0.158488 Oct 15 21:40:49.179213 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1789/5819] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:49 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1472 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:49.205529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:49.206268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:49.254806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31533e19-3a15-40c4-a967-1f037ba40454 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Calling method '.version_select at 0x7fe74e08cb80>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.320856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31533e19-3a15-40c4-a967-1f037ba40454 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/6e13cade98794076a20e38791ad0cddc" status: 200 len: 371 microversion: 2.1 time: 0.120687 Oct 15 21:40:49.321460 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4031/5820] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:40:49 2022] GET /compute/v2.1/os-quota-sets/6e13cade98794076a20e38791ad0cddc => generated 371 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:49.333534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3c9fe671-b90f-4447-8305-ba23ad5a5b22 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 51201, "vcpus": 1, "disk": 5, "name": "tempest-ServersAdminNegativeTestJSON-flavor-1228022685", "id": 1202248193, "os-flavor-access:is_public": "True"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:49.351320 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3c9fe671-b90f-4447-8305-ba23ad5a5b22 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 438 microversion: 2.1 time: 0.021428 Oct 15 21:40:49.351880 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1790/5821] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:40:49 2022] POST /compute/v2.1/flavors => generated 438 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:49.364530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "1202248193"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:49.366827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:49.373605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.374095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.374537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.375232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.375691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.376121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.462715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': 'e4c0698f643f4e7cb3f898c453f51fab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e13cade98794076a20e38791ad0cddc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:49.469309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Old instance type m1.heat2, new instance type tempest-ServersAdminNegativeTestJSON-flavor-1228022685 {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:40:49.475940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.476425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.476875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.477638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.478098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.478523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.499120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Getting quotas for project e41304ad412e441eb0941973df577f98. Resources: {'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:40:49.504103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Getting quotas for user 11e52af42d814aed8ea14a256678bce9 and project e41304ad412e441eb0941973df577f98. Resources: {'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:40:49.516907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.compute.api [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] ram quota exceeded for 6e13cade98794076a20e38791ad0cddc, tried to resize instance. Oct 15 21:40:49.517477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] HTTP exception thrown: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Request method failure captured: Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291/action HTTP/1.1 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 39 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyjVeE_n5HhPoaDDIyT3XR8nJbxt-Tu3t5DI1Qv0lB3-Za55WBzsUennl8TNr_0xtLjoRXu8tuuru6sXPKDrIJ4RS5jnZVOCWTEM0_iHn2Nnt5QNp1eKurqm7xSYqxGC5YhuHi55J0-dvARolz4cMRt-P8lWi4hXxxhYa0c4o_sqs9M Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 6e13cade98794076a20e38791ad0cddc Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1970586952 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member,admin Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member,admin Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6e13cade98794076a20e38791ad0cddc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6e13cade98794076a20e38791ad0cddc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6e13cade98794076a20e38791ad0cddc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6e13cade98794076a20e38791ad0cddc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1970586952 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 6e13cade98794076a20e38791ad0cddc Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1970586952 Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-1970586952-project-admin Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: e4c0698f643f4e7cb3f898c453f51fab Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1970586952-project-admin Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: {"resize": {"flavorRef": "1202248193"}} Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: exception: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '05e08192-67d5-45ae-8bd0-4744c66b4291', 'body': {'resize': {'flavorRef': '1202248193'}}} Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3878, in _check_quota_for_upsize Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, res_deltas, Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 613, in limit_check_project_and_user Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['ram'] Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:40:49.519509 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4244, in resize Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._check_quota_for_upsize(context, instance, Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3896, in _check_quota_for_upsize Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1056, in _resize Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Returning 403 to user: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a0ea85e-abf3-4dd3-92d9-5aa648d2b675 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291/action" status: 403 len: 118 microversion: 2.1 time: 0.160219 Oct 15 21:40:49.524177 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4032/5822] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:40:49 2022] POST /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291/action => generated 118 bytes in 161 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:49.530029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-32cdb5cc-6fd6-4284-8553-081c2f25e718 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.548088 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-32cdb5cc-6fd6-4284-8553-081c2f25e718 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1202248193" status: 202 len: 0 microversion: 2.1 time: 0.021415 Oct 15 21:40:49.548633 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1791/5823] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 21:40:49 2022] DELETE /compute/v2.1/flavors/1202248193 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:40:49.556405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31ae6360-ea1a-4bc1-a743-2bb65aa59b6b tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.577128 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31ae6360-ea1a-4bc1-a743-2bb65aa59b6b tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.024282 Oct 15 21:40:49.577704 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4033/5824] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:40:49 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:49.597727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cf826c51-7448-474d-8e91-6cd29f4ff5af tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Calling method '.version_select at 0x7fe74c2c4f70>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.668915 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cf826c51-7448-474d-8e91-6cd29f4ff5af tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/6e13cade98794076a20e38791ad0cddc" status: 200 len: 371 microversion: 2.1 time: 0.074700 Oct 15 21:40:49.669522 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1792/5825] 10.222.0.16 () {58 vars in 1242 bytes} [Sat Oct 15 21:40:49 2022] GET /compute/v2.1/os-quota-sets/6e13cade98794076a20e38791ad0cddc => generated 371 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:49.681665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ab7baed-d8a2-4384-8112-0af21e8117c7 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 21, "disk": 5, "name": "tempest-ServersAdminNegativeTestJSON-flavor-462923932", "id": 1858381135, "os-flavor-access:is_public": "True"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:49.699886 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ab7baed-d8a2-4384-8112-0af21e8117c7 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 436 microversion: 2.1 time: 0.021889 Oct 15 21:40:49.700458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4034/5826] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:40:49 2022] POST /compute/v2.1/flavors => generated 436 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:49.713197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "1858381135"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:49.715509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:49.721706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.722229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.722678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.723378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.723844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.724280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.808748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': 'e4c0698f643f4e7cb3f898c453f51fab', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e13cade98794076a20e38791ad0cddc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:49.815219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Old instance type m1.heat2, new instance type tempest-ServersAdminNegativeTestJSON-flavor-462923932 {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:40:49.821724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.822210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.822660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.823390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.823854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.824288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.844029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Getting quotas for project e41304ad412e441eb0941973df577f98. Resources: {'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:40:49.848994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Getting quotas for user 11e52af42d814aed8ea14a256678bce9 and project e41304ad412e441eb0941973df577f98. Resources: {'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:40:49.861582 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.api [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] cores quota exceeded for 6e13cade98794076a20e38791ad0cddc, tried to resize instance. Oct 15 21:40:49.862142 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] HTTP exception thrown: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Request method failure captured: Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291/action HTTP/1.1 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 39 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyjVeE_n5HhPoaDDIyT3XR8nJbxt-Tu3t5DI1Qv0lB3-Za55WBzsUennl8TNr_0xtLjoRXu8tuuru6sXPKDrIJ4RS5jnZVOCWTEM0_iHn2Nnt5QNp1eKurqm7xSYqxGC5YhuHi55J0-dvARolz4cMRt-P8lWi4hXxxhYa0c4o_sqs9M Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 6e13cade98794076a20e38791ad0cddc Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1970586952 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member,admin Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member,admin Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6e13cade98794076a20e38791ad0cddc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6e13cade98794076a20e38791ad0cddc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6e13cade98794076a20e38791ad0cddc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6e13cade98794076a20e38791ad0cddc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1970586952 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 6e13cade98794076a20e38791ad0cddc Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1970586952 Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersAdminNegativeTestJSON-1970586952-project-admin Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: e4c0698f643f4e7cb3f898c453f51fab Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1970586952-project-admin Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: {"resize": {"flavorRef": "1858381135"}} Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: exception: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '05e08192-67d5-45ae-8bd0-4744c66b4291', 'body': {'resize': {'flavorRef': '1858381135'}}} Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3878, in _check_quota_for_upsize Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, res_deltas, Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 613, in limit_check_project_and_user Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['cores'] Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:40:49.864201 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4244, in resize Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._check_quota_for_upsize(context, instance, Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3896, in _check_quota_for_upsize Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1056, in _resize Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Returning 403 to user: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-46f6e94d-e15b-40ff-a6d8-3090ceb26ade tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291/action" status: 403 len: 113 microversion: 2.1 time: 0.156322 Oct 15 21:40:49.868791 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1793/5827] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:40:49 2022] POST /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291/action => generated 113 bytes in 157 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:49.877534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f9ac7730-d43a-461e-af4f-091efa3a168b tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.895099 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f9ac7730-d43a-461e-af4f-091efa3a168b tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1858381135" status: 202 len: 0 microversion: 2.1 time: 0.021926 Oct 15 21:40:49.895663 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4035/5828] 10.222.0.16 () {58 vars in 1161 bytes} [Sat Oct 15 21:40:49 2022] DELETE /compute/v2.1/flavors/1858381135 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:40:49.903762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-de01e326-089b-4af1-ab99-28e98511c444 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.925371 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-de01e326-089b-4af1-ab99-28e98511c444 tempest-ServersAdminNegativeTestJSON-1970586952 tempest-ServersAdminNegativeTestJSON-1970586952-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.025338 Oct 15 21:40:49.925922 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1794/5829] 10.222.0.16 () {58 vars in 1146 bytes} [Sat Oct 15 21:40:49 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:40:49.946569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:49.947071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:49.953582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.954071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.954517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.955216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:49.955680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:49.956114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:49.982091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:40:50.060279 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55ae7b4d-059a-455c-ac73-3623fd3ca293 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "DELETE /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 204 len: 0 microversion: 2.1 time: 0.116906 Oct 15 21:40:50.061459 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4036/5830] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:40:49 2022] DELETE /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 15 21:40:50.071277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:50.071951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:50.079570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:50.080088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:50.080547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:50.081338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:50.081821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:50.082269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:50.213027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:50.220954 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c75a387-9613-4ce3-8108-8f97cfce2cdd tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1478 microversion: 2.1 time: 0.155023 Oct 15 21:40:50.221547 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1795/5831] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:50 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1478 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:51.245309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:51.246140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:51.253097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:51.253593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:51.254038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:51.254744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:51.255209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:51.255636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:51.385569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '11e52af42d814aed8ea14a256678bce9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e41304ad412e441eb0941973df577f98', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:51.394418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be4c09ed-10c2-4f82-ad90-799075249761 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 200 len: 1478 microversion: 2.1 time: 0.154340 Oct 15 21:40:51.394998 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4037/5832] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:51 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 1478 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:40:51.905718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5aef3733-9823-4c81-9aba-35b1dfe1d318 req-80bf39a1-7c71-4ed6-b69f-e70ffbdb703f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05e08192-67d5-45ae-8bd0-4744c66b4291", "name": "network-vif-deleted", "tag": "a9fa12fa-1a88-457a-8107-2acfca54362d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:51.917473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5aef3733-9823-4c81-9aba-35b1dfe1d318 req-80bf39a1-7c71-4ed6-b69f-e70ffbdb703f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:51.918227 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5aef3733-9823-4c81-9aba-35b1dfe1d318 req-80bf39a1-7c71-4ed6-b69f-e70ffbdb703f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:51.918877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5aef3733-9823-4c81-9aba-35b1dfe1d318 req-80bf39a1-7c71-4ed6-b69f-e70ffbdb703f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:51.931414 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5aef3733-9823-4c81-9aba-35b1dfe1d318 req-80bf39a1-7c71-4ed6-b69f-e70ffbdb703f service nova] Creating event network-vif-deleted:a9fa12fa-1a88-457a-8107-2acfca54362d for instance 05e08192-67d5-45ae-8bd0-4744c66b4291 on nch2-master-1 Oct 15 21:40:51.936383 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5aef3733-9823-4c81-9aba-35b1dfe1d318 req-80bf39a1-7c71-4ed6-b69f-e70ffbdb703f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035255 Oct 15 21:40:51.938013 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1796/5833] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:40:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:40:52.417050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:52.417681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] [instance: 05e08192-67d5-45ae-8bd0-4744c66b4291] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:52.425433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:52.425920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:52.426362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:52.427055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:52.427511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:52.427939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:52.441406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] HTTP exception thrown: Instance 05e08192-67d5-45ae-8bd0-4744c66b4291 could not be found. Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Request method failure captured: Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 HTTP/1.1 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyjVXWMWaz1gJONYsLccxnABr9N1Q4L4veXjW4R22H2io9UY-dassixYC9ogn0NRyuU14cCEjq2b6djr72CpQ63XPRB6-W6BJlEze8m6bHu389THhbVxusjHh9320qjdnG05sX13WTRrS4y_WgubrJ8iTCMSIo2hSQMnqW9l8_oKpww Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: e41304ad412e441eb0941973df577f98 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1972313775 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e41304ad412e441eb0941973df577f98"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e41304ad412e441eb0941973df577f98"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e41304ad412e441eb0941973df577f98"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e41304ad412e441eb0941973df577f98"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1972313775 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: e41304ad412e441eb0941973df577f98 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1972313775 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersAdminNegativeTestJSON-1972313775-project Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 11e52af42d814aed8ea14a256678bce9 Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1972313775-project Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 05e08192-67d5-45ae-8bd0-4744c66b4291 could not be found. Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '05e08192-67d5-45ae-8bd0-4744c66b4291'} Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05e08192-67d5-45ae-8bd0-4744c66b4291 could not be found. Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05e08192-67d5-45ae-8bd0-4744c66b4291 could not be found. Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:52.443055 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05e08192-67d5-45ae-8bd0-4744c66b4291 could not be found. Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] Returning 404 to user: Instance 05e08192-67d5-45ae-8bd0-4744c66b4291 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fcae5d8-4f4e-4b96-90df-135ec925b1d8 tempest-ServersAdminNegativeTestJSON-1972313775 tempest-ServersAdminNegativeTestJSON-1972313775-project] 10.222.0.16 "GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291" status: 404 len: 111 microversion: 2.1 time: 0.031392 Oct 15 21:40:52.447338 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4038/5834] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:52 2022] GET /compute/v2.1/servers/05e08192-67d5-45ae-8bd0-4744c66b4291 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:40:58.956560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1214143270", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "327f4eaf-581d-4efd-af99-d08686017fe3"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:40:59.016757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] validate_networks() for [('327f4eaf-581d-4efd-af99-d08686017fe3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:40:59.418137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:40:59.418850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:40:59.419568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:40:59.420071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:40:59.420525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:40:59.430259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:59.430769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:59.431233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:59.431986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:40:59.432466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:40:59.432911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:40:59.451233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Getting quotas for project 512ac462334c424eb45066ea7647a3a9. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:40:59.457580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Getting quotas for user c1bffd7ffed94cf595a69eaa7fa758ed and project 512ac462334c424eb45066ea7647a3a9. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:40:59.470423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:40:59.472886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:40:59.526288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-124d9db5-b228-4def-b581-0200734bd548 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.708887 Oct 15 21:40:59.527216 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1797/5835] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:40:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 711 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:40:59.538963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33d5fbb8-7567-4e73-9cca-dd91c90c0406 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:40:59.540237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33d5fbb8-7567-4e73-9cca-dd91c90c0406 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:40:59.543078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:59.543942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:40:59.606419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33d5fbb8-7567-4e73-9cca-dd91c90c0406 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:40:59.613663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33d5fbb8-7567-4e73-9cca-dd91c90c0406 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1215 microversion: 2.1 time: 0.078917 Oct 15 21:40:59.614243 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4039/5836] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:40:59 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1215 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:41:00.638996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:00.640610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:00.648003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:00.648600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:00.649342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:00.650304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:00.651008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:00.651699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:00.739701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:00.748036 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-17622131-b278-4ad6-87f4-44dc5f8ec2d1 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1285 microversion: 2.1 time: 0.114455 Oct 15 21:41:00.749301 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1798/5837] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:00 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1285 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:01.777708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:01.779263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:01.789526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:01.790334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:01.791008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:01.791931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:01.792662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:01.793325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:02.020351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:02.029485 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f22122b3-7379-46e0-a269-957073b3624a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1315 microversion: 2.1 time: 0.260292 Oct 15 21:41:02.030155 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4040/5838] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:01 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1315 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:41:02.129099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0fdfd145-88cd-4176-b2b4-d35ca49eb9a7 req-2fd1f72a-c5bb-4534-8368-2778ebaa74b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ea8a94-a2a8-463b-90d5-952e534f4979", "tag": "47f01327-e9a1-4ade-9be3-3f4a1e24b982"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:02.144268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0fdfd145-88cd-4176-b2b4-d35ca49eb9a7 req-2fd1f72a-c5bb-4534-8368-2778ebaa74b4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:02.145009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0fdfd145-88cd-4176-b2b4-d35ca49eb9a7 req-2fd1f72a-c5bb-4534-8368-2778ebaa74b4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:02.145665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0fdfd145-88cd-4176-b2b4-d35ca49eb9a7 req-2fd1f72a-c5bb-4534-8368-2778ebaa74b4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:02.163708 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0fdfd145-88cd-4176-b2b4-d35ca49eb9a7 req-2fd1f72a-c5bb-4534-8368-2778ebaa74b4 service nova] Creating event network-changed:47f01327-e9a1-4ade-9be3-3f4a1e24b982 for instance b6ea8a94-a2a8-463b-90d5-952e534f4979 on nch2-master-1 Oct 15 21:41:02.172289 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0fdfd145-88cd-4176-b2b4-d35ca49eb9a7 req-2fd1f72a-c5bb-4534-8368-2778ebaa74b4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046894 Oct 15 21:41:02.173257 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1799/5839] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:41:03.053754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:03.054376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:03.061352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:03.061853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:03.063028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:03.063743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:03.064213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:03.064651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:03.212150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:03.220491 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0227db3e-8554-424b-8a93-89cc19e73b3c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1315 microversion: 2.1 time: 0.170739 Oct 15 21:41:03.221096 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4041/5840] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:03 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:04.244874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:04.245620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:04.251932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:04.252428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:04.252869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:04.253603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:04.254073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:04.254511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:04.384618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:04.393073 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e9023354-33d6-40b1-9ef0-120c2d67943c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1315 microversion: 2.1 time: 0.153267 Oct 15 21:41:04.393644 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1800/5841] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:04 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:05.416487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:05.417148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:05.425097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:05.425603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:05.426048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:05.426850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:05.427319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:05.427755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:05.561595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:05.569982 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d7abf0ec-ca14-497f-8d5d-dc7f15e48077 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1315 microversion: 2.1 time: 0.157454 Oct 15 21:41:05.570552 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4042/5842] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:05 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1315 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:06.593017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:06.593846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:06.600699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:06.601246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:06.601705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:06.602409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:06.602875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:06.603316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:06.735873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:06.743498 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8f078daf-45e1-4cb1-b24e-02f9652c588e tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1315 microversion: 2.1 time: 0.154438 Oct 15 21:41:06.744054 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1801/5843] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:06 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:07.155379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5d5c59a4-c4f9-497f-baf8-2a2c808eb73a req-0c7576b6-b59b-4879-85bc-f8a3e531482b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6ea8a94-a2a8-463b-90d5-952e534f4979", "name": "network-vif-plugged", "status": "completed", "tag": "47f01327-e9a1-4ade-9be3-3f4a1e24b982"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:07.166790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5d5c59a4-c4f9-497f-baf8-2a2c808eb73a req-0c7576b6-b59b-4879-85bc-f8a3e531482b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:07.167274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5d5c59a4-c4f9-497f-baf8-2a2c808eb73a req-0c7576b6-b59b-4879-85bc-f8a3e531482b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:07.167705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5d5c59a4-c4f9-497f-baf8-2a2c808eb73a req-0c7576b6-b59b-4879-85bc-f8a3e531482b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:07.180196 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5d5c59a4-c4f9-497f-baf8-2a2c808eb73a req-0c7576b6-b59b-4879-85bc-f8a3e531482b service nova] Creating event network-vif-plugged:47f01327-e9a1-4ade-9be3-3f4a1e24b982 for instance b6ea8a94-a2a8-463b-90d5-952e534f4979 on nch2-master-1 Oct 15 21:41:07.185229 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5d5c59a4-c4f9-497f-baf8-2a2c808eb73a req-0c7576b6-b59b-4879-85bc-f8a3e531482b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033586 Oct 15 21:41:07.186104 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4043/5844] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:41:07.767461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:07.768001 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:07.774840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:07.775342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:07.775798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:07.776506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:07.776982 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:07.777452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:07.908282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:07.915955 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-860ea18b-75cf-476d-9fee-8d9eee80b4b6 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1315 microversion: 2.1 time: 0.153181 Oct 15 21:41:07.916517 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1802/5845] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:07 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:08.940318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:08.941020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:08.944086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=4,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:08.949962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:08.950478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:08.950919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:08.951618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:08.952079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:08.952507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:08.955666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:09.394726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:09.402490 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f7c0aeaa-08db-46ce-bd31-f41abdb711ef tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1501 microversion: 2.1 time: 0.466380 Oct 15 21:41:09.403057 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4044/5846] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:08 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1501 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:09.424747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:09.427144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:09.433946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:09.434485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:09.434939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:09.435673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:09.436141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:09.436579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:09.442073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=5 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:09.466241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:09.469053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=10 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:09.471778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Old instance type m1.heat2, new instance type m1.heat {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:41:09.478653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:41:09.479087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:41:09.479572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:41:09.479991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:41:09.480412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:41:09.543919 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-198fe544-47b9-42a5-9896-2e586ec57983 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "POST /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979/action" status: 202 len: 0 microversion: 2.1 time: 0.123285 Oct 15 21:41:09.544530 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1803/5847] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:41:09 2022] POST /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979/action => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:41:09.552663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:09.553435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:09.559505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:09.559991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:09.560428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:09.561147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:09.561609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:09.562538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:09.610661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:09.697408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:09.705862 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4b27e2c-4cc1-4b06-ad34-413b2e7f7f1a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1510 microversion: 2.1 time: 0.157083 Oct 15 21:41:09.706438 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4045/5848] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:09 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1510 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:10.730430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:10.731057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:10.732963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=4,nova_api:DELETE=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:10.740112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:10.740770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:10.741263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:10.741970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:10.742469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:10.742908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:10.747457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:10.875836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:10.883890 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d7eb9e50-1158-45ba-ac45-1b523c4cc44b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1510 microversion: 2.1 time: 0.158193 Oct 15 21:41:10.884464 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1804/5849] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:10 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1510 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:11.908513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:11.909176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:11.916051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:11.916539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:11.916979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:11.917697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:11.918160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:11.918588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:12.049441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:12.057352 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8786667-a59b-4d4d-a658-ef226ee5cff0 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.153499 Oct 15 21:41:12.057928 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4046/5850] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:11 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:13.081450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:13.082197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:13.088883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:13.089418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:13.089869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:13.090615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:13.091157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:13.091672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:13.231684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:13.239734 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-66b26746-978a-4339-a96e-19c6c47fbc5a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.163126 Oct 15 21:41:13.240302 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1805/5851] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:13 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:14.263892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:14.264507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:14.271932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:14.272415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:14.272851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:14.273587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:14.274045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:14.274471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:14.403224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:14.412171 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33b5b1ca-d097-40fb-9818-05b487f8821c tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.152383 Oct 15 21:41:14.412747 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4047/5852] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:14 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:15.434688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:15.435302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:15.442200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:15.442693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:15.443139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:15.443848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:15.444318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:15.444863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:15.579366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:15.587103 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6af01f76-adfb-4891-ae9d-f7238fb7416f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.156234 Oct 15 21:41:15.587665 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1806/5853] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:15 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:16.611625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:16.612180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:16.619715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:16.620200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:16.620633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:16.621357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:16.621816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:16.622244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:16.750661 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:16.758340 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ae31dcc9-a537-488c-b986-fb22f0b21798 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.151553 Oct 15 21:41:16.758908 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4048/5854] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:16 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:17.782672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:17.783301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:17.790926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:17.791420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:17.791867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:17.792610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:17.793099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:17.793540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:17.922954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:17.930780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23e96667-2153-4a6b-b065-d7b56c1275c5 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.152723 Oct 15 21:41:17.931347 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1807/5855] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:17 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:18.954428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:18.955041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:18.961539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:18.962029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:18.962467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:18.963158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:18.963615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:18.964041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:19.098725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:19.106655 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fc560668-9653-4373-aa36-0915d7587be9 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.156344 Oct 15 21:41:19.107235 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4049/5856] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:18 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:20.129652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:20.130278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:20.137231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:20.137981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:20.138426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:20.139126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:20.139593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:20.140036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:20.268936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:20.276847 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aed393b7-2271-4e15-b29c-bf30ad8570e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.151036 Oct 15 21:41:20.281198 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1808/5857] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:20 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:41:21.300759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:21.301414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:21.308497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:21.308987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:21.309453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:21.310151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:21.310611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:21.311041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:21.446144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:21.454656 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9abb47c8-42ae-40d6-9902-0c5c6a7b9cbc tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.158795 Oct 15 21:41:21.455224 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4050/5858] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:21 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:22.477267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:22.477844 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:22.484770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:22.485311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:22.486040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:22.486771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:22.487254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:22.487703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:22.616472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:22.624371 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3d442bec-9dda-468d-a6fe-59ffc3330c85 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.151097 Oct 15 21:41:22.624943 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1809/5859] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:22 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:23.649114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:23.649768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:23.656572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:23.657089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:23.657537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:23.658519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:23.658979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:23.659411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:23.788643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:23.796503 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0cb5c40-2fb8-4bec-bc28-68cf64609678 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.151402 Oct 15 21:41:23.797089 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4051/5860] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:23 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:24.819788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:24.820374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:24.827267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:24.827758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:24.828203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:24.828929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:24.829922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:24.830362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:24.964948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:24.973510 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4f28569c-05bb-4ae9-8a4b-3fd52da99204 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.157657 Oct 15 21:41:24.974071 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1810/5861] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:24 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:25.996853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:25.997538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:26.004881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:26.005404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:26.005853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:26.006562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:26.007030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:26.007467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:26.137187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:26.145995 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9ab72ab0-1a23-4349-8939-cb1d0b60a5e4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.153086 Oct 15 21:41:26.146578 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4052/5862] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:25 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:27.169089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:27.169733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:27.176616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:27.177170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:27.177628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:27.178344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:27.178822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:27.179265 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:27.316118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:27.323975 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a74322fb-176f-4094-82cf-d239d23f3962 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.158809 Oct 15 21:41:27.324553 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1811/5863] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:27 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:28.347881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:28.348492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:28.355414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:28.355898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:28.356339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:28.357054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:28.357519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:28.357948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:28.490737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:28.498613 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80408b5e-c308-43b3-b99c-9256b39a9605 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.154688 Oct 15 21:41:28.499179 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4053/5864] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:28 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:29.524705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:29.525389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:29.532115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:29.533229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:29.533680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:29.534387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:29.534855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:29.535290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:29.667114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:29.675356 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-decb165a-9219-49d6-a56c-6c25f9a07dde tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.154558 Oct 15 21:41:29.675944 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1812/5865] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:29 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:30.699489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:30.700137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:30.707600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:30.708084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:30.708525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:30.709244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:30.709703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:30.710130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:30.839533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:30.848061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af6fabb4-eec4-4821-a529-e1666ed25969 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.152503 Oct 15 21:41:30.848628 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4054/5866] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:30 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:31.871988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:31.872681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:31.879700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:31.880207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:31.880667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:31.881408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:31.881890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:31.882336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:32.019519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:32.027255 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1f6fbfc9-467c-4457-9a26-3e14aae97784 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.159717 Oct 15 21:41:32.027819 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1813/5867] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:31 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:33.050598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:33.051221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:33.058437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:33.058931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:33.059371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:33.060076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:33.060563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:33.060996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:33.199477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:33.207891 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27ab7537-9e0d-4d19-930c-dd289e83c9ba tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.161556 Oct 15 21:41:33.208457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4055/5868] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:33 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:34.234819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:34.235447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:34.242986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:34.243513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:34.243970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:34.244680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:34.245189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:34.245635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:34.375225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:34.383080 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23647dfa-23ab-446c-9288-401466c285bf tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.152112 Oct 15 21:41:34.383656 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1814/5869] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:34 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:35.406505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:35.407122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:35.414605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:35.415085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:35.415523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:35.416211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:35.416665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:35.417119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:35.552007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:35.559786 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-684c6fbd-3859-43a7-9499-d98ec0697e81 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.157240 Oct 15 21:41:35.560377 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4056/5870] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:35 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:36.583489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:36.584029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:36.590904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:36.591402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:36.591848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:36.592548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:36.593013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:36.593491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:36.723200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:36.731043 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e3c314f3-32ce-434a-9522-9493f26cd632 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.151474 Oct 15 21:41:36.731620 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1815/5871] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:36 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:37.754855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:37.755484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:37.762462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:37.763214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:37.763651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:37.764344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:37.764804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:37.765264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:37.895067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:37.902874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-12dc0494-4732-4014-817c-8f55dd9754ea tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1515 microversion: 2.1 time: 0.152574 Oct 15 21:41:37.903452 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4057/5872] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:37 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:38.833871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6ea8a94-a2a8-463b-90d5-952e534f4979", "name": "network-vif-unplugged", "status": "completed", "tag": "47f01327-e9a1-4ade-9be3-3f4a1e24b982"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:38.844398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:38.845715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:38.846322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:38.858812 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] Creating event network-vif-unplugged:47f01327-e9a1-4ade-9be3-3f4a1e24b982 for instance b6ea8a94-a2a8-463b-90d5-952e534f4979 on nch1-master-1 Oct 15 21:41:38.865060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:41:38.876600 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-63fa025b-2e26-4498-8d27-17712b8e902b req-2f6718e8-4ac9-44fc-9b89-c749f0d69042 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046573 Oct 15 21:41:38.877575 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1816/5873] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:38 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:41:38.926698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:38.927579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:38.935123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:38.935837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:38.936362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:38.937302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:38.938004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:38.938504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:39.126049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:39.128425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=88 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:39.135526 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fbb3fbcf-9856-4383-b2a4-6567648d7113 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1514 microversion: 2.1 time: 0.212991 Oct 15 21:41:39.136127 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4058/5874] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:38 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1514 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:40.161506 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:40.162231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:40.168434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:40.168937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:40.169420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:40.170158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:40.170633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:40.171076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:40.312587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:40.320406 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-318ac88f-6abb-4c9a-87b2-6af7e5df8a0b tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1512 microversion: 2.1 time: 0.164742 Oct 15 21:41:40.320969 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1817/5875] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:40 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1512 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:40.893540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6ea8a94-a2a8-463b-90d5-952e534f4979", "tag": "47f01327-e9a1-4ade-9be3-3f4a1e24b982"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:40.899207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=30 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:40.905057 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:40.905550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:40.905987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:40.918254 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] Creating event network-changed:47f01327-e9a1-4ade-9be3-3f4a1e24b982 for instance b6ea8a94-a2a8-463b-90d5-952e534f4979 on nch1-master-1 Oct 15 21:41:40.924248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:41:40.935468 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-59e5ca68-e80c-4417-a97a-e06b2234bff9 req-41d22034-5d80-4972-80ae-c52fd8ce67ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046167 Oct 15 21:41:40.936108 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4059/5876] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:41:41.343932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:41.344569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:41.347987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:41.353822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:41.354334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:41.354793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:41.355516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:41.355996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:41.356745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:41.360959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=82 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:41.644772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:41.652678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-84ee09ef-1bc1-4371-b97c-afeaedb3ee34 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1509 microversion: 2.1 time: 0.312937 Oct 15 21:41:41.653293 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1818/5877] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:41 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1509 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:41.677937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:41.678636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:41.684820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:41.685336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:41.685779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:41.686475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:41.686935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:41.687365 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:41.713893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:41:41.764892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.compute.api [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Found an unconfirmed migration during delete, id: 15, status: finished Oct 15 21:41:43.476415 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-77c70681-9357-4103-b5e4-553d9b5b93ad tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 204 len: 0 microversion: 2.1 time: 1.803288 Oct 15 21:41:43.477126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4060/5878] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:41:41 2022] DELETE /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 0 bytes in 1805 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:41:43.486193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:43.486888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:43.494095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:43.494598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:43.495051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:43.495764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:43.496241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:43.496685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:43.624566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:43.632916 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70e0cbb9-78bd-4464-8d52-47dee717631f tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1501 microversion: 2.1 time: 0.150690 Oct 15 21:41:43.633519 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1819/5879] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:43 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1501 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:44.403212 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6074d529-98ca-4d42-a56a-80e5a1d2a151 req-5675e48d-4af9-4132-af82-9fd73f8f1b08 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6ea8a94-a2a8-463b-90d5-952e534f4979", "name": "network-vif-plugged", "status": "completed", "tag": "47f01327-e9a1-4ade-9be3-3f4a1e24b982"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:44.418371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6074d529-98ca-4d42-a56a-80e5a1d2a151 req-5675e48d-4af9-4132-af82-9fd73f8f1b08 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:44.418855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6074d529-98ca-4d42-a56a-80e5a1d2a151 req-5675e48d-4af9-4132-af82-9fd73f8f1b08 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:44.419290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6074d529-98ca-4d42-a56a-80e5a1d2a151 req-5675e48d-4af9-4132-af82-9fd73f8f1b08 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:44.430317 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6074d529-98ca-4d42-a56a-80e5a1d2a151 req-5675e48d-4af9-4132-af82-9fd73f8f1b08 service nova] Creating event network-vif-plugged:47f01327-e9a1-4ade-9be3-3f4a1e24b982 for instance b6ea8a94-a2a8-463b-90d5-952e534f4979 on nch1-master-1 Oct 15 21:41:44.435893 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6074d529-98ca-4d42-a56a-80e5a1d2a151 req-5675e48d-4af9-4132-af82-9fd73f8f1b08 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036747 Oct 15 21:41:44.436520 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4061/5880] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:41:44.656706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:44.658268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:44.667100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:44.667671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:44.668145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:44.669061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:44.669547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:44.670000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:44.838923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1bffd7ffed94cf595a69eaa7fa758ed', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '512ac462334c424eb45066ea7647a3a9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:44.847885 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3d6629e5-80ba-4fc8-8fcc-46338a97b06a tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 200 len: 1501 microversion: 2.1 time: 0.195245 Oct 15 21:41:44.848471 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1820/5881] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:44 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 1501 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:45.254420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6ca7d803-bfdf-4b61-975f-7c47a1927ee4 req-0d2d5924-aa48-4687-8fc7-4db626a986a3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6ea8a94-a2a8-463b-90d5-952e534f4979", "name": "network-vif-deleted", "tag": "47f01327-e9a1-4ade-9be3-3f4a1e24b982"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:45.271375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ca7d803-bfdf-4b61-975f-7c47a1927ee4 req-0d2d5924-aa48-4687-8fc7-4db626a986a3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.272484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ca7d803-bfdf-4b61-975f-7c47a1927ee4 req-0d2d5924-aa48-4687-8fc7-4db626a986a3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.274400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ca7d803-bfdf-4b61-975f-7c47a1927ee4 req-0d2d5924-aa48-4687-8fc7-4db626a986a3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.290290 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6ca7d803-bfdf-4b61-975f-7c47a1927ee4 req-0d2d5924-aa48-4687-8fc7-4db626a986a3 service nova] Creating event network-vif-deleted:47f01327-e9a1-4ade-9be3-3f4a1e24b982 for instance b6ea8a94-a2a8-463b-90d5-952e534f4979 on nch1-master-1 Oct 15 21:41:45.296426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6ca7d803-bfdf-4b61-975f-7c47a1927ee4 req-0d2d5924-aa48-4687-8fc7-4db626a986a3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046376 Oct 15 21:41:45.297685 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4062/5882] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:41:45.871712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:45.872342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:45.879911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.880416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.880868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.881638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.882115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.882558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.896091 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] HTTP exception thrown: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Request method failure captured: Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 HTTP/1.1 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyjq__C5fxbfZkt8pa5sgfkEZvo3p2acmNPE-cMadzI8kld2gnyzIzUiZfWIou1B5QJykiaSDr2G7-2Cufh5-rOjzWYMEM6ad0tcgRBhEX02TA38ay6NIyD7q93xMRRD7Fi10AVrUz0e5tLkktlOFAAW3_vmFUXo8RUkILFJIdL40Ko Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 512ac462334c424eb45066ea7647a3a9 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_512ac462334c424eb45066ea7647a3a9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 512ac462334c424eb45066ea7647a3a9 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-DeleteServersTestJSON-191375456-project Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c1bffd7ffed94cf595a69eaa7fa758ed Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-DeleteServersTestJSON-191375456-project Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'b6ea8a94-a2a8-463b-90d5-952e534f4979'} Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:41:45.897778 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Returning 404 to user: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c65d91d9-e467-4251-9dda-8e95f26a5ac3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 404 len: 111 microversion: 2.1 time: 0.031401 Oct 15 21:41:45.903199 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1821/5883] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:45 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:41:45.915312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:45.915938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:45.922209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.922700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.923150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.923852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.925028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.925561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.938895 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] HTTP exception thrown: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Request method failure captured: Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 HTTP/1.1 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyjq__C5fxbfZkt8pa5sgfkEZvo3p2acmNPE-cMadzI8kld2gnyzIzUiZfWIou1B5QJykiaSDr2G7-2Cufh5-rOjzWYMEM6ad0tcgRBhEX02TA38ay6NIyD7q93xMRRD7Fi10AVrUz0e5tLkktlOFAAW3_vmFUXo8RUkILFJIdL40Ko Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 512ac462334c424eb45066ea7647a3a9 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_512ac462334c424eb45066ea7647a3a9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 512ac462334c424eb45066ea7647a3a9 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-DeleteServersTestJSON-191375456-project Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: c1bffd7ffed94cf595a69eaa7fa758ed Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-DeleteServersTestJSON-191375456-project Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b6ea8a94-a2a8-463b-90d5-952e534f4979'} Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:41:45.941512 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Returning 404 to user: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83608450-6330-455d-b9f2-aefdef239ab3 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 404 len: 111 microversion: 2.1 time: 0.032881 Oct 15 21:41:45.946098 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4063/5884] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:41:45 2022] DELETE /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:41:45.951731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:45.952363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] [instance: b6ea8a94-a2a8-463b-90d5-952e534f4979] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:45.959114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.959644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.960150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.960912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:45.961430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:45.961882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:45.975809 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] HTTP exception thrown: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Request method failure captured: Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 HTTP/1.1 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyjq__C5fxbfZkt8pa5sgfkEZvo3p2acmNPE-cMadzI8kld2gnyzIzUiZfWIou1B5QJykiaSDr2G7-2Cufh5-rOjzWYMEM6ad0tcgRBhEX02TA38ay6NIyD7q93xMRRD7Fi10AVrUz0e5tLkktlOFAAW3_vmFUXo8RUkILFJIdL40Ko Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 512ac462334c424eb45066ea7647a3a9 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_512ac462334c424eb45066ea7647a3a9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/512ac462334c424eb45066ea7647a3a9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 512ac462334c424eb45066ea7647a3a9 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-DeleteServersTestJSON-191375456 Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-DeleteServersTestJSON-191375456-project Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: c1bffd7ffed94cf595a69eaa7fa758ed Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-DeleteServersTestJSON-191375456-project Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'b6ea8a94-a2a8-463b-90d5-952e534f4979'} Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:41:45.977358 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] Returning 404 to user: Instance b6ea8a94-a2a8-463b-90d5-952e534f4979 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8ba90e2a-94ae-4b61-a3b7-54813b70cbd4 tempest-DeleteServersTestJSON-191375456 tempest-DeleteServersTestJSON-191375456-project] 10.222.0.16 "GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979" status: 404 len: 111 microversion: 2.1 time: 0.031006 Oct 15 21:41:45.982070 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1822/5885] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:45 2022] GET /compute/v2.1/servers/b6ea8a94-a2a8-463b-90d5-952e534f4979 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:41:54.345259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1758658762", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "06922610-48ea-411d-a78d-91c276880f81"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:54.422449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] validate_networks() for [('06922610-48ea-411d-a78d-91c276880f81', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:41:54.847651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:41:54.848087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:41:54.848601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:41:54.849022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:41:54.849685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:41:54.862851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:54.863642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:54.864163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:54.864923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:54.865520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:54.866118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:54.890543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Getting quotas for project 9059ee7869df4e5abed98104bae1ef7d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:41:54.897249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Getting quotas for user 6d864b55e1d94113bed6ec710c0c0fc6 and project 9059ee7869df4e5abed98104bae1ef7d. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:41:54.912587 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:41:54.915279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:41:54.975753 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4bb8f833-fb51-4038-bcfe-e2dc5b1cebce tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.765541 Oct 15 21:41:54.977113 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4064/5886] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:41:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 768 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:41:54.989356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fdf9b79b-280c-472e-a89f-ecaa44df4fa9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:54.990456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fdf9b79b-280c-472e-a89f-ecaa44df4fa9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:55.066863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fdf9b79b-280c-472e-a89f-ecaa44df4fa9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:55.075821 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fdf9b79b-280c-472e-a89f-ecaa44df4fa9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1218 microversion: 2.1 time: 0.091688 Oct 15 21:41:55.077655 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1823/5887] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:54 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1218 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:56.108062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:56.108893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:56.117478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:56.117982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:56.118595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:56.119455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:56.120200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:56.120625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:56.195528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:56.206031 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af1395a2-2adf-49ea-85cb-6072dcd6bd76 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1288 microversion: 2.1 time: 0.104043 Oct 15 21:41:56.206614 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4065/5888] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:56 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1288 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:57.229376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:57.230441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:57.232443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=21 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:41:57.241667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:57.242274 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:57.242959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:57.244155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:57.244726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:57.245282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:57.726589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:57.735266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e51985c5-55fa-4278-9f19-f78f7f0bdb16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1318 microversion: 2.1 time: 0.509811 Oct 15 21:41:57.749500 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1824/5889] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:57 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1318 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:57.870663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-566363c8-2741-4cc7-b135-ac441b2c9c8b req-d0e02354-d27c-4b3d-8be9-b20e6607d9cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "197d303b-699c-48e7-aff9-d45927350e5a", "tag": "165cb7ce-5072-49f0-b7eb-5efecd370cf5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:41:57.888231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-566363c8-2741-4cc7-b135-ac441b2c9c8b req-d0e02354-d27c-4b3d-8be9-b20e6607d9cb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:57.888647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-566363c8-2741-4cc7-b135-ac441b2c9c8b req-d0e02354-d27c-4b3d-8be9-b20e6607d9cb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:57.889156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-566363c8-2741-4cc7-b135-ac441b2c9c8b req-d0e02354-d27c-4b3d-8be9-b20e6607d9cb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:57.901121 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-566363c8-2741-4cc7-b135-ac441b2c9c8b req-d0e02354-d27c-4b3d-8be9-b20e6607d9cb service nova] Creating event network-changed:165cb7ce-5072-49f0-b7eb-5efecd370cf5 for instance 197d303b-699c-48e7-aff9-d45927350e5a on nch1-master-1 Oct 15 21:41:57.909331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-566363c8-2741-4cc7-b135-ac441b2c9c8b req-d0e02354-d27c-4b3d-8be9-b20e6607d9cb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047550 Oct 15 21:41:57.910799 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4066/5890] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:41:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:41:58.759684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:58.760309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:58.767464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:58.768045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:58.768495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:58.769228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:58.769699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:58.770134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:58.909500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:41:58.917387 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-637e739c-7973-4c22-b02e-f36c40390eff tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1318 microversion: 2.1 time: 0.161602 Oct 15 21:41:58.917895 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1825/5891] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:58 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1318 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:41:59.942122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:41:59.942833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:41:59.949029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:59.949540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:59.949979 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:41:59.950701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:41:59.951173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:41:59.951583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:00.085030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:00.092853 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bfe8efd6-0cf2-41dd-88b9-6db15a088ce1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1318 microversion: 2.1 time: 0.156009 Oct 15 21:42:00.093461 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4067/5892] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:41:59 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:01.115736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:01.116438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:01.123273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:01.123773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:01.124230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:01.124945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:01.125444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:01.125884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:01.502994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:01.510733 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-40084812-addf-45b5-92e4-702e4c360903 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1318 microversion: 2.1 time: 0.399500 Oct 15 21:42:01.511301 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1826/5893] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:01 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1318 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:02.523951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-1d341ad7-de46-4df9-97fc-6992333c7609 req-7095391c-f470-4a3e-b87b-1a0e09d02f3c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "197d303b-699c-48e7-aff9-d45927350e5a", "name": "network-vif-plugged", "status": "completed", "tag": "165cb7ce-5072-49f0-b7eb-5efecd370cf5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:02.535547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:02.536072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1d341ad7-de46-4df9-97fc-6992333c7609 req-7095391c-f470-4a3e-b87b-1a0e09d02f3c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:02.536254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:02.537086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1d341ad7-de46-4df9-97fc-6992333c7609 req-7095391c-f470-4a3e-b87b-1a0e09d02f3c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:02.537537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-1d341ad7-de46-4df9-97fc-6992333c7609 req-7095391c-f470-4a3e-b87b-1a0e09d02f3c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:02.543917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:02.544640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:02.545154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:02.545886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:02.546363 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:02.546815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:02.550042 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-1d341ad7-de46-4df9-97fc-6992333c7609 req-7095391c-f470-4a3e-b87b-1a0e09d02f3c service nova] Creating event network-vif-plugged:165cb7ce-5072-49f0-b7eb-5efecd370cf5 for instance 197d303b-699c-48e7-aff9-d45927350e5a on nch1-master-1 Oct 15 21:42:02.555568 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-1d341ad7-de46-4df9-97fc-6992333c7609 req-7095391c-f470-4a3e-b87b-1a0e09d02f3c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035400 Oct 15 21:42:02.556176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4068/5894] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:02.723645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:02.731425 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-14210880-a5b7-420a-94e3-1e2614590479 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1318 microversion: 2.1 time: 0.200913 Oct 15 21:42:02.731989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1827/5895] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:02 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1318 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:03.756273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:03.757013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:03.764411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:03.764901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:03.765374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:03.766073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:03.766533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:03.766962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:03.903927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:03.912700 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-02b48ce7-3fa5-42d9-b1bd-a483efdc2019 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1507 microversion: 2.1 time: 0.161842 Oct 15 21:42:03.913307 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4069/5896] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:03 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1507 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:03.935954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:03.936582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:03.943611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:03.944111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:03.944565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:03.945298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:03.945773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:03.946210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.077055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:04.084860 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0dbc033e-0b77-4240-9b37-b31a0941392f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1507 microversion: 2.1 time: 0.152808 Oct 15 21:42:04.085614 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1828/5897] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:03 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:04.107800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:04.110231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:04.117374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.117863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.118307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.119008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.119471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.119911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.187043 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-516be5b1-3714-452f-9602-6a8d3c6e281b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "PUT /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1079 microversion: 2.1 time: 0.083191 Oct 15 21:42:04.187695 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4070/5898] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Oct 15 21:42:04 2022] PUT /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1079 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:04.207679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:04.208320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:04.214364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.214869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.215328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.216043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.216525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.216972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.350008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:04.358020 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-29cc5bf8-fa9a-4529-b2ac-63295df91067 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1505 microversion: 2.1 time: 0.154877 Oct 15 21:42:04.358612 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1829/5899] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:04 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1505 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:04.380354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:04.380975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:04.387174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.387749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.388446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.389167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.389629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.390065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.521883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:04.529687 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b83ed1d-59af-4c0a-848f-19ab7fa3b15b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1505 microversion: 2.1 time: 0.153311 Oct 15 21:42:04.530255 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4071/5900] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:04 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1505 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:04.552818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451", "OS-DCF:diskConfig": "MANUAL"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:04.555070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:04.561057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.561610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.562071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.562810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.563285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.563728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.593662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:04.600985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Old instance type m1.heat2, new instance type m1.heat {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:42:04.608061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:42:04.608502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:42:04.609012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:42:04.609478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:42:04.609916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:42:04.678846 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81651ba1-9d58-4103-b572-3484e5cfc9cf tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a/action" status: 202 len: 0 microversion: 2.1 time: 0.130564 Oct 15 21:42:04.679636 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1830/5901] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:42:04 2022] POST /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:42:04.688505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:04.689827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:04.696106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.696595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.697045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.697772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:04.698227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:04.698655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:04.833446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:04.841477 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ccf184af-d6bd-4290-91d0-ca2f4945838e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1516 microversion: 2.1 time: 0.157264 Oct 15 21:42:04.842209 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4072/5902] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:04 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1516 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:05.864665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:05.865437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:05.867432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:05.874180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:05.874693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:05.875155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:05.875914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:05.876394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:05.876843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:06.092979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:06.101660 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-89f4fd45-546a-42be-9587-c66cfc2b4d73 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1516 microversion: 2.1 time: 0.241071 Oct 15 21:42:06.102235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1831/5903] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:05 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1516 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:07.125859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:07.127089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:07.129130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:07.130015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:07.136535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:07.137024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:07.137490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:07.138473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:07.138936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:07.139369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:07.270334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:07.278139 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2efe1bfa-9b1a-4436-9a48-0ca8a032a74b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.156184 Oct 15 21:42:07.278717 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4073/5904] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:07 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:08.301705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:08.302346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:08.309469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:08.309983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:08.310446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:08.311196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:08.311678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:08.312128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:08.447243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:08.454965 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e5975299-bfed-430e-931f-175856517d9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.157198 Oct 15 21:42:08.455523 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1832/5905] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:08 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:09.478785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:09.479320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:09.486663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:09.487158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:09.487629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:09.488331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:09.488794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:09.489255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:09.621768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:09.629665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fbc5a233-fba8-4c76-a168-5db7e6167d5d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.154968 Oct 15 21:42:09.630235 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4074/5906] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:09 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:10.653624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:10.654258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:10.662550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:10.663164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:10.663618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:10.664353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:10.664824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:10.665288 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:10.811128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:10.819110 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-645e9c12-2ece-44bb-a9e4-d24ec9642e72 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.169449 Oct 15 21:42:10.819679 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1833/5907] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:10 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:11.842504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:11.843147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:11.846018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:11.852159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:11.852686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:11.853190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:11.853927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:11.854396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:11.854856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:11.858039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:11.989361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:11.997568 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb4284bd-e608-430e-8d86-5aa4ed9830d7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.158990 Oct 15 21:42:11.998150 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4075/5908] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:11 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:13.023979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:13.024618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:13.027480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=38 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:13.032965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:13.033504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:13.033962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:13.034691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:13.035195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:13.035645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:13.169855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:13.178159 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8588914b-3d0f-4475-859d-66d296a807e5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.158079 Oct 15 21:42:13.178930 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1834/5909] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:13 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:14.202714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:14.203333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:14.210405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:14.210889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:14.211327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:14.212018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:14.212483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:14.212913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:14.346704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:14.354506 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fba8af69-f55e-4782-b950-dc057ec58722 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.155747 Oct 15 21:42:14.355075 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4076/5910] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:14 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:15.379336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:15.379964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:15.387046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:15.387481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:15.387931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:15.388890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:15.389388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:15.389827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:15.520007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:15.527690 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-31e1c270-c292-46c0-9c3c-3247d9fa374f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.152720 Oct 15 21:42:15.528255 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1835/5911] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:15 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:16.551029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:16.551603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:16.558540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:16.559019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:16.559456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:16.560142 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:16.560597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:16.561022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:16.694737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:16.702465 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c55e9b0b-c01d-47ff-a3bf-88b4d8c03cc1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.155518 Oct 15 21:42:16.703034 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4077/5912] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:16 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:17.726889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:17.727523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:17.734570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:17.735078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:17.735577 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:17.736797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:17.737308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:17.737761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:17.869607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:17.878212 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-99097df5-7bf0-465c-b070-a9e1a5d91bbc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.155245 Oct 15 21:42:17.878789 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1836/5913] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:17 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:18.902443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:18.903093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:18.910004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:18.910513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:18.911230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:18.911644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:18.912149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:18.912605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:19.047543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:19.055488 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b8ad1ce4-a9e1-4748-87a1-9275c84b6b60 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.157598 Oct 15 21:42:19.056052 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4078/5914] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:18 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:20.080384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:20.081024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:20.088052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:20.088559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:20.089014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:20.089753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:20.090226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:20.090671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:20.221142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:20.228915 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b20c571d-675b-4db7-adc0-3fc05be3f5b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.153160 Oct 15 21:42:20.229510 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1837/5915] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:20 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:21.252439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:21.253077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:21.260219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:21.260711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:21.261187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:21.261899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:21.262361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:21.262795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:21.392122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:21.400619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ec7cc865-ecd7-4698-95f6-6c51de5be640 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.152083 Oct 15 21:42:21.401315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4079/5916] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:21 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:22.426258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:22.426902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:22.433894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:22.434402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:22.434859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:22.435574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:22.436051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:22.436498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:22.573261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:22.581223 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6e104427-12fc-4f8d-8314-ea8e90d09c40 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.159316 Oct 15 21:42:22.581798 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1838/5917] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:22 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:23.604428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:23.605100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:23.612596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:23.613122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:23.613647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:23.614638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:23.616421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:23.616860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:23.747947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:23.756511 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-897ac5bb-579d-4734-9a6b-131705145c61 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.155947 Oct 15 21:42:23.757104 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4080/5918] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:23 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:24.780423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:24.781069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:24.788175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:24.788678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:24.789172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:24.789892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:24.790371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:24.790822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:24.794826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:24.925645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:24.933481 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e553ddac-8768-4e61-9191-2aa8148f4152 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.157025 Oct 15 21:42:24.934053 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1839/5919] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:24 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:25.957015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:25.957669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:25.964293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:25.964783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:25.965273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:25.965974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:25.966474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:25.966907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:26.095786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:26.103804 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0057c3de-51a9-47b4-88de-d8d17a769bd1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.151099 Oct 15 21:42:26.104382 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4081/5920] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:25 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:27.127511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:27.128129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:27.135020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:27.135514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:27.135963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:27.136663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:27.137167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:27.137608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:27.271254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:27.279057 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1716d068-0506-4cfb-9520-ca3b37c7ef2d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.155558 Oct 15 21:42:27.279628 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1840/5921] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:27 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:28.303498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:28.304152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:28.311242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:28.311731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:28.312175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:28.312878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:28.313376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:28.313812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:28.447938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:28.456083 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a5570ab-0d55-4281-b022-5d0a0cafd579 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.157142 Oct 15 21:42:28.456613 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4082/5922] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:28 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:29.479672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:29.480209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:29.487728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:29.488231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:29.488683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:29.489432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:29.489907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:29.490350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:29.618861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:29.627250 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-47088e5d-849c-4f76-b971-40f6f1834d16 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.150954 Oct 15 21:42:29.627813 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1841/5923] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:29 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:30.650795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:30.651413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:30.659119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:30.659604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:30.660039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:30.660725 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:30.661208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:30.661634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:30.798850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:30.806998 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a50f3dec-d0a1-4808-b03f-f7f2f4fb70b0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.160132 Oct 15 21:42:30.807575 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4083/5924] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:30 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:31.831032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:31.831658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:31.838549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:31.839045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:31.839499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:31.840200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:31.840673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:31.841140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:31.973731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:31.981526 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4c004a49-9f28-451a-a7b1-32a92c2895a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.154518 Oct 15 21:42:31.982104 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1842/5925] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:31 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:33.006080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:33.006705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:33.013951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:33.014432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:33.014887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:33.015588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:33.016044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:33.016472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:33.151008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:33.159226 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-13d837a8-b335-4e77-a34c-ceff44c85f69 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1521 microversion: 2.1 time: 0.157780 Oct 15 21:42:33.159812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4084/5926] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:33 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:33.755185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "197d303b-699c-48e7-aff9-d45927350e5a", "name": "network-vif-unplugged", "status": "completed", "tag": "165cb7ce-5072-49f0-b7eb-5efecd370cf5"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:33.766046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:33.766538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:33.766989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:33.779313 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] Creating event network-vif-unplugged:165cb7ce-5072-49f0-b7eb-5efecd370cf5 for instance 197d303b-699c-48e7-aff9-d45927350e5a on nch2-master-1 Oct 15 21:42:33.785427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] Instance 197d303b-699c-48e7-aff9-d45927350e5a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:42:33.796678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-6fc32bc8-3f1e-4c2d-9df4-e548c3c73a58 req-2aa52bec-0ad6-4e3f-a4a2-ec3cdb598b47 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045313 Oct 15 21:42:33.797504 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1843/5927] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:34.183746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:34.184457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:34.191511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:34.192118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:34.192565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:34.193300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:34.193763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:34.194197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:34.410764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:34.418726 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b9a1723-5561-4be7-965d-715f263a1f22 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1520 microversion: 2.1 time: 0.238869 Oct 15 21:42:34.419316 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4085/5928] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:34 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1520 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:35.445555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:35.446188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:35.452949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:35.453620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:35.454197 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:35.455693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:35.456373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:35.456983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:35.675146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:35.682913 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c08f15e3-5611-4a8d-8bb9-9a2b376d76ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1518 microversion: 2.1 time: 0.242792 Oct 15 21:42:35.683481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1844/5929] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:35 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1518 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:35.824961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "197d303b-699c-48e7-aff9-d45927350e5a", "tag": "165cb7ce-5072-49f0-b7eb-5efecd370cf5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:35.836038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:35.836520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:35.836952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:35.848897 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] Creating event network-changed:165cb7ce-5072-49f0-b7eb-5efecd370cf5 for instance 197d303b-699c-48e7-aff9-d45927350e5a on nch2-master-1 Oct 15 21:42:35.854414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] Instance 197d303b-699c-48e7-aff9-d45927350e5a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:42:35.865446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-72bd9a95-eb44-4dda-b2f6-35bd3b64163d req-1b4d7562-e2cf-4790-a809-e67a81486dfc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044190 Oct 15 21:42:35.866045 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4086/5930] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:36.707486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:36.708123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:36.714973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:36.715477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:36.715933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:36.716651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:36.717155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:36.717604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:36.851510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:36.859308 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-82101387-4d86-414a-952a-021ac72d1d5f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1518 microversion: 2.1 time: 0.155921 Oct 15 21:42:36.859872 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1845/5931] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:36 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1518 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:37.884544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:37.885273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:37.891860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:37.892519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:37.892882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:37.893601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:37.894060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:37.894488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:38.026795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:38.034801 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e8644192-5075-4e9e-85d1-11dea0e08e99 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1515 microversion: 2.1 time: 0.155400 Oct 15 21:42:38.035374 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4087/5932] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:37 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1515 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:38.057453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:38.058093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:38.065797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:38.066304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:38.066764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:38.067488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:38.067969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:38.068454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:38.125903 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ed1fa74b-b82c-44ff-8db4-d951105c264b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a/action" status: 204 len: 0 microversion: 2.1 time: 0.072287 Oct 15 21:42:38.127137 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1846/5933] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:42:38 2022] POST /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a/action => generated 0 bytes in 74 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:42:38.135684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:38.136231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:38.143258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:38.144114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:38.144527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:38.145407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:38.145873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:38.146595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:38.655034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:38.663279 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82e9949a-bcea-4d18-a490-8260adce7dd5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1515 microversion: 2.1 time: 0.531677 Oct 15 21:42:38.663868 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4088/5934] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:38 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1515 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:39.687798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:39.688524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:39.697514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:39.698023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:39.698479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:39.699198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:39.699668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:39.700109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:39.832798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:39.840571 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-91f6b0ba-a61f-43c7-b407-a2e7e919e4f4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1507 microversion: 2.1 time: 0.157087 Oct 15 21:42:39.841160 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1847/5935] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:39 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1507 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:39.863774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:39.864407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:39.871395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:39.871886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:39.872327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:39.873068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:39.873531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:39.873967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:40.012246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:40.021090 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c19f485-5e2c-45d9-8e58-4ff27ccc714b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1507 microversion: 2.1 time: 0.161837 Oct 15 21:42:40.021672 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4089/5936] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:39 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1507 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:40.047186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c9cabada-460f-4a56-a4a9-a4f81084c043 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'create', calling method: .version_select at 0x7fe74ebff700>, body: {"keypair": {"name": "tempest-keypair-1302602644", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDgeDAj3oLTVwk+SI3faQ0QrTe9mT1igk21JsIeuk4kwUb1EXFNhxoBGIZa/iTkc8FoQ6n7WxiKLDLDKSHND8Zp16P9ysRNVrRDI/QCuttDnGuSLWFP/PAi1rVwGdS3PVQ=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:40.062739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-c9cabada-460f-4a56-a4a9-a4f81084c043 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Getting quotas for project 9059ee7869df4e5abed98104bae1ef7d. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:42:40.067917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-c9cabada-460f-4a56-a4a9-a4f81084c043 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Getting quotas for user 6d864b55e1d94113bed6ec710c0c0fc6 and project 9059ee7869df4e5abed98104bae1ef7d. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:42:40.081673 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [None req-c9cabada-460f-4a56-a4a9-a4f81084c043 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [9bd635f8-0214-4cfd-9c77-5a99b26eed51] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:42:41.089313 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [None req-c9cabada-460f-4a56-a4a9-a4f81084c043 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [9bd635f8-0214-4cfd-9c77-5a99b26eed51] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 43410. Oct 15 21:42:41.106124 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c9cabada-460f-4a56-a4a9-a4f81084c043 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.062874 Oct 15 21:42:41.106709 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1848/5937] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:42:40 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1064 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:41.547754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ad1b4d26-fae1-4e04-9f39-cd32069ff246 req-04b47547-fe0b-4c60-833a-9ed54194b4cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "197d303b-699c-48e7-aff9-d45927350e5a", "name": "network-vif-plugged", "status": "completed", "tag": "165cb7ce-5072-49f0-b7eb-5efecd370cf5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:41.574421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad1b4d26-fae1-4e04-9f39-cd32069ff246 req-04b47547-fe0b-4c60-833a-9ed54194b4cd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:41.575209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad1b4d26-fae1-4e04-9f39-cd32069ff246 req-04b47547-fe0b-4c60-833a-9ed54194b4cd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:41.575825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad1b4d26-fae1-4e04-9f39-cd32069ff246 req-04b47547-fe0b-4c60-833a-9ed54194b4cd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:41.591089 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ad1b4d26-fae1-4e04-9f39-cd32069ff246 req-04b47547-fe0b-4c60-833a-9ed54194b4cd service nova] Creating event network-vif-plugged:165cb7ce-5072-49f0-b7eb-5efecd370cf5 for instance 197d303b-699c-48e7-aff9-d45927350e5a on nch2-master-1 Oct 15 21:42:41.597300 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ad1b4d26-fae1-4e04-9f39-cd32069ff246 req-04b47547-fe0b-4c60-833a-9ed54194b4cd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059659 Oct 15 21:42:41.598236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4090/5938] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:42.343255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:42.343892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:42.350790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:42.351294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:42.351752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:42.352496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:42.352973 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:42.353453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:42.486808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:42.494657 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4075a2c-e032-486c-b628-84c0dd0c73cd tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 200 len: 1507 microversion: 2.1 time: 0.156188 Oct 15 21:42:42.495235 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1849/5939] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:42 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 1507 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:42.518499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:42.519123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:42.521143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:42.526738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:42.527226 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:42.527665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:42.528357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:42.528815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:42.529306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:42.534037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=90 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:42.555956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:42:42.631923 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac61f457-a233-4e8c-a5a8-abe5b585e498 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "DELETE /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 204 len: 0 microversion: 2.1 time: 0.117307 Oct 15 21:42:42.632464 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4091/5940] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:42:42 2022] DELETE /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:42:42.691396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-182265219", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "06922610-48ea-411d-a78d-91c276880f81"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:42.744424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] validate_networks() for [('06922610-48ea-411d-a78d-91c276880f81', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:42:43.126273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:42:43.127281 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:42:43.127779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:42:43.128200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:42:43.128628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:42:43.130779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:43.140378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:43.140875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:43.141356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:43.142102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:43.143232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:43.143671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:43.161153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Getting quotas for project 9059ee7869df4e5abed98104bae1ef7d. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:42:43.166367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Getting quotas for user 6d864b55e1d94113bed6ec710c0c0fc6 and project 9059ee7869df4e5abed98104bae1ef7d. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:42:43.178748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:42:43.181007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:42:43.227019 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52ff7caa-a115-44e6-bb47-01f37436318f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.539680 Oct 15 21:42:43.227664 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1850/5941] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:42:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 541 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:42:43.239768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab99692c-7144-400b-964f-b31e350e0eac tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:43.240468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab99692c-7144-400b-964f-b31e350e0eac tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:43.302593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab99692c-7144-400b-964f-b31e350e0eac tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:43.310093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab99692c-7144-400b-964f-b31e350e0eac tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1217 microversion: 2.1 time: 0.075074 Oct 15 21:42:43.310704 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4092/5942] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:43 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1217 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:44.334723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:44.335397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:44.342704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:44.343213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:44.343674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:44.344403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:44.344944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:44.346122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:44.462138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:44.471027 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5aa667d1-f983-4d84-89ad-cf6e4d8a7766 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1269 microversion: 2.1 time: 0.141189 Oct 15 21:42:44.471868 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1851/5943] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:44 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1269 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:44.649060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-78d6ae80-8f29-43a9-9876-88bb4bffa33e req-2d554bc4-2f8a-458d-b5e4-2aa6c6de580d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "197d303b-699c-48e7-aff9-d45927350e5a", "name": "network-vif-deleted", "tag": "165cb7ce-5072-49f0-b7eb-5efecd370cf5"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:44.663140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-78d6ae80-8f29-43a9-9876-88bb4bffa33e req-2d554bc4-2f8a-458d-b5e4-2aa6c6de580d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:44.663623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-78d6ae80-8f29-43a9-9876-88bb4bffa33e req-2d554bc4-2f8a-458d-b5e4-2aa6c6de580d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:44.664055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-78d6ae80-8f29-43a9-9876-88bb4bffa33e req-2d554bc4-2f8a-458d-b5e4-2aa6c6de580d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:44.685391 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-78d6ae80-8f29-43a9-9876-88bb4bffa33e req-2d554bc4-2f8a-458d-b5e4-2aa6c6de580d service nova] Creating event network-vif-deleted:165cb7ce-5072-49f0-b7eb-5efecd370cf5 for instance 197d303b-699c-48e7-aff9-d45927350e5a on nch2-master-1 Oct 15 21:42:44.691292 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-78d6ae80-8f29-43a9-9876-88bb4bffa33e req-2d554bc4-2f8a-458d-b5e4-2aa6c6de580d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046226 Oct 15 21:42:44.691916 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4093/5944] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:42:45.495977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:45.497649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:45.510276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:45.510790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:45.511246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:45.511969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:45.512439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:45.512949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:45.714613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:45.723547 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-be6c2c36-ce4b-44e0-a50b-d9264e31567d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.232269 Oct 15 21:42:45.724252 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1852/5945] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:45 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:46.714958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4b04643c-8b7e-481d-94f7-1066f5cbd56f req-51c3fe0a-7c8b-4a60-ae85-449ca40236f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "266b0bfc-5451-4ab1-8169-1122816eec3e", "tag": "5a6b2b84-c87d-47a8-b733-4e31513243ae"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:46.724442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4b04643c-8b7e-481d-94f7-1066f5cbd56f req-51c3fe0a-7c8b-4a60-ae85-449ca40236f3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:46.724917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4b04643c-8b7e-481d-94f7-1066f5cbd56f req-51c3fe0a-7c8b-4a60-ae85-449ca40236f3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:46.725376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4b04643c-8b7e-481d-94f7-1066f5cbd56f req-51c3fe0a-7c8b-4a60-ae85-449ca40236f3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:46.736704 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4b04643c-8b7e-481d-94f7-1066f5cbd56f req-51c3fe0a-7c8b-4a60-ae85-449ca40236f3 service nova] Creating event network-changed:5a6b2b84-c87d-47a8-b733-4e31513243ae for instance 266b0bfc-5451-4ab1-8169-1122816eec3e on nch2-master-1 Oct 15 21:42:46.741588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4b04643c-8b7e-481d-94f7-1066f5cbd56f req-51c3fe0a-7c8b-4a60-ae85-449ca40236f3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031640 Oct 15 21:42:46.742157 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4094/5946] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:46.752252 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:46.752928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:46.759939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:46.760442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:46.760896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:46.761645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:46.762123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:46.762595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:46.895571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:46.903317 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0b50a708-c3aa-4084-b1d8-e64d3f9c100d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.155573 Oct 15 21:42:46.903882 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1853/5947] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:46 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:47.928221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:47.928941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:47.936179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:47.936674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:47.937152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:47.937869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:47.938331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:47.938764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:48.069896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:48.077726 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1d47bcb-ec00-4eb6-8e82-7518a99e9847 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.154649 Oct 15 21:42:48.078382 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4095/5948] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:47 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:49.101912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:49.102490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:49.109380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:49.109880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:49.110332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:49.111663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:49.112131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:49.112573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:49.242038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:49.249806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ec5d23e-b1c0-425b-bc25-b76b655ff2b7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.151927 Oct 15 21:42:49.250370 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1854/5949] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:49 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:50.272921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:50.273566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:50.280102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:50.280586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:50.281019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:50.281732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:50.282188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:50.282618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:50.416109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:50.424394 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2af720dd-101a-40d6-833e-0569b826b03c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.155578 Oct 15 21:42:50.424968 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4096/5950] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:50 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:51.448463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:51.449254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:51.458450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:51.458887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:51.459345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:51.460078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:51.460575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:51.461019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:51.599342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:51.607161 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6458a72c-1f5e-4336-b1bd-6824585992c0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.162668 Oct 15 21:42:51.607729 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1855/5951] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:51 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:52.632154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:52.632694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:52.640071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:52.640555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:52.640996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:52.641719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:52.642182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:52.642613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:52.776070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:52.783826 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8baf615f-0178-4105-afc4-f185bf3b4ba0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.156543 Oct 15 21:42:52.784595 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4097/5952] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:52 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:53.579301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-319fcc30-bd84-4968-b75e-39b04361f68b req-f2d84049-273c-48c1-a8d6-36d49074fcae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "266b0bfc-5451-4ab1-8169-1122816eec3e", "name": "network-vif-plugged", "status": "completed", "tag": "5a6b2b84-c87d-47a8-b733-4e31513243ae"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:53.586061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:53.586767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=10 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:53.594219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-319fcc30-bd84-4968-b75e-39b04361f68b req-f2d84049-273c-48c1-a8d6-36d49074fcae service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:53.594726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-319fcc30-bd84-4968-b75e-39b04361f68b req-f2d84049-273c-48c1-a8d6-36d49074fcae service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:53.595180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-319fcc30-bd84-4968-b75e-39b04361f68b req-f2d84049-273c-48c1-a8d6-36d49074fcae service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:53.609841 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-319fcc30-bd84-4968-b75e-39b04361f68b req-f2d84049-273c-48c1-a8d6-36d49074fcae service nova] Creating event network-vif-plugged:5a6b2b84-c87d-47a8-b733-4e31513243ae for instance 266b0bfc-5451-4ab1-8169-1122816eec3e on nch2-master-1 Oct 15 21:42:53.615544 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-319fcc30-bd84-4968-b75e-39b04361f68b req-f2d84049-273c-48c1-a8d6-36d49074fcae service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040001 Oct 15 21:42:53.616166 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1856/5953] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:42:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:42:53.807164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:53.807792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:53.809810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:53.816492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:53.816985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:53.817459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:53.818163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:53.818697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:53.819140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:53.948654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:53.956932 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d9a0255-01c2-42f0-8430-e2af16bbc298 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.153692 Oct 15 21:42:53.957591 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4098/5954] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:53 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:54.981956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:54.982683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:54.989504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:54.990007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:54.990463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:54.991173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:54.991649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:54.992091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:54.996260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:42:55.133654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:55.141640 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6fb22b7d-131a-44d0-b4bd-55176b15d18e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1317 microversion: 2.1 time: 0.164894 Oct 15 21:42:55.142217 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1857/5955] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:54 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1317 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:56.165405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:56.166041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:56.172819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.173344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.173794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.174501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.175529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.175973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.306579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:56.314353 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bcea18ac-5dcc-441b-9984-01c1cc9c7586 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1507 microversion: 2.1 time: 0.153044 Oct 15 21:42:56.314928 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4099/5956] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:56 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:56.336796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:56.337454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:56.344275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.344769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.345256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.345962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.346432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.347411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.481383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:56.489196 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-948c446b-9a85-4f36-8af7-ba5595bd91b1 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1507 microversion: 2.1 time: 0.156183 Oct 15 21:42:56.489761 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1858/5957] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:56 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1507 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:42:56.512022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451", "OS-DCF:diskConfig": "AUTO"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:42:56.514637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:56.521296 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.521794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.522245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.522959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.523421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.523861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.552781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:56.558253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Old instance type m1.heat2, new instance type m1.heat {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:42:56.565058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:42:56.566233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:42:56.566733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:42:56.567333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:42:56.567863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:42:56.617773 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:42:56.618030 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:42:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e/action (ip 10.222.0.16) !!! Oct 15 21:42:56.631786 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6db92793-b98e-426f-b935-e48d774a1849 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e/action" status: 202 len: 0 microversion: 2.1 time: 0.123873 Oct 15 21:42:56.632405 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4100/5958] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:42:56 2022] POST /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e/action => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:42:56.645691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:56.646328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:56.652192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.652694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.653248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.653963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:56.654441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:56.654877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:56.787383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:56.795615 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5d122231-6f8d-4ea0-9738-f477d59fcf88 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1514 microversion: 2.1 time: 0.154306 Oct 15 21:42:56.796015 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1859/5959] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:56 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:42:57.819937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:57.821247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:57.829666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:57.830154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:57.830594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:57.831829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:57.832286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:57.832713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:58.051821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:58.059863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-23b266f1-f1d3-4ad6-a8e8-cd6c84d87fa5 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1514 microversion: 2.1 time: 0.244710 Oct 15 21:42:58.060441 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4101/5960] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:57 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1514 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:42:59.084235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:42:59.084867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:42:59.092160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:59.092664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:59.093155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:59.093870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:42:59.094344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:42:59.094790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:42:59.233160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:42:59.241729 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b4805dfb-1abc-453a-b81a-df9dd8694c9b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.161911 Oct 15 21:42:59.242295 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1860/5961] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:42:59 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:00.265287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:00.265907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:00.273457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:00.273943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:00.274398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:00.275072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:00.275528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:00.275955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:00.410918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:00.418851 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43c95f00-c766-420e-a991-59250a791b0e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.157732 Oct 15 21:43:00.419423 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4102/5962] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:00 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:01.442300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:01.442906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:01.449805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:01.450302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:01.450752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:01.451520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:01.452305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:01.453000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:01.586002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:01.594610 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-50a1152d-0670-4639-ab9a-b629a62005a7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.156695 Oct 15 21:43:01.595185 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1861/5963] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:01 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:02.618145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:02.618684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:02.625674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:02.626164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:02.626606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:02.627301 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:02.628511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:02.629171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:02.760406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:02.769073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d1d98535-7c54-4ac5-8e3f-bdac3acac8fb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.154339 Oct 15 21:43:02.769655 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4103/5964] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:02 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:03.791438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:03.792067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:03.799111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:03.799667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:03.800157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:03.800876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:03.801380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:03.801827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:03.932589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:03.941415 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ecea69f8-f794-4ac7-a841-631eb3a0a0b4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.152905 Oct 15 21:43:03.942056 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1862/5965] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:03 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:04.964894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:04.965558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:04.973362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:04.973860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:04.974309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:04.975021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:04.975490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:04.975928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:05.106134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:05.114707 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-452251a8-150a-4407-81c5-f4b08e9d0b4b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.153777 Oct 15 21:43:05.115293 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4104/5966] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:04 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:06.137924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:06.138835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:06.145181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:06.145685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:06.146136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:06.146843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:06.147317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:06.147758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:06.282114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:06.290149 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f63d3be2-2c43-4202-8ddd-cb60c16c9b84 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.156424 Oct 15 21:43:06.290717 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1863/5967] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:06 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:07.313830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:07.314453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:07.321648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:07.322137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:07.322581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:07.323278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:07.323739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:07.324170 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:07.452441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:07.462053 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e2b2ee61-c339-4c0e-8f54-2b1ee3d732c9 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.152110 Oct 15 21:43:07.462622 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4105/5968] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:07 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:08.485928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:08.486486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:08.493468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:08.493965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:08.494413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:08.495115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:08.495607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:08.496044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:08.625258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:08.633598 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3311072e-f0bd-4e30-9146-f73970d5386d tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.151684 Oct 15 21:43:08.634164 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1864/5969] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:08 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:09.658010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:09.659168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:09.666127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:09.666616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:09.667056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:09.667758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:09.668224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:09.668656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:09.800545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:09.808409 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e1476ebb-e18c-4ebd-bf95-deb4647fb4cb tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.154401 Oct 15 21:43:09.808996 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4106/5970] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:09 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:10.831760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:10.832388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:10.839340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:10.839836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:10.840283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:10.840985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:10.841474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:10.841913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:10.973195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:10.981026 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5ee37026-a968-4543-85d1-f1aeda6e72e4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.153372 Oct 15 21:43:10.981612 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1865/5971] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:10 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:12.005602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:12.006208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:12.013252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:12.014004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:12.014451 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:12.015159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:12.015627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:12.016063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:12.018738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:12.149831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:12.157741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef4dd8ad-a829-4b96-8b91-19360b1dafc2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.156827 Oct 15 21:43:12.158315 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4107/5972] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:11 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:13.180831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:13.181482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:13.184229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:13.190195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:13.190633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:13.191083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:13.191796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:13.192262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:13.192703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:13.324469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:13.332279 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2b44dbc0-a781-4450-a878-817054fa0225 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.155438 Oct 15 21:43:13.332848 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1866/5973] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:13 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:14.355995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:14.356640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:14.359476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:14.365446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:14.365938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:14.366380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:14.367077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:14.367572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:14.368002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:14.501373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:14.509587 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbba3657-2143-4c43-a884-ba4369b86fc8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.157597 Oct 15 21:43:14.510179 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4108/5974] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:14 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:15.533713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:15.534251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:15.542000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:15.542507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:15.542964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:15.543680 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:15.544157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:15.544604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:15.682322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:15.690288 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d36f0ca4-e300-42e5-868c-7d99fbaad3e8 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.160645 Oct 15 21:43:15.690873 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1867/5975] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:15 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:16.714286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:16.714926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:16.721963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:16.722512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:16.722947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:16.723638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:16.724095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:16.724522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:16.853119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:16.860810 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d299877d-aca7-4a0e-ad7e-8df33b54bc5e tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.150514 Oct 15 21:43:16.861477 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4109/5976] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:16 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:17.884286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:17.884914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:17.892059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:17.892561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:17.893016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:17.893854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:17.895036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:17.895484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:18.028809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:18.036705 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-391059e6-3114-45d2-99ac-974c17e0367c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.156346 Oct 15 21:43:18.037286 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1868/5977] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:17 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:19.060688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:19.061346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:19.068313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:19.068844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:19.069333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:19.070081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:19.070576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:19.071014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:19.201270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:19.208861 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33136cc1-bc2f-49fb-a36f-39e1b4f16819 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.152243 Oct 15 21:43:19.209475 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4110/5978] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:19 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:20.231786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:20.232414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:20.239827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:20.240326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:20.240782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:20.241525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:20.242000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:20.242441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:20.383318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:20.391635 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-01fe70b1-d082-43b2-a024-933b0664cd68 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.163703 Oct 15 21:43:20.397089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1869/5979] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:20 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:21.415270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:21.415900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:21.422922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:21.423420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:21.423872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:21.424584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:21.425073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:21.425511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:21.848521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:21.856634 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-751ee840-169f-4a4c-b84c-9233e2cb1032 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.445606 Oct 15 21:43:21.857243 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4111/5980] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:21 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:22.881348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:22.882008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:22.889200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:22.889739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:22.890200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:22.890913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:22.891394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:22.891839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:23.024010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:23.031797 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0cb35312-dca4-4b67-b5ba-2c2e3c2b19cc tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.154948 Oct 15 21:43:23.032360 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1870/5981] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:22 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:24.055236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:24.055832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:24.062779 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:24.063263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:24.063700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:24.064420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:24.064879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:24.065334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:24.199229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:24.207008 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2740ada5-9419-4983-a86e-4d1f1b757704 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.155901 Oct 15 21:43:24.208088 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4112/5982] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:24 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:25.231856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:25.232490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:25.240371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:25.240888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:25.241375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:25.242110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:25.242592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:25.243046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:25.246825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=84 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:25.435095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:25.443739 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6760a0bd-d07d-4b47-aab8-d784fe342281 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1519 microversion: 2.1 time: 0.215863 Oct 15 21:43:25.444324 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1871/5983] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:25 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1519 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:25.780398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "266b0bfc-5451-4ab1-8169-1122816eec3e", "name": "network-vif-unplugged", "status": "completed", "tag": "5a6b2b84-c87d-47a8-b733-4e31513243ae"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:25.791245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:25.791734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:25.792166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:25.804741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] Creating event network-vif-unplugged:5a6b2b84-c87d-47a8-b733-4e31513243ae for instance 266b0bfc-5451-4ab1-8169-1122816eec3e on nch1-master-1 Oct 15 21:43:25.810331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] Instance 266b0bfc-5451-4ab1-8169-1122816eec3e is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:43:25.821361 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7e69cf58-6267-49b9-8623-83a2106fdea9 req-e0750507-be89-4edf-8d15-7d0c22cdd034 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044755 Oct 15 21:43:25.821986 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4113/5984] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:43:25 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:43:26.467899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:26.468580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:26.475624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:26.476192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:26.476659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:26.477420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:26.477932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:26.478383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:26.658042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:26.667180 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d1f19c22-a7ce-4a86-be1f-0b43b4d059ef tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1518 microversion: 2.1 time: 0.203255 Oct 15 21:43:26.667791 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1872/5985] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:26 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1518 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:27.693617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:27.694325 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:27.700746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:27.701267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:27.701712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:27.702414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:27.702879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:27.703311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:27.835324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:27.843258 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8bb3ca2-40d2-4b6b-bf5f-323f23a14086 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1516 microversion: 2.1 time: 0.155942 Oct 15 21:43:27.843834 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4114/5986] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:27 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1516 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:27.844400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "266b0bfc-5451-4ab1-8169-1122816eec3e", "tag": "5a6b2b84-c87d-47a8-b733-4e31513243ae"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:27.855317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:27.855821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:27.856269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:27.868561 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] Creating event network-changed:5a6b2b84-c87d-47a8-b733-4e31513243ae for instance 266b0bfc-5451-4ab1-8169-1122816eec3e on nch1-master-1 Oct 15 21:43:27.873968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] Instance 266b0bfc-5451-4ab1-8169-1122816eec3e is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:43:27.885398 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4a2b1577-94fb-4e89-a081-29881f28d587 req-a42a1f0c-f780-43d1-8715-f7d9f468d7c5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044636 Oct 15 21:43:27.886000 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1873/5987] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:43:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:43:28.868162 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:28.868790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:28.876732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:28.877275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:28.877726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:28.878460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:28.878922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:28.879856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:29.033517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:29.042156 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1ed2914a-3c4a-4e98-9380-4e1e45ef018b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1513 microversion: 2.1 time: 0.178325 Oct 15 21:43:29.042772 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4115/5988] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:28 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1513 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:29.069069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:29.069727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:29.076605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:29.077362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:29.078146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:29.079221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:29.079897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:29.080658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:29.145594 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-57b3fc89-1cf7-4221-9e0b-009415af115c tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "POST /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e/action" status: 204 len: 0 microversion: 2.1 time: 0.081953 Oct 15 21:43:29.146261 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1874/5989] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:43:29 2022] POST /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e/action => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:43:29.155836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:29.156460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:29.163707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:29.164749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:29.165436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:29.166202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:29.167648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:29.168110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:29.385426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:29.393619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-80a83f68-f959-4111-b319-ef72a6a8b94b tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1513 microversion: 2.1 time: 0.241852 Oct 15 21:43:29.394200 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4116/5990] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:29 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1513 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:30.418258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:30.418986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:30.426808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.427374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.427854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.428581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.429127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.429595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.564712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:30.572639 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1893cca2-85cb-45db-a703-f6c75a6ea6ab tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1505 microversion: 2.1 time: 0.158823 Oct 15 21:43:30.573236 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1875/5991] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:30 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1505 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:30.595560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:30.596182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:30.603929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.604419 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.604873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.605635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.606099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.606533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.736127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:30.744173 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2483e3da-2aae-4d18-8bbd-aeacc5d1da36 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1505 microversion: 2.1 time: 0.152435 Oct 15 21:43:30.745574 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4117/5992] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:30 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1505 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:30.767437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:30.768003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:30.775496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.776008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.776468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.777232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.777717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.778217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.915910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:30.924026 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e33d8879-0ffa-42a9-8955-ba1abd90d251 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1505 microversion: 2.1 time: 0.160853 Oct 15 21:43:30.924694 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1876/5993] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:30 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1505 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:30.948168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:30.948529 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:30.955127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.955627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.956078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.956792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:30.957307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:30.957808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:30.984287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:43:31.073059 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2c09110d-7979-45b5-bf52-a670ad00ac31 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "DELETE /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 204 len: 0 microversion: 2.1 time: 0.128798 Oct 15 21:43:31.073669 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4118/5994] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:43:30 2022] DELETE /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:43:31.090854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:31.091365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:31.097402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:31.098778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:31.099266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:31.099974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:31.100445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:31.100879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:31.127789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:43:31.174201 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.compute.api [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Instance is already in deleting state, ignoring this request Oct 15 21:43:31.175053 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e4dbca5b-513b-49a3-b839-f4e0fcbb6716 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "DELETE /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 204 len: 0 microversion: 2.1 time: 0.088256 Oct 15 21:43:31.175629 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1877/5995] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:43:31 2022] DELETE /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:43:31.185236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:31.185956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:31.191989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:31.192484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:31.192931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:31.193669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:31.194138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:31.194579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:31.329217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:31.338082 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-12ffcbc9-b2ed-4ec6-8ba8-5e7e45fc50c2 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1511 microversion: 2.1 time: 0.157190 Oct 15 21:43:31.338675 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4119/5996] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:31 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1511 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:32.362031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:32.362660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:32.369595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:32.370090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:32.370542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:32.371256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:32.371724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:32.372166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:32.543942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d864b55e1d94113bed6ec710c0c0fc6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9059ee7869df4e5abed98104bae1ef7d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:32.551795 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-18de32e2-146d-4128-b3d0-b873a8b7d799 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 200 len: 1511 microversion: 2.1 time: 0.193655 Oct 15 21:43:32.552864 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1878/5997] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:32 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 1511 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:32.953401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5db9f378-2629-4d88-adb0-2ef3abb0de58 req-2aaab621-c8fa-4188-ae62-fe9bd70d55db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "266b0bfc-5451-4ab1-8169-1122816eec3e", "name": "network-vif-deleted", "tag": "5a6b2b84-c87d-47a8-b733-4e31513243ae"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:32.965704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5db9f378-2629-4d88-adb0-2ef3abb0de58 req-2aaab621-c8fa-4188-ae62-fe9bd70d55db service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:32.966101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5db9f378-2629-4d88-adb0-2ef3abb0de58 req-2aaab621-c8fa-4188-ae62-fe9bd70d55db service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:32.966539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5db9f378-2629-4d88-adb0-2ef3abb0de58 req-2aaab621-c8fa-4188-ae62-fe9bd70d55db service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:32.979061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5db9f378-2629-4d88-adb0-2ef3abb0de58 req-2aaab621-c8fa-4188-ae62-fe9bd70d55db service nova] Creating event network-vif-deleted:5a6b2b84-c87d-47a8-b733-4e31513243ae for instance 266b0bfc-5451-4ab1-8169-1122816eec3e on nch1-master-1 Oct 15 21:43:32.984127 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5db9f378-2629-4d88-adb0-2ef3abb0de58 req-2aaab621-c8fa-4188-ae62-fe9bd70d55db service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034586 Oct 15 21:43:32.984913 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4120/5998] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:43:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:43:33.576813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:33.577478 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 266b0bfc-5451-4ab1-8169-1122816eec3e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:33.584937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:33.585461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:33.585916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:33.586624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:33.587092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:33.587534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:33.601398 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] HTTP exception thrown: Instance 266b0bfc-5451-4ab1-8169-1122816eec3e could not be found. Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Request method failure captured: Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e HTTP/1.1 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSykiUwo1RXQwojgF_UM_IcRGbcV-5uzItljYz7Sb2aWt6FFMD3vrYs4rMiWvof_dXLKJWbGHz-1kVqwTmALaaNXLnSj1m80flv78yG4MO4Y9dgP-KaHE-w-V3tO1My8Wek4Es-iEatnRAa-w8QNlkTl16WebImVl-DYBA6UFff5ceVU Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 9059ee7869df4e5abed98104bae1ef7d Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 9059ee7869df4e5abed98104bae1ef7d Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerDiskConfigTestJSON-327355494-project Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 6d864b55e1d94113bed6ec710c0c0fc6 Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerDiskConfigTestJSON-327355494-project Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 266b0bfc-5451-4ab1-8169-1122816eec3e could not be found. Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '266b0bfc-5451-4ab1-8169-1122816eec3e'} Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 266b0bfc-5451-4ab1-8169-1122816eec3e could not be found. Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 266b0bfc-5451-4ab1-8169-1122816eec3e could not be found. Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:43:33.603050 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 266b0bfc-5451-4ab1-8169-1122816eec3e could not be found. Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Returning 404 to user: Instance 266b0bfc-5451-4ab1-8169-1122816eec3e could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-daa7ae88-9b8c-4c3c-8c3e-1f09207b7fe7 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e" status: 404 len: 111 microversion: 2.1 time: 0.031661 Oct 15 21:43:33.608008 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1879/5999] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:33 2022] GET /compute/v2.1/servers/266b0bfc-5451-4ab1-8169-1122816eec3e => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:43:33.613769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c95e22e-7242-432e-8cf3-856235451ae4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '.version_select at 0x7fe74e2a2280>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:33.637602 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c95e22e-7242-432e-8cf3-856235451ae4 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1302602644" status: 202 len: 0 microversion: 2.1 time: 0.027890 Oct 15 21:43:33.638174 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4121/6000] 10.222.0.16 () {58 vars in 1221 bytes} [Sat Oct 15 21:43:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1302602644 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:43:34.247660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:34.248202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:34.254312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:34.254822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:34.255887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:34.256689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:34.257198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:34.257644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:34.272030 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] HTTP exception thrown: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Request method failure captured: Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a HTTP/1.1 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSykiUwo1RXQwojgF_UM_IcRGbcV-5uzItljYz7Sb2aWt6FFMD3vrYs4rMiWvof_dXLKJWbGHz-1kVqwTmALaaNXLnSj1m80flv78yG4MO4Y9dgP-KaHE-w-V3tO1My8Wek4Es-iEatnRAa-w8QNlkTl16WebImVl-DYBA6UFff5ceVU Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: 9059ee7869df4e5abed98104bae1ef7d Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: 9059ee7869df4e5abed98104bae1ef7d Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerDiskConfigTestJSON-327355494-project Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 6d864b55e1d94113bed6ec710c0c0fc6 Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerDiskConfigTestJSON-327355494-project Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '197d303b-699c-48e7-aff9-d45927350e5a'} Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:43:34.273662 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Returning 404 to user: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a765bb1f-3e06-46fc-aff9-1ef1a3d1c5a0 tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "DELETE /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 404 len: 111 microversion: 2.1 time: 0.031436 Oct 15 21:43:34.277996 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1880/6001] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:43:34 2022] DELETE /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:43:34.284602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:34.285411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] [instance: 197d303b-699c-48e7-aff9-d45927350e5a] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:34.292288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:34.292775 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:34.293253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:34.293958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:34.294422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:34.294859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:34.310748 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] HTTP exception thrown: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Request method failure captured: Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a HTTP/1.1 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSykiUwo1RXQwojgF_UM_IcRGbcV-5uzItljYz7Sb2aWt6FFMD3vrYs4rMiWvof_dXLKJWbGHz-1kVqwTmALaaNXLnSj1m80flv78yG4MO4Y9dgP-KaHE-w-V3tO1My8Wek4Es-iEatnRAa-w8QNlkTl16WebImVl-DYBA6UFff5ceVU Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 9059ee7869df4e5abed98104bae1ef7d Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9059ee7869df4e5abed98104bae1ef7d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 9059ee7869df4e5abed98104bae1ef7d Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-327355494 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerDiskConfigTestJSON-327355494-project Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6d864b55e1d94113bed6ec710c0c0fc6 Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerDiskConfigTestJSON-327355494-project Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '197d303b-699c-48e7-aff9-d45927350e5a'} Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:43:34.312387 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] Returning 404 to user: Instance 197d303b-699c-48e7-aff9-d45927350e5a could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c60b404-06f0-4caf-a484-896da4ed136f tempest-ServerDiskConfigTestJSON-327355494 tempest-ServerDiskConfigTestJSON-327355494-project] 10.222.0.16 "GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a" status: 404 len: 111 microversion: 2.1 time: 0.034109 Oct 15 21:43:34.317103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4122/6002] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:34 2022] GET /compute/v2.1/servers/197d303b-699c-48e7-aff9-d45927350e5a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:43:42.995556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9824e24b-03e0-4743-a416-a08e36dc6246 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'create', calling method: .version_select at 0x7fe74c2b7700>, body: {"keypair": {"name": "tempest-keypair-350156625", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMPIbz15cJFDcxH4N46mMjE3OoWYeUU0ednRJBL5akQjwQmzsNB1BERutvTM39QkRF51Wz3vroUX2dkK7giey1BmDJCxahnJeqcOSj8l9R5tIKmnypkCBpbjM3N4IyzSBw=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:43.022707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9824e24b-03e0-4743-a416-a08e36dc6246 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:43:43.028369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9824e24b-03e0-4743-a416-a08e36dc6246 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:43:43.057341 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9824e24b-03e0-4743-a416-a08e36dc6246 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.186028 Oct 15 21:43:43.057961 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1881/6003] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:43:42 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:43:44.381578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1843128608", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7ec4edb6-cfee-4995-af86-9b425fd0a6bb"}], "security_groups": [{"name": "tempest-securitygroup--1671305992"}], "key_name": "tempest-keypair-350156625", "user_data": "}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:44.391799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:44.392500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:44.493094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] validate_networks() for [('7ec4edb6-cfee-4995-af86-9b425fd0a6bb', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:43:44.933615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:43:44.934169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:43:44.934652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:43:44.935107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:43:44.935501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:43:44.946121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:44.946609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:44.947049 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:44.947783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:44.948245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:44.948671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:44.969699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:43:44.975612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:43:44.990913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:43:44.993502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:43:45.045201 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9a039d42-3c20-4101-94ce-9da6813f4efa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.668539 Oct 15 21:43:45.046058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4123/6004] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:43:44 2022] POST /compute/v2.1/servers => generated 409 bytes in 670 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:43:45.057912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-34c305aa-2774-4cfa-95d9-fd6903c4e1be tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:45.058553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-34c305aa-2774-4cfa-95d9-fd6903c4e1be tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:45.073944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=61,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:45.074879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:45.148045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-34c305aa-2774-4cfa-95d9-fd6903c4e1be tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:45.157459 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-34c305aa-2774-4cfa-95d9-fd6903c4e1be tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1238 microversion: 2.1 time: 0.103359 Oct 15 21:43:45.159215 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1882/6005] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:45 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1238 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:46.183850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:46.184473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:46.192076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:46.193959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:46.194574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:46.195509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:46.196144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:46.196753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:46.265373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:46.275305 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b59d4b8f-abdc-4341-a35d-93471abb7b85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1290 microversion: 2.1 time: 0.096264 Oct 15 21:43:46.276219 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4124/6006] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:46 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1290 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:47.301477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:47.302712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:47.310549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:47.311270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:47.311886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:47.312746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:47.313540 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:47.314050 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:47.543401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:47.551094 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1d76b7c0-cb93-46b5-93e9-2bb570dd4965 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.255127 Oct 15 21:43:47.559924 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1883/6007] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:47 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:48.099140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4b69d790-a3d4-4904-b8fe-96870463ca73 req-4f6ccb26-d16f-4edd-879d-5be10e2416fb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:48.111579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4b69d790-a3d4-4904-b8fe-96870463ca73 req-4f6ccb26-d16f-4edd-879d-5be10e2416fb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:48.119437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4b69d790-a3d4-4904-b8fe-96870463ca73 req-4f6ccb26-d16f-4edd-879d-5be10e2416fb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:48.119997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4b69d790-a3d4-4904-b8fe-96870463ca73 req-4f6ccb26-d16f-4edd-879d-5be10e2416fb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:48.133402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4b69d790-a3d4-4904-b8fe-96870463ca73 req-4f6ccb26-d16f-4edd-879d-5be10e2416fb service nova] Creating event network-changed:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch2-master-1 Oct 15 21:43:48.138367 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4b69d790-a3d4-4904-b8fe-96870463ca73 req-4f6ccb26-d16f-4edd-879d-5be10e2416fb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043422 Oct 15 21:43:48.139229 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4125/6008] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:43:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:43:48.575043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:48.575672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:48.582581 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:48.583229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:48.583685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:48.584393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:48.584916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:48.585356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:48.719500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:48.727805 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d896adba-048f-4cbb-8360-d92d7465134f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.156666 Oct 15 21:43:48.728361 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1884/6009] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:48 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:49.751892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:49.752599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:49.759456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:49.759950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:49.760397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:49.761116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:49.761581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:49.762013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:49.894382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:49.902792 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc505e99-886b-4d92-b51f-b7e250a852dc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.155334 Oct 15 21:43:49.903379 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4126/6010] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:49 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:50.927944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:50.928582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:50.936085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:50.936597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:50.937072 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:50.937810 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:50.938284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:50.938729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:51.070635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:51.078602 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7dc3b464-0ad7-43de-894d-7837e9661606 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.155660 Oct 15 21:43:51.079176 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1885/6011] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:50 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:52.101496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:52.102064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:52.108894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:52.109414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:52.109856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:52.110550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:52.111011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:52.111439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:52.265711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:52.273940 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-96372857-7ac3-415a-bee6-21f50724d7f6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.176346 Oct 15 21:43:52.274539 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4127/6012] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:52 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:53.297962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:53.298526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:53.305772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:53.306487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:53.306740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:53.307471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:53.307951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:53.308400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:53.440291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:53.449058 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-805d6087-b406-4d60-aafe-200271f4e3fa tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.155050 Oct 15 21:43:53.449633 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1886/6013] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:53 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:53.842987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8917bea5-ff98-4e82-a564-85ffec5f7f9c req-525a45de-a794-41ad-97ff-fa753ea2f177 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "name": "network-vif-plugged", "status": "completed", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:53.853459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8917bea5-ff98-4e82-a564-85ffec5f7f9c req-525a45de-a794-41ad-97ff-fa753ea2f177 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:53.853944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8917bea5-ff98-4e82-a564-85ffec5f7f9c req-525a45de-a794-41ad-97ff-fa753ea2f177 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:53.854380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8917bea5-ff98-4e82-a564-85ffec5f7f9c req-525a45de-a794-41ad-97ff-fa753ea2f177 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:53.866126 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8917bea5-ff98-4e82-a564-85ffec5f7f9c req-525a45de-a794-41ad-97ff-fa753ea2f177 service nova] Creating event network-vif-plugged:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch2-master-1 Oct 15 21:43:53.871281 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8917bea5-ff98-4e82-a564-85ffec5f7f9c req-525a45de-a794-41ad-97ff-fa753ea2f177 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031996 Oct 15 21:43:53.872069 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4128/6014] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:43:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:43:54.473179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:54.473917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:54.481143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:54.481656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:54.482115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:54.482841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:54.483318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:54.483766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:54.612736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:54.621080 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d575be5-60c3-4ebd-8d08-9a339e45e277 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1364 microversion: 2.1 time: 0.151880 Oct 15 21:43:54.621946 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1887/6015] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:54 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1364 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:55.643715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:55.644486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:55.648809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:55.649559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:55.656628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:55.657138 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:55.657578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:55.658266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:55.658723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:55.659149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:55.792245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:43:55.800081 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ac4eb93d-3db2-479d-a9c1-0c25e16080f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1551 microversion: 2.1 time: 0.158222 Oct 15 21:43:55.800664 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4129/6016] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:43:55 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1551 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:43:55.823793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:43:55.824536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:43:55.826427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:43:55.834290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:55.835022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:55.835590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:55.836495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:55.837244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:55.837698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:55.919435 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-eb2fbb7a-581f-48a8-a13b-2989298e94f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099730 Oct 15 21:43:55.920043 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1888/6017] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:43:55 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:43:56.847929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-f32febe4-5abc-4590-8010-cd52a611c207 req-84e21cf0-0c07-449b-b8b3-f6fd238a9816 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:43:56.859253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f32febe4-5abc-4590-8010-cd52a611c207 req-84e21cf0-0c07-449b-b8b3-f6fd238a9816 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:43:56.859749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f32febe4-5abc-4590-8010-cd52a611c207 req-84e21cf0-0c07-449b-b8b3-f6fd238a9816 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:43:56.860188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-f32febe4-5abc-4590-8010-cd52a611c207 req-84e21cf0-0c07-449b-b8b3-f6fd238a9816 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:43:56.876561 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-f32febe4-5abc-4590-8010-cd52a611c207 req-84e21cf0-0c07-449b-b8b3-f6fd238a9816 service nova] Creating event network-changed:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch2-master-1 Oct 15 21:43:56.882822 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-f32febe4-5abc-4590-8010-cd52a611c207 req-84e21cf0-0c07-449b-b8b3-f6fd238a9816 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038713 Oct 15 21:43:56.883525 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4130/6018] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:43:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:45:01.330661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=15 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:01.331463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=34 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:01.333814 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 15 21:45:01.341524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:01.342151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:01.349190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.349693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.350145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.350853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.351321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.351763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.482940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:01.491343 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fd1ed09c-87a8-4965-836a-14d2331798e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1670 microversion: 2.1 time: 0.163350 Oct 15 21:45:01.492336 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1889/6019] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:01 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1670 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:01.513244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:01.513999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:01.519303 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:01.520007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:01.528593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.529172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.529614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.530313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.530770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.531195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.661944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:01.669794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0e8eb6a-25de-4ac6-98ea-d936d31ecb32 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1670 microversion: 2.1 time: 0.159302 Oct 15 21:45:01.670362 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4131/6020] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:01 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1670 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:01.692485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:01.693292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:01.699217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.699794 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.700251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.700965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.701465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.701907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.833833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:01.841562 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df1e0ee9-cc3d-45ad-8d14-b3fbe12a4ed5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1670 microversion: 2.1 time: 0.153527 Oct 15 21:45:01.842128 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1890/6021] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:01 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1670 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:01.864648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:01.866888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:01.873284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.873777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.874220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.874925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.875390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.875826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.904847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:01.910231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Old instance type m1.heat2, new instance type m1.heat {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:45:01.917387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:45:01.917817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:45:01.918305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:45:01.918722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:45:01.919143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:45:01.981210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b9bdef41-f54c-46bb-82d8-797733b7779b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8/action" status: 202 len: 0 microversion: 2.1 time: 0.121047 Oct 15 21:45:01.981831 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4132/6022] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:45:01 2022] POST /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8/action => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:45:01.990326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:01.990974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:01.996947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.997496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.997958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:01.998685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:01.999169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:01.999619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:02.136305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:02.144572 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-db853f1c-9487-4fbb-b7b8-f5d9528e14f8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1679 microversion: 2.1 time: 0.158376 Oct 15 21:45:02.146004 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1891/6023] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:01 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1679 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:03.171564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:03.172333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:03.180321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:03.180825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:03.181299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:03.182032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:03.182499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:03.182936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:03.687962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:03.696694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3e01c33c-4ecd-4da2-b281-980c1e49bc98 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1679 microversion: 2.1 time: 0.530141 Oct 15 21:45:03.697503 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4133/6024] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:03 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1679 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:04.720650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:04.721333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:04.728258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:04.728762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:04.729255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:04.729968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:04.730467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:04.730909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:04.862720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:04.870692 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6f2092a7-ea89-4b6f-93a3-0975a795bad7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.153944 Oct 15 21:45:04.871271 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1892/6025] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:04 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:05.895005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:05.895628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:05.902566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:05.903055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:05.903499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:05.904202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:05.904674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:05.905132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:06.039507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:06.047627 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a3f0966-c165-45f9-b864-24626360e14f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.156574 Oct 15 21:45:06.048214 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4134/6026] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:05 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:07.071652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:07.072282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:07.079292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:07.079793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:07.080253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:07.080966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:07.081471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:07.081917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:07.213795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:07.221497 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-788940bd-2efe-46b9-8981-9ab21a9db080 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.154045 Oct 15 21:45:07.222059 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1893/6027] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:07 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:08.244556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:08.245200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:08.252260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:08.252744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:08.253220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:08.253916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:08.255188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:08.255788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:08.392229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:08.401265 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46d3fda7-6691-439d-bda1-c2cd61710186 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.160537 Oct 15 21:45:08.401842 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4135/6028] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:08 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:09.425307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:09.425947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:09.433404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:09.433911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:09.434366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:09.435085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:09.435559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:09.436008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:09.568909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:09.576752 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-47f8615f-1a49-4f99-baeb-196a92a6f275 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.155429 Oct 15 21:45:09.577354 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1894/6029] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:09 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:45:10.601244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:10.601857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:10.608714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:10.609232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:10.609674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:10.610399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:10.610856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:10.611284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:10.742862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:10.750582 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0afead7-7673-447f-9e7f-5f3842055563 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.153899 Oct 15 21:45:10.751153 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4136/6030] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:10 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:11.774414 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:11.775046 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:11.782146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:11.782652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:11.783102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:11.783815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:11.784290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:11.784734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:11.917188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:11.924947 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-23717657-3886-4151-bb27-fe34b0e14d16 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.154614 Oct 15 21:45:11.925553 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1895/6031] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:11 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:12.947889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:12.948502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:12.955607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:12.956093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:12.956530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:12.957318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:12.957828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:12.958326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:13.087562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:13.095365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f0760258-4235-4e08-b2a0-245edc2847a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.151370 Oct 15 21:45:13.095939 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4137/6032] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:12 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:14.118573 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:14.119178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:14.126135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:14.126633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:14.127087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:14.127787 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:14.128257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:14.128696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:14.263983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:14.272689 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec631cfe-c770-4820-8b7a-427e8a3d618e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.158371 Oct 15 21:45:14.273290 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1896/6033] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:14 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:15.295581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:15.296135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:15.303552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:15.304046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:15.304490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:15.305225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:15.305692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:15.306126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:15.438590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:15.446962 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f3f8112-df1f-48f1-9ad5-b9f91ac97472 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1684 microversion: 2.1 time: 0.155418 Oct 15 21:45:15.447547 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4138/6034] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:15 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1684 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:16.472164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:16.472840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:16.479942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:16.480446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:16.480898 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:16.481640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:16.482206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:16.482806 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:16.493962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "name": "network-vif-unplugged", "status": "completed", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:16.508719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:16.509499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:16.510195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:16.526341 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] Creating event network-vif-unplugged:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch1-master-1 Oct 15 21:45:16.531957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] Instance cc2752f4-8611-45a9-be62-a077e31fdde8 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:45:16.543067 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a402d418-110b-44cb-83ea-f0320c13178b req-acb343a2-4fd2-4be2-8b08-2a3d6f133c59 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054255 Oct 15 21:45:16.543861 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4139/6035] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:16 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:45:16.800349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:16.808832 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-953040f4-e08f-40d3-93ee-ac299c15a876 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1683 microversion: 2.1 time: 0.340620 Oct 15 21:45:16.809429 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1897/6036] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:16 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1683 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:17.835337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:17.836044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:17.842281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:17.842774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:17.843219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:17.843918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:17.844381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:17.844816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:17.976241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:17.984064 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-82cdc8d8-bf04-4d46-90f5-2a65c4ca3a0a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1681 microversion: 2.1 time: 0.154161 Oct 15 21:45:17.984637 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4140/6037] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:17 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1681 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:18.564268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:18.574708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:18.575200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:18.575646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:18.590262 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] Creating event network-changed:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch1-master-1 Oct 15 21:45:18.595582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] Instance cc2752f4-8611-45a9-be62-a077e31fdde8 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:45:18.607320 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4a5a4163-ac23-413f-9f20-547b6b0ef5cc req-ff25728d-0a1d-45df-a102-5a3f20139a32 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046740 Oct 15 21:45:18.608001 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1898/6038] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:45:19.009291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:19.009929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:19.016902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:19.017546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:19.018000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:19.018724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:19.019199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:19.019669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:19.308219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:19.316205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eb4b0ccf-0fd0-4259-a4c0-501ffbc35a88 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1681 microversion: 2.1 time: 0.311998 Oct 15 21:45:19.318323 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4141/6039] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:19 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1681 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:20.342247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:20.342998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:20.349292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:20.349798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:20.350253 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:20.350967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:20.351444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:20.351891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:20.490971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:20.498946 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aba1c996-476d-4d3f-9d19-99b2f08614d7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1678 microversion: 2.1 time: 0.161929 Oct 15 21:45:20.499510 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1899/6040] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:20 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1678 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:20.522183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:20.522747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:20.529624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:20.530108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:20.530545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:20.531240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:20.531701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:20.532125 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:20.595282 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79560650-4094-43be-b08f-42d56ab52071 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8/action" status: 204 len: 0 microversion: 2.1 time: 0.076824 Oct 15 21:45:20.595922 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4142/6041] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:45:20 2022] POST /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8/action => generated 0 bytes in 78 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:45:20.605718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:20.606381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:20.613084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:20.614324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:20.614963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:20.615784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:20.616267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:20.616718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:21.121488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:21.130116 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-32f46554-95f0-4ff8-b2e9-16baa28d810e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1678 microversion: 2.1 time: 0.529155 Oct 15 21:45:21.130690 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1900/6042] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:20 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1678 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:22.155673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:22.156389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:22.163076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.163581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.164026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.164747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.165422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.165653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.315413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:22.323556 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3927d87-def5-4ca5-942a-741f535e7178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1670 microversion: 2.1 time: 0.172998 Oct 15 21:45:22.324136 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4143/6043] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:22 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1670 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:22.347056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:22.348320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:22.355278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.355775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.356228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.356934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.357433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.357874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.486443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:22.494414 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ab4b82b3-fd8a-455f-93c6-18c7242676bd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1670 microversion: 2.1 time: 0.151258 Oct 15 21:45:22.494994 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1901/6044] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:22 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1670 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:22.517282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:22.517911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:22.524484 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.524981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.525480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.526214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.526684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.527123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.654597 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:22.662497 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad45f192-2aac-432b-8c49-1196f58e4986 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1670 microversion: 2.1 time: 0.149388 Oct 15 21:45:22.663078 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4144/6045] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:22 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1670 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:22.685359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:22.685809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:22.692297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.692792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.693272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.693975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.694442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.694951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.720572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:45:22.796007 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-311d2e1f-cc04-4b9b-8c06-dc53aa7dfddc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 204 len: 0 microversion: 2.1 time: 0.114376 Oct 15 21:45:22.796663 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1902/6046] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:45:22 2022] DELETE /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:45:22.806068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:22.806714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:22.812739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.813283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.813715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.814415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:22.814881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:22.815314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:22.832153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:22.950782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:22.958910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25f9becc-d289-48de-8b97-fa99807b2d5b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1676 microversion: 2.1 time: 0.157936 Oct 15 21:45:22.959486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4145/6047] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:22 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1676 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:23.402320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-7c031789-4f38-46af-befa-8e0cc22f94f2 req-f180cac0-8754-4d6e-9291-283d41d8a28b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "name": "network-vif-plugged", "status": "completed", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:23.413884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7c031789-4f38-46af-befa-8e0cc22f94f2 req-f180cac0-8754-4d6e-9291-283d41d8a28b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:23.414402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7c031789-4f38-46af-befa-8e0cc22f94f2 req-f180cac0-8754-4d6e-9291-283d41d8a28b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:23.414875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-7c031789-4f38-46af-befa-8e0cc22f94f2 req-f180cac0-8754-4d6e-9291-283d41d8a28b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:23.425517 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-7c031789-4f38-46af-befa-8e0cc22f94f2 req-f180cac0-8754-4d6e-9291-283d41d8a28b service nova] Creating event network-vif-plugged:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch1-master-1 Oct 15 21:45:23.431217 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-7c031789-4f38-46af-befa-8e0cc22f94f2 req-f180cac0-8754-4d6e-9291-283d41d8a28b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032946 Oct 15 21:45:23.431841 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1903/6048] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:45:23.981818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:23.982450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:23.989479 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:23.989968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:23.990523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:23.991229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:23.991690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:23.992124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:24.174394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:24.183694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f1464536-3443-49db-86f0-1a4121e1fed3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 200 len: 1676 microversion: 2.1 time: 0.205821 Oct 15 21:45:24.184289 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4146/6049] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:23 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 1676 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:24.953952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-67e7db35-0699-4c52-9602-ed329ad346ce req-eb049965-9bcf-4713-8bb6-3bf30837c36f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc2752f4-8611-45a9-be62-a077e31fdde8", "name": "network-vif-deleted", "tag": "c3707a37-0c70-4a14-b45e-21f5bb3a0b1f"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:24.970048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-67e7db35-0699-4c52-9602-ed329ad346ce req-eb049965-9bcf-4713-8bb6-3bf30837c36f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:24.970637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-67e7db35-0699-4c52-9602-ed329ad346ce req-eb049965-9bcf-4713-8bb6-3bf30837c36f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:24.971174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-67e7db35-0699-4c52-9602-ed329ad346ce req-eb049965-9bcf-4713-8bb6-3bf30837c36f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:24.991459 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-67e7db35-0699-4c52-9602-ed329ad346ce req-eb049965-9bcf-4713-8bb6-3bf30837c36f service nova] Creating event network-vif-deleted:c3707a37-0c70-4a14-b45e-21f5bb3a0b1f for instance cc2752f4-8611-45a9-be62-a077e31fdde8 on nch1-master-1 Oct 15 21:45:24.996332 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-67e7db35-0699-4c52-9602-ed329ad346ce req-eb049965-9bcf-4713-8bb6-3bf30837c36f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047369 Oct 15 21:45:24.996965 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1904/6050] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:45:25.208554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:25.209248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:25.216376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:25.216949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:25.217915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:25.218420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:25.219021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:25.219520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:25.234513 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 HTTP/1.1 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'cc2752f4-8611-45a9-be62-a077e31fdde8'} Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:45:25.236344 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5e674487-0bfe-4ec9-9e85-6d6552770c41 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 404 len: 111 microversion: 2.1 time: 0.034231 Oct 15 21:45:25.242366 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4147/6051] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:25 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:45:25.252394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:25.252999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:25.259864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:25.260398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:25.260839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:25.261582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:25.262090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:25.262538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:25.280238 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 HTTP/1.1 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'cc2752f4-8611-45a9-be62-a077e31fdde8'} Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:45:25.281994 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5746521e-5a78-4136-99e9-da054898a6c7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 404 len: 111 microversion: 2.1 time: 0.035574 Oct 15 21:45:25.286543 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1905/6052] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:25 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:45:25.336668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-472311963", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7ec4edb6-cfee-4995-af86-9b425fd0a6bb"}], "security_groups": [{"name": "tempest-securitygroup--1671305992"}], "key_name": "tempest-keypair-350156625", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:25.453017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] validate_networks() for [('7ec4edb6-cfee-4995-af86-9b425fd0a6bb', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:45:25.848433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:45:25.848876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:45:25.849392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:45:25.849811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:45:25.850233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:45:25.859845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:25.860343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:25.860790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:25.861565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:25.862033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:25.862470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:25.882526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:45:25.887989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:45:25.901448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:45:25.903800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:45:25.954168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8ecd8e78-0de9-4f28-b8fb-8e326eb99dd6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.622273 Oct 15 21:45:25.955013 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4148/6053] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:45:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 624 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:45:25.966391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3e9d9f37-16b4-4b17-b4a8-8b3db04aafe1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:25.967085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3e9d9f37-16b4-4b17-b4a8-8b3db04aafe1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:26.027518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3e9d9f37-16b4-4b17-b4a8-8b3db04aafe1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:26.035793 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3e9d9f37-16b4-4b17-b4a8-8b3db04aafe1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1237 microversion: 2.1 time: 0.073244 Oct 15 21:45:26.036698 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1906/6054] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:25 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1237 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:27.061737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:27.062428 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:27.070024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:27.070520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:27.070963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:27.071680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:27.072173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:27.072652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:27.153731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:27.161817 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4627888c-d211-4c49-9c19-0f9c08833af8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1295 microversion: 2.1 time: 0.105057 Oct 15 21:45:27.162397 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4149/6055] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:27 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1295 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:28.185977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:28.186612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:28.194614 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:28.195127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:28.195988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:28.197902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:28.198553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:28.199004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:28.203658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:28.397278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:28.405889 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-887eee80-9c02-4a66-b564-5d52eacfdf13 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1363 microversion: 2.1 time: 0.223851 Oct 15 21:45:28.406685 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1907/6056] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:28 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1363 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:28.427160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2156e4d8-b7a1-4198-819c-85aebc7b659a req-082a979e-2bcc-4c39-a7e5-0d4bf62846d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:28.438221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2156e4d8-b7a1-4198-819c-85aebc7b659a req-082a979e-2bcc-4c39-a7e5-0d4bf62846d5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:28.438704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2156e4d8-b7a1-4198-819c-85aebc7b659a req-082a979e-2bcc-4c39-a7e5-0d4bf62846d5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:28.439348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2156e4d8-b7a1-4198-819c-85aebc7b659a req-082a979e-2bcc-4c39-a7e5-0d4bf62846d5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:28.454430 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2156e4d8-b7a1-4198-819c-85aebc7b659a req-082a979e-2bcc-4c39-a7e5-0d4bf62846d5 service nova] Creating event network-changed:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch1-master-1 Oct 15 21:45:28.460085 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2156e4d8-b7a1-4198-819c-85aebc7b659a req-082a979e-2bcc-4c39-a7e5-0d4bf62846d5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036599 Oct 15 21:45:28.460694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4150/6057] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:45:29.429985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:29.430663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:29.437541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:29.438040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:29.438488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:29.439190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:29.439662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:29.440098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:29.570014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:29.577883 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-81611374-9be1-444d-9106-af52113345e5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1363 microversion: 2.1 time: 0.152151 Oct 15 21:45:29.578472 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1908/6058] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:29 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1363 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:30.603186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:30.603931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:30.610918 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:30.611417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:30.611861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:30.612710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:30.613230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:30.613866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:30.743151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:30.750887 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f9676f7-eed4-4222-adf3-9576116f9ecb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1363 microversion: 2.1 time: 0.153670 Oct 15 21:45:30.751455 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4151/6059] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:30 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1363 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:31.776229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:31.776935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:31.779767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:31.785877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:31.786547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:31.787010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:31.788101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:31.788586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:31.789054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:32.250255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:32.258678 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-66649cfb-ead1-403b-90a6-9abaa63fab73 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1363 microversion: 2.1 time: 0.486541 Oct 15 21:45:32.259254 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1909/6060] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:31 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1363 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:33.283455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:33.284003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:33.287543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:33.293245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:33.293734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:33.294174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:33.294905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:33.295366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:33.295821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:33.434793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:33.442898 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d6d6259-bbdd-4db9-942e-fe816d847a8d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1363 microversion: 2.1 time: 0.163657 Oct 15 21:45:33.444049 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4152/6061] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:33 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:33.454391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0f56a60c-727f-477f-afca-8c2abd7954f9 req-90bc5c83-bb38-4385-aabc-01df61a43e08 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "name": "network-vif-plugged", "status": "completed", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:33.467360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0f56a60c-727f-477f-afca-8c2abd7954f9 req-90bc5c83-bb38-4385-aabc-01df61a43e08 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:33.467866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0f56a60c-727f-477f-afca-8c2abd7954f9 req-90bc5c83-bb38-4385-aabc-01df61a43e08 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:33.468318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0f56a60c-727f-477f-afca-8c2abd7954f9 req-90bc5c83-bb38-4385-aabc-01df61a43e08 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:33.480951 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0f56a60c-727f-477f-afca-8c2abd7954f9 req-90bc5c83-bb38-4385-aabc-01df61a43e08 service nova] Creating event network-vif-plugged:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch1-master-1 Oct 15 21:45:33.486214 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0f56a60c-727f-477f-afca-8c2abd7954f9 req-90bc5c83-bb38-4385-aabc-01df61a43e08 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035855 Oct 15 21:45:33.486808 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1910/6062] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:45:34.468035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:34.468568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:34.476078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:34.476575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:34.477023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:34.477750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:34.478219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:34.478654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:34.610443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:34.621473 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-83d6674c-3d5a-42d2-a055-137f9501fb45 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1363 microversion: 2.1 time: 0.157657 Oct 15 21:45:34.622161 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4153/6063] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:34 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:35.647531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:35.648260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:35.657021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:35.657646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:35.658116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:35.658850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:35.659332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:35.659901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:35.800932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:45:35.809315 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ff73f8c3-70a6-42d0-b663-aaf2f06ff424 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1550 microversion: 2.1 time: 0.167188 Oct 15 21:45:35.809902 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1911/6064] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:45:35 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1550 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:45:35.833359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:45:35.833876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:45:35.840703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:35.841233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:35.841660 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:35.842350 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:35.842806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:35.843229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:35.870522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:35.895279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:35.921332 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8fc9077f-166e-4380-a59d-6c16bf2c119f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092507 Oct 15 21:45:35.921973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4154/6065] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:45:35 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:45:36.864829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0d0d5dd5-ee69-4e52-8917-42a6e67d2591 req-8cbaf58e-8f57-4cd3-964d-cb02ccf8404c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:45:36.869958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:45:36.876708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0d0d5dd5-ee69-4e52-8917-42a6e67d2591 req-8cbaf58e-8f57-4cd3-964d-cb02ccf8404c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:45:36.877730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0d0d5dd5-ee69-4e52-8917-42a6e67d2591 req-8cbaf58e-8f57-4cd3-964d-cb02ccf8404c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:45:36.877730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0d0d5dd5-ee69-4e52-8917-42a6e67d2591 req-8cbaf58e-8f57-4cd3-964d-cb02ccf8404c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:45:36.889132 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0d0d5dd5-ee69-4e52-8917-42a6e67d2591 req-8cbaf58e-8f57-4cd3-964d-cb02ccf8404c service nova] Creating event network-changed:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch1-master-1 Oct 15 21:45:36.894516 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0d0d5dd5-ee69-4e52-8917-42a6e67d2591 req-8cbaf58e-8f57-4cd3-964d-cb02ccf8404c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034025 Oct 15 21:45:36.895118 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1912/6066] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:45:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:11.959154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:11.959906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=43 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:11.964826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6c53e78d-36d3-4da7-998a-109412c72a4b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:11.971960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c53e78d-36d3-4da7-998a-109412c72a4b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:11.972453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c53e78d-36d3-4da7-998a-109412c72a4b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:11.972900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6c53e78d-36d3-4da7-998a-109412c72a4b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:11.996682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6c53e78d-36d3-4da7-998a-109412c72a4b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 21:46:12.038947 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6c53e78d-36d3-4da7-998a-109412c72a4b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action" status: 202 len: 0 microversion: 2.1 time: 0.081820 Oct 15 21:46:12.042644 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4155/6067] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:46:11 2022] POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:46:12.046558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=7 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:12.047382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=26 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:12.057667 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:12.058704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:12.067454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:12.067966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:12.068426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:12.069190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:12.069673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:12.070124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:12.269178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:12.277620 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-67b43a64-8597-46a2-8145-a7d9259c80de tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.233298 Oct 15 21:46:12.278213 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1913/6068] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:12 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:13.303245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:13.303883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:13.311072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:13.311564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:13.312000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:13.312709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:13.313213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:13.313652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:13.566470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:13.574267 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4e1583e-8824-4823-875a-dad42a6cab3d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.276052 Oct 15 21:46:13.574854 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4156/6069] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:13 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:14.598207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:14.599128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:14.606102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:14.606611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:14.607068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:14.607785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:14.608257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:14.608704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:14.736526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:14.744983 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-91db2b33-a874-4d8a-a223-89ec65963822 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.151080 Oct 15 21:46:14.745656 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1914/6070] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:14 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:15.769517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:15.770406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:15.777304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:15.777809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:15.778239 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:15.778938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:15.779399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:15.779833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:15.915431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:15.923291 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e01f9aa8-6fe0-4b92-b87d-c8ed900a4ab8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.157731 Oct 15 21:46:15.923877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4157/6071] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:15 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:16.948520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:16.949171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:16.956262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:16.956763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:16.957287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:16.958063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:16.958642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:16.959089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:17.088292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:17.096088 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0103dc61-9900-4baa-b4e6-533abaa9bc05 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.151521 Oct 15 21:46:17.096657 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1915/6072] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:16 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:18.120153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:18.120680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:18.127658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:18.128144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:18.128585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:18.129329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:18.129792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:18.130223 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:18.259987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:18.268115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d216c517-ddc4-4016-b0cd-d007b5e03ea2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.151988 Oct 15 21:46:18.268694 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4158/6073] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:18 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:19.292179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:19.292773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:19.299679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:19.300185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:19.300650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:19.301399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:19.301882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:19.302335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:19.434130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:19.442002 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-61a17a4b-a761-4800-89b7-f6b00677c75f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.153893 Oct 15 21:46:19.442581 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1916/6074] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:19 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:20.466369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:20.466896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:20.474572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:20.475058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:20.475503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:20.476207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:20.476664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:20.477109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:20.606277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:20.614380 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9d9ff272-323f-4b00-ac65-37dffcf21ec3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.152112 Oct 15 21:46:20.614970 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4159/6075] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:20 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:21.638304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:21.638941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:21.646368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:21.646885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:21.647347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:21.648070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:21.648550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:21.649002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:21.778911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:21.787305 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-66427375-fb53-4885-ad7a-36beb8a132e6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.152938 Oct 15 21:46:21.787878 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1917/6076] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:21 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:22.811407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:22.812026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:22.820263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:22.820757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:22.821230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:22.821970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:22.822401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:22.822874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:22.952548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:22.960969 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-07b2e5b8-29ab-4b2d-aaa0-095aead1e4e3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.153471 Oct 15 21:46:22.961563 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4160/6077] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:22 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:23.984654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:23.985314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:23.993246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:23.993759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:23.994216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:23.994932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:23.995413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:23.995864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:24.127383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:24.136067 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cd8fdb8d-f8b2-401c-bf77-34c19704bcdf tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.155350 Oct 15 21:46:24.136632 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1918/6078] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:23 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:25.160882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:25.161565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:25.168412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:25.168907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:25.169394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:25.170103 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:25.170570 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:25.171008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:25.299265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:25.307691 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d0d98b70-8d2c-4342-a8c4-12f8deca9d1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.151151 Oct 15 21:46:25.308262 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4161/6079] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:25 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:26.331763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:26.333372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:26.340290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:26.340783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:26.341268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:26.341984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:26.342461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:26.342909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:26.484112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:26.492838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-eb99ecd5-91dc-471b-a0e7-742158a376ce tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.164939 Oct 15 21:46:26.493562 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1919/6080] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:26 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:27.519903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:27.520798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:27.527430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:27.527916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:27.528359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:27.529078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:27.529543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:27.529974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:27.662277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:27.670087 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-774230bd-3512-494f-ad43-3dbcacefc32b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1656 microversion: 2.1 time: 0.154803 Oct 15 21:46:27.670673 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4162/6081] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:27 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1656 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:27.693457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:27.695731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:27.702606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:27.703254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:27.703790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:27.704503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:27.704976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:27.705447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:27.733623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:27.739677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Old instance type m1.heat2, new instance type m1.heat {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:46:27.746182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:46:27.746625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:46:27.747117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:46:27.747547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:46:27.747977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:46:27.813892 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e0fef194-5ad0-4bed-87eb-d9d0a6ee7769 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action" status: 202 len: 0 microversion: 2.1 time: 0.124797 Oct 15 21:46:27.814429 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1920/6082] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:46:27 2022] POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action => generated 0 bytes in 126 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:46:27.822693 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:27.823323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:27.829722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:27.830220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:27.830668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:27.831379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:27.831921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:27.832584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:27.972123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:27.980460 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58cbbc9b-4ca3-4701-a444-f2903ac32a60 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.161454 Oct 15 21:46:27.981226 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4163/6083] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:27 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:29.006324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:29.006877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:29.013753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:29.014254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:29.014960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:29.015662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:29.016135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:29.016576 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:29.154821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:29.163485 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-704eade4-70d4-4680-8743-e1aee8462b31 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1679 microversion: 2.1 time: 0.162692 Oct 15 21:46:29.164137 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1921/6084] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:28 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1679 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:30.187180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:30.187828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:30.195045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:30.195547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:30.195998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:30.196713 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:30.197231 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:30.197672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:30.330784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:30.339348 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48971c1e-ce55-48c3-8017-fd1beaeeaf55 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1684 microversion: 2.1 time: 0.156109 Oct 15 21:46:30.339930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4164/6085] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:30 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1684 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:31.363931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:31.365247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:31.372303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:31.372814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:31.373319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:31.374043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:31.374523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:31.375003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:31.549527 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:31.558369 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b5d084a2-09c3-4a1a-b30b-0ae5e110c355 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1684 microversion: 2.1 time: 0.198431 Oct 15 21:46:31.559067 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1922/6086] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:31 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1684 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:32.278290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "name": "network-vif-unplugged", "status": "completed", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:32.289192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:32.289684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:32.290121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:32.303015 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] Creating event network-vif-unplugged:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch2-master-1 Oct 15 21:46:32.308646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:46:32.320779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-4f21b2f4-97dd-4326-b866-f07455a82233 req-dbe134e2-1982-4111-b658-191c8534b3c3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.101105 Oct 15 21:46:32.321525 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4165/6087] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:32 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:32.582873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:32.583631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:32.590867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:32.591381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:32.591836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:32.592561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:32.593055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:32.593509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:32.844836 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:32.855032 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-55fa172e-7fa5-4f9a-8519-f945d04184d1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1683 microversion: 2.1 time: 0.276142 Oct 15 21:46:32.855622 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1923/6088] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:32 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1683 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:33.881908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:33.882606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:33.888759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:33.889282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:33.889998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:33.890692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:33.891150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:33.891577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:34.027214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:34.035084 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a358b52-40e2-48a4-a6e9-ddb72649cfbd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1681 microversion: 2.1 time: 0.158938 Oct 15 21:46:34.035662 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4166/6089] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:33 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:34.652556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:34.663039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:34.663541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:34.663996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:34.675693 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] Creating event network-changed:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch2-master-1 Oct 15 21:46:34.681056 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:46:34.691715 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-a8bc200c-2ae9-4318-a2f4-0da7e0a03c37 req-cbf6b9e2-b1a4-4e34-9991-c3aa30769e5a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043432 Oct 15 21:46:34.692305 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1924/6090] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:35.059278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:35.059900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:35.066709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:35.067200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:35.068131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:35.068864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:35.069346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:35.069809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:35.275081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:35.283024 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ec50492-3d58-430a-bbbe-e8fbc813f49b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1681 microversion: 2.1 time: 0.227602 Oct 15 21:46:35.283587 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4167/6091] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:35 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1681 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:36.308962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:36.309652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:36.316891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:36.318062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:36.318529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:36.319255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:36.319735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:36.320190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:36.546994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:36.556338 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d19d6504-3061-4a8b-9b7d-282393dc25b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1677 microversion: 2.1 time: 0.251630 Oct 15 21:46:36.557049 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1925/6092] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:36 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1677 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:36.580561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:36.581260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:36.589185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:36.589683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:36.590121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:36.590828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:36.591280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:36.591972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:36.656170 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dc01d346-57ae-4e35-b58c-94b4469fb7ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action" status: 204 len: 0 microversion: 2.1 time: 0.079279 Oct 15 21:46:36.656837 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4168/6093] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:46:36 2022] POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action => generated 0 bytes in 81 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:46:36.666943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:36.669159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:36.676331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:36.678597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:36.679203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:36.680004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:36.680500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:36.681193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:36.935892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:36.943816 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9e842d63-eac1-4c8e-91e0-99b0de2382d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1677 microversion: 2.1 time: 0.282768 Oct 15 21:46:36.944505 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1926/6094] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:36 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1677 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:37.970674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:37.971382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:37.978913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:37.979401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:37.979865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:37.980988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:37.981240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:37.981669 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:38.126495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:38.129448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:38.137851 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3ed24f6-c35b-4021-91ad-a3ac0b72d740 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1656 microversion: 2.1 time: 0.173119 Oct 15 21:46:38.138439 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4169/6095] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:37 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1656 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:38.162886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:38.163530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:38.170625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:38.171140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:38.171603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:38.172332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:38.172815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:38.173298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:38.303597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:38.312203 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c166ac24-e747-4ec2-a042-f0eae05063a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1656 microversion: 2.1 time: 0.154008 Oct 15 21:46:38.312778 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1927/6096] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:38 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1656 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:38.336968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0fde793f-6cf5-4dfb-b3a8-f1ce4f786cdc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:38.345591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fde793f-6cf5-4dfb-b3a8-f1ce4f786cdc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:38.346086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fde793f-6cf5-4dfb-b3a8-f1ce4f786cdc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:38.346527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fde793f-6cf5-4dfb-b3a8-f1ce4f786cdc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:38.374329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0fde793f-6cf5-4dfb-b3a8-f1ce4f786cdc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Going to try to start instance {{(pid=138688) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 21:46:38.419127 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0fde793f-6cf5-4dfb-b3a8-f1ce4f786cdc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action" status: 202 len: 0 microversion: 2.1 time: 0.088598 Oct 15 21:46:38.419743 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4170/6097] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:46:38 2022] POST /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5/action => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:46:38.428986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:38.429725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:38.436633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:38.437496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:38.438095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:38.438971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:38.440123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:38.440683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:38.463024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:38.683427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:38.691381 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5b0396e-1433-4313-b147-f19b2963684c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1665 microversion: 2.1 time: 0.267073 Oct 15 21:46:38.691962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1928/6098] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:38 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1665 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:39.716832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:39.717571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:39.724307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:39.724788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:39.725259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:39.725946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:39.726403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:39.726827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:39.857491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:39.865977 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-43006589-38ce-45fd-8deb-39f820c8490c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1669 microversion: 2.1 time: 0.154310 Oct 15 21:46:39.866546 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4171/6099] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:39 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1669 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:39.888450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:39.888980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:39.896964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:39.897553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:39.898010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:39.898724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:39.899195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:39.899694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:39.925518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:46:39.999491 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1754c27e-dcbc-4a9c-9ad0-b9ed49e028f7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 204 len: 0 microversion: 2.1 time: 0.114545 Oct 15 21:46:40.000082 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1929/6100] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:46:39 2022] DELETE /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:46:40.008333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:40.008945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:40.015257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:40.015745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:40.016211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:40.016903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:40.017391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:40.017820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:40.151095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:40.158943 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-055572d2-18aa-4120-809d-9bbf4392d041 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1675 microversion: 2.1 time: 0.154550 Oct 15 21:46:40.159509 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4172/6101] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:40 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1675 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:40.644187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b6b2d596-b02b-47c5-945a-c66cc676ed2c req-e7a10bf7-cf1a-4a50-b06f-77bc9903a63f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "name": "network-vif-plugged", "status": "completed", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:40.654409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b6b2d596-b02b-47c5-945a-c66cc676ed2c req-e7a10bf7-cf1a-4a50-b06f-77bc9903a63f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:40.654902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b6b2d596-b02b-47c5-945a-c66cc676ed2c req-e7a10bf7-cf1a-4a50-b06f-77bc9903a63f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:40.655353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b6b2d596-b02b-47c5-945a-c66cc676ed2c req-e7a10bf7-cf1a-4a50-b06f-77bc9903a63f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:40.666652 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b6b2d596-b02b-47c5-945a-c66cc676ed2c req-e7a10bf7-cf1a-4a50-b06f-77bc9903a63f service nova] Creating event network-vif-plugged:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch2-master-1 Oct 15 21:46:40.671576 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b6b2d596-b02b-47c5-945a-c66cc676ed2c req-e7a10bf7-cf1a-4a50-b06f-77bc9903a63f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079923 Oct 15 21:46:40.672155 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1930/6102] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:41.184583 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:41.185220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:41.192385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:41.192884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:41.193359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:41.194216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:41.194694 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:41.195137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:41.325363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:41.333191 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2600844-5440-475a-a4a1-4f5b8b64f7db tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1675 microversion: 2.1 time: 0.152925 Oct 15 21:46:41.333770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4173/6103] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:41 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1675 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:42.177001 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:46:42.177579 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:46:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 15 21:46:42.182585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d5e6a6fc-66a2-4dc3-9f37-0cc0de775dab req-873f7acf-d577-4ad7-bcb4-98158762c003 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "309ca9c6-85ac-47ae-bda8-3c8391ac41b5", "name": "network-vif-deleted", "tag": "65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:42.187875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:42.205874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d5e6a6fc-66a2-4dc3-9f37-0cc0de775dab req-873f7acf-d577-4ad7-bcb4-98158762c003 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:42.206580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d5e6a6fc-66a2-4dc3-9f37-0cc0de775dab req-873f7acf-d577-4ad7-bcb4-98158762c003 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:42.207223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d5e6a6fc-66a2-4dc3-9f37-0cc0de775dab req-873f7acf-d577-4ad7-bcb4-98158762c003 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:42.222373 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d5e6a6fc-66a2-4dc3-9f37-0cc0de775dab req-873f7acf-d577-4ad7-bcb4-98158762c003 service nova] Creating event network-vif-deleted:65eb6a4b-a0d5-45a5-9e6a-a4cad16c37e8 for instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 on nch2-master-1 Oct 15 21:46:42.227027 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d5e6a6fc-66a2-4dc3-9f37-0cc0de775dab req-873f7acf-d577-4ad7-bcb4-98158762c003 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052541 Oct 15 21:46:42.227645 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1931/6104] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:42.358250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:42.359650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:42.363454 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:46:42.363589 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:46:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 (ip 10.222.0.16) !!! Oct 15 21:46:42.367164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=39 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:42.380559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:42.381146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:42.381596 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:42.382319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:42.382797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:42.383238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:42.415025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:42.415439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:42.415886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:42.416582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lazy-loading 'fault' on Instance uuid 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:46:42.462002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:42.470074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30812fcb-61e3-4c1e-9e48-68a1c9bef11d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 200 len: 1327 microversion: 2.1 time: 0.116225 Oct 15 21:46:42.471058 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4174/6105] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:42 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 1327 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:43.496639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:43.497319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:43.503592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:43.504098 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:43.504557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:43.505304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:43.505782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:43.506232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:43.520170 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 HTTP/1.1 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '309ca9c6-85ac-47ae-bda8-3c8391ac41b5'} Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:46:43.521853 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2ef8a3fc-5cbb-4bec-a843-31181e40c4ef tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 404 len: 111 microversion: 2.1 time: 0.031956 Oct 15 21:46:43.526729 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1932/6106] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:43 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:46:43.539985 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:43.540635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:43.547588 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:43.548072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:43.548543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:43.549457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:43.549920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:43.550352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:43.566422 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 HTTP/1.1 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '309ca9c6-85ac-47ae-bda8-3c8391ac41b5'} Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:46:43.568355 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1238269b-b1e4-41f9-aaca-0b2799c5e8e7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 404 len: 111 microversion: 2.1 time: 0.034306 Oct 15 21:46:43.573080 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4175/6107] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:43 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:46:43.619434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1903770297", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7ec4edb6-cfee-4995-af86-9b425fd0a6bb"}], "security_groups": [{"name": "tempest-securitygroup--1671305992"}], "key_name": "tempest-keypair-350156625", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:43.723862 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] validate_networks() for [('7ec4edb6-cfee-4995-af86-9b425fd0a6bb', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:46:44.164733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:46:44.165221 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:46:44.165719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:46:44.166151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:46:44.166589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:46:44.175515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:44.176025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:44.177138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:44.177948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:44.178410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:44.179404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:44.197093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:46:44.202591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:46:44.215649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:46:44.218029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:46:44.265675 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a5a35030-9655-4f74-b1c1-0ea6876d8877 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.650057 Oct 15 21:46:44.266331 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1933/6108] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:46:43 2022] POST /compute/v2.1/servers => generated 406 bytes in 651 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:46:44.277676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c10c656-689c-43cd-a6e7-4cf7805ea993 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:44.278386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c10c656-689c-43cd-a6e7-4cf7805ea993 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:44.337712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4c10c656-689c-43cd-a6e7-4cf7805ea993 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:44.344842 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c10c656-689c-43cd-a6e7-4cf7805ea993 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1238 microversion: 2.1 time: 0.070975 Oct 15 21:46:44.345419 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4176/6109] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:44 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1238 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:46:45.373604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:45.374374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:45.382231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:45.382846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:45.383323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:45.384061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:45.384547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:45.385000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:45.464853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:45.473130 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c52b3355-9ead-42b0-9034-156958feec44 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1296 microversion: 2.1 time: 0.104620 Oct 15 21:46:45.473709 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1934/6110] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:45 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1296 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:46.498675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:46.499304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:46.507210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:46.507697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:46.508141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:46.508849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:46.509341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:46.509783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:46.711001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:46.719040 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1fbd560d-582f-4617-b31c-0e47a10513c5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.224298 Oct 15 21:46:46.719625 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4177/6111] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:46 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:47.004286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-890d276e-dc23-47ae-8c24-ad1653288f0c req-f672099b-f9b9-453c-8993-9d507cbb5416 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:47.028817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-890d276e-dc23-47ae-8c24-ad1653288f0c req-f672099b-f9b9-453c-8993-9d507cbb5416 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:47.029352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-890d276e-dc23-47ae-8c24-ad1653288f0c req-f672099b-f9b9-453c-8993-9d507cbb5416 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:47.029809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-890d276e-dc23-47ae-8c24-ad1653288f0c req-f672099b-f9b9-453c-8993-9d507cbb5416 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:47.043092 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-890d276e-dc23-47ae-8c24-ad1653288f0c req-f672099b-f9b9-453c-8993-9d507cbb5416 service nova] Creating event network-changed:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:46:47.048490 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-890d276e-dc23-47ae-8c24-ad1653288f0c req-f672099b-f9b9-453c-8993-9d507cbb5416 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047825 Oct 15 21:46:47.049158 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1935/6112] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:47.743909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:47.744455 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:47.751393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:47.751884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:47.752329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:47.753024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:47.753516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:47.753947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:47.887837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:47.895874 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40b9fbc3-ac87-444f-aef4-ed4ed166b1d4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.156156 Oct 15 21:46:47.896451 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4178/6113] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:47 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:48.921393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:48.922138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:48.928510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:48.929006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:48.929482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:48.930188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:48.930659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:48.931100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:49.064567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:49.072420 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ad67c64-e0ed-4ee6-8376-76d7bb4887cc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.156213 Oct 15 21:46:49.072983 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1936/6114] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:48 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:50.096118 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:50.096643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:50.103865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:50.104358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:50.104802 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:50.105539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:50.106001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:50.106433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:50.237139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:50.245622 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a933072-3a60-469d-bf3b-af1b59859268 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.153710 Oct 15 21:46:50.246204 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4179/6115] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:50 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:51.268419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:51.269061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:51.276473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:51.276970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:51.277452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:51.278182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:51.278651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:51.279091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:51.413450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:51.421259 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a309d73-cdac-4cfa-a94d-a9ca2cbd0864 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.156927 Oct 15 21:46:51.421868 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1937/6116] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:51 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:52.445687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:52.446331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:52.454011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:52.454571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:52.455108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:52.455850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:52.456322 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:52.456758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:52.591173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:52.599057 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5b23e1c-da23-43f7-9554-20622011d40b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.158151 Oct 15 21:46:52.599883 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4180/6117] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:52 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:52.623207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-6650a4b8-93d9-4403-a069-7046a6895bd9 req-53feb696-7952-4c54-ad59-961292bef11e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "name": "network-vif-plugged", "status": "completed", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:52.634544 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6650a4b8-93d9-4403-a069-7046a6895bd9 req-53feb696-7952-4c54-ad59-961292bef11e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:52.635117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6650a4b8-93d9-4403-a069-7046a6895bd9 req-53feb696-7952-4c54-ad59-961292bef11e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:52.635641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6650a4b8-93d9-4403-a069-7046a6895bd9 req-53feb696-7952-4c54-ad59-961292bef11e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:52.647670 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-6650a4b8-93d9-4403-a069-7046a6895bd9 req-53feb696-7952-4c54-ad59-961292bef11e service nova] Creating event network-vif-plugged:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:46:52.652676 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-6650a4b8-93d9-4403-a069-7046a6895bd9 req-53feb696-7952-4c54-ad59-961292bef11e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033260 Oct 15 21:46:52.653317 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1938/6118] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:53.626423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:53.627037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:53.634279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:53.634771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:53.635217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:53.635945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:53.636402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:53.636828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:53.766963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:53.775020 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-05209f1e-633f-49e9-a96d-89ccd3aa4aa5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1364 microversion: 2.1 time: 0.152529 Oct 15 21:46:53.775590 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4181/6119] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:53 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:54.797536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:54.798352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:54.802668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:54.803525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:54.810644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:54.811151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:54.811620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:54.812370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:54.812852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:54.813333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:54.946239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:46:54.954154 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fc799a42-c712-480a-b26c-842221adf9d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1551 microversion: 2.1 time: 0.158453 Oct 15 21:46:54.954746 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1939/6120] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:46:54 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1551 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:46:54.977510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:46:54.978080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:46:54.980006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:46:54.986513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:54.987000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:54.987441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:54.988136 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:54.988593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:54.989023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:55.070739 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eae9d1c4-cf25-488f-8917-472a5789c323 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.097127 Oct 15 21:46:55.071342 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4182/6121] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:46:54 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:46:55.997645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8df1b225-ab5c-4852-ae55-f4c8e97423fb req-bbcc536a-d46b-4715-8ee2-50693ec6b084 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:46:56.008188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8df1b225-ab5c-4852-ae55-f4c8e97423fb req-bbcc536a-d46b-4715-8ee2-50693ec6b084 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:46:56.008691 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8df1b225-ab5c-4852-ae55-f4c8e97423fb req-bbcc536a-d46b-4715-8ee2-50693ec6b084 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:46:56.009176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8df1b225-ab5c-4852-ae55-f4c8e97423fb req-bbcc536a-d46b-4715-8ee2-50693ec6b084 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:46:56.020551 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8df1b225-ab5c-4852-ae55-f4c8e97423fb req-bbcc536a-d46b-4715-8ee2-50693ec6b084 service nova] Creating event network-changed:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:46:56.026444 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8df1b225-ab5c-4852-ae55-f4c8e97423fb req-bbcc536a-d46b-4715-8ee2-50693ec6b084 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032469 Oct 15 21:46:56.027066 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1940/6122] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:46:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:07.004190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=21 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:07.005406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:07.010688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:07.013013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:07.027157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:07.027653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:07.028089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:07.028808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:07.029288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:07.029719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:07.060733 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:07.066397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Old instance type m1.heat2, new instance type m1.heat {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:48:07.073509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:48:07.073963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:48:07.074436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:48:07.074854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:48:07.075269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:48:07.141440 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [e8f8a3fb-35ec-40da-86d5-324eb17c0caa] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:48:08.148765 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [e8f8a3fb-35ec-40da-86d5-324eb17c0caa] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 44196. Oct 15 21:48:08.152231 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a8776471-0319-4457-92b6-19ea89aab207 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee/action" status: 202 len: 0 microversion: 2.1 time: 1.150501 Oct 15 21:48:08.152848 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4183/6123] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:48:07 2022] POST /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee/action => generated 0 bytes in 1152 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:48:08.159904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=32,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:08.161194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:08.166445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:08.167720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:08.179065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:08.179575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:08.181162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:08.181921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:08.182437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:08.182916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:08.388730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:08.397377 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e1d38a11-3fe8-4215-a294-60c2902cdc85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1679 microversion: 2.1 time: 0.240117 Oct 15 21:48:08.398240 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1941/6124] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:08 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1679 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:48:09.427150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:09.427878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:09.434968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:09.435633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:09.436146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:09.436927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:09.437480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:09.438058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:09.683659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:09.691564 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a5a8bb1-a32c-4ba1-baf1-f4f45d47305d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1679 microversion: 2.1 time: 0.273002 Oct 15 21:48:09.692149 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4184/6125] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:09 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1679 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:48:10.714914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:10.715561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:10.724872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:10.725420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:10.725888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:10.727426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:10.727583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:10.728037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:10.858021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:10.865878 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2fb4a748-d275-41f2-b6e4-69dc64785d76 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.154953 Oct 15 21:48:10.866440 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1942/6126] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:10 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:11.890497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:11.891108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:11.898048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:11.898541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:11.899033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:11.899770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:11.900229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:11.900658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:12.029555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:12.037299 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4d60b1fe-f4e9-4aaa-b539-42049e0d4689 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.150961 Oct 15 21:48:12.037874 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4185/6127] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:11 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:13.061709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:13.062348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:13.071485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:13.071995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:13.072459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:13.073212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:13.073704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:13.074160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:13.209140 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:13.217745 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a3e574ae-285a-486b-b064-b5ac93c723ca tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.159962 Oct 15 21:48:13.218315 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1943/6128] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:13 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:14.242555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:14.243195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:14.250459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:14.250949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:14.251392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:14.252096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:14.252557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:14.252996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:14.385468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:14.393694 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-76a35b23-0d9d-443d-b4cc-38db94520069 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.155361 Oct 15 21:48:14.394266 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4186/6129] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:14 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:15.418126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:15.418879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:15.427942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:15.428443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:15.428905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:15.430251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:15.430721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:15.431162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:15.864846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:15.873584 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3b3752f2-ad63-4ce0-a350-d49a75273bb3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.459591 Oct 15 21:48:15.874147 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1944/6130] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:15 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:16.897011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:16.897657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:16.905181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:16.905695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:16.906132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:16.906825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:16.907281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:16.907707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:17.040408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:17.048225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-454e073f-d894-4559-ac8b-c1cadc2ff7f3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.155155 Oct 15 21:48:17.048806 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4187/6131] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:16 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:18.072214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:18.072858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:18.082045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:18.082486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:18.082943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:18.083660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:18.084138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:18.084583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:18.220727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:18.229297 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c97c3215-4381-437e-8368-f4a4640c17f5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.161421 Oct 15 21:48:18.229882 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1945/6132] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:18 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:19.253835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:19.254463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:19.261980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:19.262473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:19.262919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:19.263625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:19.264093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:19.264530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:19.397078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:19.404957 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f036e106-7ac5-4a4c-8d53-9587f5a3b178 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.155066 Oct 15 21:48:19.405549 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4188/6133] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:19 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:20.428165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:20.428809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:20.435845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:20.436357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:20.436823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:20.437583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:20.438068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:20.438520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:20.577902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:20.586360 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3f32906e-608e-4c53-a4e9-76d1d176f7c1 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.161693 Oct 15 21:48:20.587141 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1946/6134] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:20 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:21.610375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:21.610991 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:21.618022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:21.618506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:21.618945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:21.619638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:21.620100 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:21.620528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:21.750434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:21.758703 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-be425a26-0930-4892-bb26-c1051a51e0f4 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1684 microversion: 2.1 time: 0.152214 Oct 15 21:48:21.759273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4189/6135] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:21 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1684 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:22.522441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "name": "network-vif-unplugged", "status": "completed", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:22.534484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:22.535111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:22.535596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:22.548940 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] Creating event network-vif-unplugged:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch1-master-1 Oct 15 21:48:22.555466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:48:22.566428 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-650e3848-f625-4375-b57c-6b18eb010674 req-5d6ae5da-3488-4d2e-ba6d-520497899e3b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047708 Oct 15 21:48:22.567597 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1947/6136] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:22 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:22.782998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:22.783913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:22.793994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:22.794500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:22.794950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:22.796276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:22.796276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:22.796557 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:23.004574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:23.012806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b4a227c-153d-45b5-8cd4-f69586064be7 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1683 microversion: 2.1 time: 0.233529 Oct 15 21:48:23.013429 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4190/6137] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:22 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1683 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:24.039896 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:24.040629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:24.046920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:24.047427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:24.047881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:24.048597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:24.049097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:24.049549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:24.181512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:24.189413 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a7d6e881-3199-48cb-9f8a-033480023335 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1681 microversion: 2.1 time: 0.155266 Oct 15 21:48:24.190003 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1948/6138] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:24 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:24.581806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:24.592051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:24.592532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:24.592964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:24.604617 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] Creating event network-changed:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch1-master-1 Oct 15 21:48:24.609932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:48:24.621204 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-61f9c1be-cfe5-4ade-8fd9-c8ab391189f2 req-6b5c3274-a2d3-4da7-92d8-ab098687f356 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043172 Oct 15 21:48:24.621796 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4191/6139] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:25.214116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:25.214786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:25.221783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.222289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.222749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.223477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.223957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.224407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.531343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:25.539376 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0cd1e6e1-a3e8-40e5-8084-cbef700e2e1c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1678 microversion: 2.1 time: 0.329346 Oct 15 21:48:25.539957 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1949/6140] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:25 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1678 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:25.563281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:25.563900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:25.571190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.572001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.572444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.573177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.573640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.574072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.613579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.614235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.614491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.615234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.615701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.616130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.641649 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:48:25.646727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:48:25.748471 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e641c2dc-4a2a-4512-8a76-d95199fd4633 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee/action" status: 202 len: 0 microversion: 2.1 time: 0.189450 Oct 15 21:48:25.753128 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4192/6141] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:48:25 2022] POST /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee/action => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:48:25.757931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:25.758579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:25.765129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.765795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.766323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.767192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:25.767682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:25.768138 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:25.969550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:25.978474 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-71468e36-dfb3-4e80-a42f-2734033ed11c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.224606 Oct 15 21:48:25.979381 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1950/6142] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:25 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:27.003828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:27.004531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:27.012390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:27.012873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:27.013335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:27.014028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:27.014485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:27.015159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:27.146332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:27.154763 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-92a90635-9f2f-4f37-ac1a-ec8f51e9e194 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.156116 Oct 15 21:48:27.155329 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4193/6143] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:26 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:28.178301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:28.179440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:28.187201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:28.187719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:28.188178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:28.188907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:28.189926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:28.190585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:28.282442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "name": "network-vif-plugged", "status": "completed", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:28.296041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:28.296761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:28.297442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:28.311591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] Creating event network-vif-plugged:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch1-master-1 Oct 15 21:48:28.318358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:48:28.329935 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ee6f3768-a1a9-41d9-b3f5-728944c4f19d req-e438f65f-95d8-40e1-b968-c49999e2bc2b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051530 Oct 15 21:48:28.330554 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4194/6144] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:28.362290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:28.371566 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-40de8599-38e3-4cb0-b263-cf181529c3b5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.197167 Oct 15 21:48:28.372143 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1951/6145] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:28 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:28.884613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "name": "network-vif-unplugged", "status": "completed", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:28.895283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:28.895876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:28.896372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:28.909259 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] Creating event network-vif-unplugged:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:48:28.915261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:48:28.926624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-a0a3efe2-5fec-41ae-a892-519905d21295 req-77782584-6532-4402-b86f-38b89dfe339e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046712 Oct 15 21:48:28.927240 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4195/6146] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:28 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:48:29.396307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:29.397150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:29.405827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:29.406336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:29.406793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:29.407520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:29.407988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:29.408434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:29.612882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:29.621576 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d6de2a74-22be-4271-b83f-805953403039 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.229183 Oct 15 21:48:29.622157 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1952/6147] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:29 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:30.647130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:30.648022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:30.654959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:30.655443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:30.655886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:30.656581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:30.657066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:30.657493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:30.661560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:30.800319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:30.808603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-33a1ad59-2ec3-4fa2-ba58-cdeda772d87b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.165688 Oct 15 21:48:30.809200 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4196/6148] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:30 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:30.968887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:30.979754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:30.980240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:30.980679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:30.993083 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] Creating event network-changed:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:48:30.998278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:48:31.008894 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-de5cf5c4-493a-4251-9d34-f30b9466cab3 req-764d2af7-8323-43fe-9c44-fe8ea2eaa3bb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047052 Oct 15 21:48:31.009520 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1953/6149] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:31.832686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:31.833342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:31.840333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:31.840823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:31.841295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:31.841998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:31.842459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:31.842888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:31.976975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:31.984785 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2243d5a0-bd0e-40bc-8d49-3124012f2307 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.156945 Oct 15 21:48:31.985380 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4197/6150] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:31 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:33.010903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:33.011940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:33.018562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:33.019061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:33.019684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:33.020585 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:33.021313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:33.021950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:33.280681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:33.290351 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1d72b424-cd44-43bf-bf0c-c16d79f5e535 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1677 microversion: 2.1 time: 0.285194 Oct 15 21:48:33.291206 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1954/6151] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:33 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1677 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:34.313861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:34.314465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:34.321698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.322199 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.322645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.323353 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.323820 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.324385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.455541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:34.463857 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ab14f7d8-196a-449f-a5d8-72ef605dc4fd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1670 microversion: 2.1 time: 0.153892 Oct 15 21:48:34.464428 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4198/6152] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:34 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1670 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:34.487515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:34.488148 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:34.494784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.495289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.495742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.496451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.496920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.497385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.625285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:34.633137 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ed7ef958-9fe3-41d9-b677-7a61bb16f63d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1670 microversion: 2.1 time: 0.149789 Oct 15 21:48:34.633701 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1955/6153] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:34 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1670 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:34.655909 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:34.656558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:34.664290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.664788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.665271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.665987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.666454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.666917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.802149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:34.810507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-086a822a-6341-4be7-9452-a43aea901ac9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1670 microversion: 2.1 time: 0.158775 Oct 15 21:48:34.811074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4199/6154] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:34 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1670 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:34.832914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:34.833465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:34.839612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.840102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.840558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.841307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.841783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.842231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.869524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:48:34.915060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:34.948975 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4c0870a8-1a89-4bc2-a083-2b2ea41ba33a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 204 len: 0 microversion: 2.1 time: 0.119547 Oct 15 21:48:34.949651 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1956/6155] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:48:34 2022] DELETE /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:48:34.958187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:34.958810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:34.965698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.966191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.966664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:34.967333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:34.967798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:34.968233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:35.100036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:35.110281 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3c965780-5c54-48a8-867c-93e4d72a67fc tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1676 microversion: 2.1 time: 0.155725 Oct 15 21:48:35.111339 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4200/6156] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:34 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1676 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:35.224042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-293a0216-d195-43dc-8ba3-e6b92460d4c7 req-a8887ccc-6f06-43c5-8a88-d68299b2da7e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "name": "network-vif-plugged", "status": "completed", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:35.234963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-293a0216-d195-43dc-8ba3-e6b92460d4c7 req-a8887ccc-6f06-43c5-8a88-d68299b2da7e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:35.235472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-293a0216-d195-43dc-8ba3-e6b92460d4c7 req-a8887ccc-6f06-43c5-8a88-d68299b2da7e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:35.235928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-293a0216-d195-43dc-8ba3-e6b92460d4c7 req-a8887ccc-6f06-43c5-8a88-d68299b2da7e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:35.247776 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-293a0216-d195-43dc-8ba3-e6b92460d4c7 req-a8887ccc-6f06-43c5-8a88-d68299b2da7e service nova] Creating event network-vif-plugged:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:48:35.252731 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-293a0216-d195-43dc-8ba3-e6b92460d4c7 req-a8887ccc-6f06-43c5-8a88-d68299b2da7e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032856 Oct 15 21:48:35.253383 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1957/6157] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:36.135112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:36.135850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:36.138823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:36.144902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:36.145459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:36.145902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:36.146624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:36.147086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:36.148180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:36.284192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:36.293265 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ee9ca87e-41cd-43e5-bc84-57d0fcc12f70 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1676 microversion: 2.1 time: 0.162829 Oct 15 21:48:36.293855 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4201/6158] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:36 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1676 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:37.318158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:37.318979 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:37.327426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:37.328025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:37.328552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:37.329764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:37.329887 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:37.330366 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:37.410062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:37.418113 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0d94661b-4182-4674-a9e5-0639770fce8b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 200 len: 1608 microversion: 2.1 time: 0.105064 Oct 15 21:48:37.418729 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1958/6159] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:37 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 1608 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:37.526509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d2177ebb-faf6-4f20-90d6-f5fa06c8f726 req-11c2a3f9-3858-458e-8101-7aa985371314 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbf96a37-4ef8-4ad6-ab74-2972d0b214ee", "name": "network-vif-deleted", "tag": "4698d520-d68d-476c-ba5f-373ebd744a32"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:37.532721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:37.539481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2177ebb-faf6-4f20-90d6-f5fa06c8f726 req-11c2a3f9-3858-458e-8101-7aa985371314 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:37.539988 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2177ebb-faf6-4f20-90d6-f5fa06c8f726 req-11c2a3f9-3858-458e-8101-7aa985371314 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:37.540431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2177ebb-faf6-4f20-90d6-f5fa06c8f726 req-11c2a3f9-3858-458e-8101-7aa985371314 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:37.555244 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d2177ebb-faf6-4f20-90d6-f5fa06c8f726 req-11c2a3f9-3858-458e-8101-7aa985371314 service nova] Creating event network-vif-deleted:4698d520-d68d-476c-ba5f-373ebd744a32 for instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee on nch2-master-1 Oct 15 21:48:37.560691 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d2177ebb-faf6-4f20-90d6-f5fa06c8f726 req-11c2a3f9-3858-458e-8101-7aa985371314 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037929 Oct 15 21:48:37.561396 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4202/6160] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:38.443013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:38.443586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:38.447123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:38.452826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:38.453401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:38.453861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:38.454590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:38.455062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:38.455507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:38.470515 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee HTTP/1.1 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'fbf96a37-4ef8-4ad6-ab74-2972d0b214ee'} Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:48:38.472209 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ebb9175-de83-4108-8c16-429b1d7d3262 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 404 len: 111 microversion: 2.1 time: 0.034836 Oct 15 21:48:38.478014 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1959/6161] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:38 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:48:38.491485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:38.492209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:38.498753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:38.499245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:38.499690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:38.500405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:38.500875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:38.501331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:38.516686 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee HTTP/1.1 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'fbf96a37-4ef8-4ad6-ab74-2972d0b214ee'} Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:48:38.518400 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09067552-4a6b-4127-aad5-e6ccb9f2728e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 404 len: 111 microversion: 2.1 time: 0.033011 Oct 15 21:48:38.528458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4203/6162] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:38 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:48:38.572535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1552185470", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "7ec4edb6-cfee-4995-af86-9b425fd0a6bb"}], "security_groups": [{"name": "tempest-securitygroup--1671305992"}], "key_name": "tempest-keypair-350156625", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:38.681175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] validate_networks() for [('7ec4edb6-cfee-4995-af86-9b425fd0a6bb', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:48:39.065395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:48:39.065842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:48:39.066382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:48:39.066808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:48:39.067268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:48:39.079261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:39.079757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:39.080208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:39.080950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:39.081448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:39.081890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:39.103467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:48:39.108879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:48:39.123256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:48:39.125697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:48:39.176801 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b33d8c34-bb8b-4fad-9861-f8bf8ac1ca36 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.608618 Oct 15 21:48:39.177482 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1960/6163] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:48:38 2022] POST /compute/v2.1/servers => generated 406 bytes in 610 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:48:39.188883 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-46fd9a4f-85a3-4924-a177-19f5b1bfeced tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:39.189970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-46fd9a4f-85a3-4924-a177-19f5b1bfeced tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:39.256306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-46fd9a4f-85a3-4924-a177-19f5b1bfeced tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:39.267679 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-46fd9a4f-85a3-4924-a177-19f5b1bfeced tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1238 microversion: 2.1 time: 0.082762 Oct 15 21:48:39.268322 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4204/6164] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:39 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1238 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:40.293899 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:40.294541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:40.301634 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:40.302349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:40.302807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:40.304052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:40.304951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:40.305568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:40.378663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:40.387067 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b62d8105-2768-458a-a246-c9311e5882ee tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1296 microversion: 2.1 time: 0.099094 Oct 15 21:48:40.387637 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1961/6165] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:40 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1296 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:41.411945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:41.412680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:41.420960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:41.421496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:41.421940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:41.422654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:41.423120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:41.423554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:41.629252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:41.637345 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3bddbb1-2bc9-465d-a853-f7810c161f85 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1364 microversion: 2.1 time: 0.229299 Oct 15 21:48:41.637946 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4205/6166] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:41 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1364 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:41.675095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-796da499-b9ee-4422-aa93-45562e498a08 req-db150ea7-591b-4d48-b891-00f938d7f37e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:41.687058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-796da499-b9ee-4422-aa93-45562e498a08 req-db150ea7-591b-4d48-b891-00f938d7f37e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:41.687644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-796da499-b9ee-4422-aa93-45562e498a08 req-db150ea7-591b-4d48-b891-00f938d7f37e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:41.688164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-796da499-b9ee-4422-aa93-45562e498a08 req-db150ea7-591b-4d48-b891-00f938d7f37e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:41.701723 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-796da499-b9ee-4422-aa93-45562e498a08 req-db150ea7-591b-4d48-b891-00f938d7f37e service nova] Creating event network-changed:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:48:41.706810 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-796da499-b9ee-4422-aa93-45562e498a08 req-db150ea7-591b-4d48-b891-00f938d7f37e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035353 Oct 15 21:48:41.707524 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1962/6167] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:42.708312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:42.708962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:42.715186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:42.715672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:42.716131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:42.716804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:42.717289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:42.717718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:42.849837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:42.858130 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0d1a482-ff1d-4e8e-84b2-d9b879e3bea9 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1364 microversion: 2.1 time: 0.200300 Oct 15 21:48:42.858696 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4206/6168] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:42 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1364 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:43.883656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:43.884384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:43.892461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:43.892986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:43.893460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:43.894182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:43.894651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:43.895091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:44.369171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:44.377182 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1ba3c9c9-9109-4f57-81fb-5bef10b787eb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1364 microversion: 2.1 time: 0.498436 Oct 15 21:48:44.378072 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1963/6169] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:43 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1364 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:45.402945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:45.403605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:45.411060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:45.411563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:45.412014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:45.413379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:45.413886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:45.414328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:45.545332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:45.553030 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-45dd3bdb-b1a4-4634-a1e2-5abb58647ae3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1364 microversion: 2.1 time: 0.154070 Oct 15 21:48:45.553713 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4207/6170] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:45 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1364 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:46.579095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:46.579630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:46.586612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:46.587109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:46.587587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:46.588294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:46.588765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:46.589244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:46.722188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:46.730409 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-89ee74bd-10ad-4382-91ee-4d3702ea7256 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1364 microversion: 2.1 time: 0.155353 Oct 15 21:48:46.730977 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1964/6171] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:46 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:47.289915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7d172590-432a-491b-9fee-fa60c6ae89af req-36e3d3dc-5f65-4eae-b50d-6bd87632d4c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "name": "network-vif-plugged", "status": "completed", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:47.301071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d172590-432a-491b-9fee-fa60c6ae89af req-36e3d3dc-5f65-4eae-b50d-6bd87632d4c6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:47.301807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d172590-432a-491b-9fee-fa60c6ae89af req-36e3d3dc-5f65-4eae-b50d-6bd87632d4c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:47.302238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7d172590-432a-491b-9fee-fa60c6ae89af req-36e3d3dc-5f65-4eae-b50d-6bd87632d4c6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:47.313889 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7d172590-432a-491b-9fee-fa60c6ae89af req-36e3d3dc-5f65-4eae-b50d-6bd87632d4c6 service nova] Creating event network-vif-plugged:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:48:47.319574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7d172590-432a-491b-9fee-fa60c6ae89af req-36e3d3dc-5f65-4eae-b50d-6bd87632d4c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033369 Oct 15 21:48:47.320185 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4208/6172] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:48:47.754470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:47.755110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:47.762685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:47.763195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:47.763656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:47.764383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:47.765141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:47.765610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:47.895675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:47.904316 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6ccf8a08-3f24-4034-ae3b-962c4527d130 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1364 microversion: 2.1 time: 0.153829 Oct 15 21:48:47.904898 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1965/6173] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:47 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1364 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:48.929805 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:48.930528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:48.936634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:48.937158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:48.937602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:48.938295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:48.938750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:48.939176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:49.075086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:48:49.083115 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2841231-9cdc-4602-8502-d4d20c278d20 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1551 microversion: 2.1 time: 0.158753 Oct 15 21:48:49.083699 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4209/6174] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:48:48 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1551 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:48:49.107792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:48:49.108459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:48:49.111350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:49.116307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:49.118476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:49.119116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:49.119569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:49.120319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:49.120819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:49.121303 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:49.200848 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fd22de83-8702-4c2f-90ca-b581756ed5b0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096850 Oct 15 21:48:49.201601 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1966/6175] 10.222.0.16 () {58 vars in 1275 bytes} [Sat Oct 15 21:48:49 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:48:50.140726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-60ff2cae-dc6e-418e-a55f-1826a6e0cba2 req-92a97e3c-69c1-43c3-9c29-8867753ace1a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:48:50.145731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:48:50.152744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-60ff2cae-dc6e-418e-a55f-1826a6e0cba2 req-92a97e3c-69c1-43c3-9c29-8867753ace1a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:48:50.153288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-60ff2cae-dc6e-418e-a55f-1826a6e0cba2 req-92a97e3c-69c1-43c3-9c29-8867753ace1a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:48:50.153711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-60ff2cae-dc6e-418e-a55f-1826a6e0cba2 req-92a97e3c-69c1-43c3-9c29-8867753ace1a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:48:50.166216 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-60ff2cae-dc6e-418e-a55f-1826a6e0cba2 req-92a97e3c-69c1-43c3-9c29-8867753ace1a service nova] Creating event network-changed:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:48:50.171209 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-60ff2cae-dc6e-418e-a55f-1826a6e0cba2 req-92a97e3c-69c1-43c3-9c29-8867753ace1a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034758 Oct 15 21:48:50.171815 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4210/6176] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:48:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:03.522865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:03.523865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=43,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:03.525294 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:50:03.525444 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:50:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c (ip 10.222.0.16) !!! Oct 15 21:50:03.531223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:03.531851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:03.541712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:03.542219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:03.542682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:03.543404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:03.543884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:03.544334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:03.677961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:03.686397 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-755a613d-59ba-4a14-8474-7178b725bfa0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1670 microversion: 2.1 time: 0.166201 Oct 15 21:50:03.686975 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1967/6177] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:03 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1670 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:03.710182 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=19 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:03.711541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=66 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:03.712692 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:50:03.712834 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:50:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-volume_attachments (ip 10.222.0.16) !!! Oct 15 21:50:03.726367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "13e9cc03-4f28-4783-a679-6ada7ad887fc"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:03.728692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:03.735685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:03.736175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:03.736612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:03.737335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:03.737795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:03.738222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:03.978036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-459f06e8-2f2d-4bd5-a623-be55fcc36d4d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.270808 Oct 15 21:50:03.978587 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4211/6178] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:50:03 2022] POST /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-volume_attachments => generated 194 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:07.284989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:07.287327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:07.294198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:07.294704 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:07.295163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:07.295879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:07.296353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:07.296799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:07.325217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:07.330874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Old instance type m1.heat2, new instance type m1.heat {{(pid=138689) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 21:50:07.338008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:50:07.338450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:50:07.338943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:50:07.339370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:50:07.339804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:50:07.405586 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8beb8532-f6c6-42a5-850c-f498af5d09a2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/action" status: 202 len: 0 microversion: 2.1 time: 0.124429 Oct 15 21:50:07.406199 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1968/6179] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:50:07 2022] POST /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/action => generated 0 bytes in 126 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:50:07.417218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:07.418359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:07.424662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:07.425180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:07.425611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:07.426309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:07.426767 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:07.427198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:07.574051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:07.582856 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-99e58c5b-af30-486c-a2c2-d0a875899342 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1725 microversion: 2.1 time: 0.172104 Oct 15 21:50:07.583448 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4212/6180] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:07 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1725 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:08.609022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:08.609683 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:08.616796 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:08.617334 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:08.618472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:08.619194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:08.619676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:08.620153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:08.786945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:08.796113 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fadb6edd-a43a-4d90-a668-fe5ce8b5a20b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1725 microversion: 2.1 time: 0.191870 Oct 15 21:50:08.800120 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1969/6181] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:08 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1725 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:09.819973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:09.820796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:09.827773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:09.828266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:09.828716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:09.829458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:09.829951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:09.830384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:09.964235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:09.972486 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7ac77945-9248-44c5-b415-2f273b4e7814 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.156479 Oct 15 21:50:09.973073 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4213/6182] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:09 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:10.997112 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:10.997759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:11.004681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:11.005233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:11.005697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:11.006421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:11.006905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:11.007354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:11.137687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:11.146692 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-09b00de3-d0d4-44f1-95d5-9a657af32166 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.153899 Oct 15 21:50:11.147272 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1970/6183] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:10 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:12.171024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:12.171650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:12.178734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:12.179229 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:12.180308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:12.181019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:12.181505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:12.181936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:12.316492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:12.324899 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fd77023-5bb7-4dc6-983d-51223b114cba tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.157860 Oct 15 21:50:12.325507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4214/6184] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:12 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:13.348657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:13.349321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:13.356276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:13.356757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:13.357243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:13.357954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:13.358427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:13.358867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:13.493753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:13.502061 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-61e7ac7b-8bfb-4a3b-b72a-e141a9703f61 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.157487 Oct 15 21:50:13.502664 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1971/6185] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:13 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:14.526576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:14.527112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:14.534209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:14.534701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:14.535144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:14.535865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:14.536311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:14.536745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:14.665062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:14.674009 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a59ad98-ee61-4e74-9982-0a0b8a8faa40 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.151925 Oct 15 21:50:14.674588 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4215/6186] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:14 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:15.698824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:15.699488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:15.706423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:15.706927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:15.707379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:15.708085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:15.708555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:15.708997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:15.843030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:15.851431 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4b291f2b-2f68-43cb-a741-3f569558c494 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.157231 Oct 15 21:50:15.852015 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1972/6187] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:15 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:16.876075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:16.876692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:16.883842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:16.884329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:16.884770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:16.885508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:16.885965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:16.886392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:17.014257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:17.022707 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0dbb7316-6535-4eca-87f4-009c48579b8c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.150599 Oct 15 21:50:17.023273 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4216/6188] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:16 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:18.047191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:18.047827 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:18.055135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:18.055643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:18.056099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:18.056811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:18.057861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:18.058315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:18.196296 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:18.205117 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-42a34484-a529-4d05-8361-6b718af80f18 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.161853 Oct 15 21:50:18.205694 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1973/6189] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:18 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:19.230367 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:19.230984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:19.237445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:19.237934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:19.238378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:19.239069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:19.239522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:19.239949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:19.370902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:19.379183 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-40a89fb2-3d05-4f15-b46a-c9d3948488d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.153841 Oct 15 21:50:19.379759 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4217/6190] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:19 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:20.403883 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:20.404520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:20.411571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:20.412083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:20.412539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:20.413295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:20.413771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:20.414216 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:20.552516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:20.561002 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-95ccd6be-f0dd-482d-a1e8-7f950ce4be0b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.162169 Oct 15 21:50:20.561595 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1974/6191] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:20 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:21.585799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:21.586358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:21.594072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:21.594554 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:21.594999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:21.595696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:21.596156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:21.596591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:21.732218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:21.740480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-608b53c1-d00c-4271-998d-c2c28894e3dd tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.158711 Oct 15 21:50:21.741078 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4218/6192] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:21 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:22.763818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:22.764487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:22.771757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:22.772260 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:22.772724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:22.773474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:22.773955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:22.774404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:22.904096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:22.912695 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3277929c-31d6-4d51-ba8d-da7a98be4054 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.153503 Oct 15 21:50:22.913287 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1975/6193] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:22 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:23.935844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:23.936457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:23.943342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:23.943821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:23.944257 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:23.944953 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:23.945436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:23.945864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:24.383043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:24.392583 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0fbabc7d-467a-42fd-bcdf-62354cabb4a8 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.460624 Oct 15 21:50:24.393213 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4219/6194] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:23 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:25.418233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:25.418797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:25.425699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:25.426204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:25.426665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:25.427382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:25.427860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:25.428308 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:25.556911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:25.565147 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8907af57-2076-4485-b28f-f0cc96c22c29 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.151678 Oct 15 21:50:25.565741 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1976/6195] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:25 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:26.588496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:26.589216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:26.596150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:26.596639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:26.597116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:26.597806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:26.598274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:26.598705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:26.736884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:26.746766 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f5e0bde5-b6e3-4a2d-b87a-765fd23e6253 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1730 microversion: 2.1 time: 0.162300 Oct 15 21:50:26.747423 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4220/6196] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:26 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1730 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:27.398739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "name": "network-vif-unplugged", "status": "completed", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:27.410263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:27.410765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:27.411223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:27.423316 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] Creating event network-vif-unplugged:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch1-master-1 Oct 15 21:50:27.428708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:50:27.443741 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4e98e35b-f1eb-4fe1-82f3-306c152960cb req-7ff5bcca-3704-4dc8-b9f5-273a27f4b319 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.048662 Oct 15 21:50:27.444543 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1977/6197] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:27 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:27.771026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:27.771637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:27.779337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:27.779831 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:27.780270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:27.780971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:27.781470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:27.781895 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:28.057429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:28.065718 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dcdf108b-86f2-459d-879d-3aafc88571d3 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1729 microversion: 2.1 time: 0.298582 Oct 15 21:50:28.066291 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4221/6198] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:27 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1729 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:29.092732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:29.093481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:29.099686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:29.100186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:29.100637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:29.101371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:29.101849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:29.102290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:29.244166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:29.252896 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d01ddf8-a5df-4e52-9616-fb9d24bacd1a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1727 microversion: 2.1 time: 0.166183 Oct 15 21:50:29.253585 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1978/6199] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:29 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:50:29.482217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:29.493673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:29.494153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:29.494581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:29.506478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] Creating event network-changed:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch1-master-1 Oct 15 21:50:29.512562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:50:29.523594 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ec37d263-15ca-40ac-a82b-76281ab7281a req-098b1590-cee9-458e-a29e-91169c8061ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047277 Oct 15 21:50:29.524205 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4222/6200] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:30.277666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:30.278343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:30.285199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:30.285709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:30.286175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:30.286895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:30.287379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:30.287829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:30.462958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:30.471376 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-717015d9-deae-4921-99d2-d40d5ac7c6b2 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1727 microversion: 2.1 time: 0.197905 Oct 15 21:50:30.471962 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1979/6201] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:30 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1727 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:31.497749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:31.498388 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:31.504687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:31.505209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:31.505652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:31.506348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:31.506809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:31.507236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:31.638476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:31.646987 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71714d16-5c90-4617-bc86-bc1a7ab6b45b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1727 microversion: 2.1 time: 0.154608 Oct 15 21:50:31.647560 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4223/6202] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:31 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:32.671598 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:32.672223 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:32.679149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:32.679675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:32.680123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:32.680825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:32.681329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:32.682379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:32.810917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:32.819997 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ca7d9344-e5d3-4bc0-9f6b-78763c484973 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1724 microversion: 2.1 time: 0.152360 Oct 15 21:50:32.820575 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1980/6203] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:32 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:32.843167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:32.843707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:32.850505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:32.850987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:32.851429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:32.852117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:32.852577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:32.853004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:32.895613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:32.896104 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:32.896541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:32.897289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:32.897743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:32.898167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:32.919806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for project d42197537d0b407882fc410a5d359e1b. Resources: {'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:50:32.924957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Getting quotas for user 4165cc6b781b4df3abd6a227170d4e46 and project d42197537d0b407882fc410a5d359e1b. Resources: {'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:50:32.983769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:33.032018 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef62626e-90bb-4a59-b37b-8d26adfcb6c0 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "POST /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/action" status: 202 len: 0 microversion: 2.1 time: 0.192661 Oct 15 21:50:33.033074 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4224/6204] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:50:32 2022] POST /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/action => generated 0 bytes in 195 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:50:33.042324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:33.042992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:33.049235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:33.049748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:33.050211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:33.050939 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:33.051421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:33.051868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:33.264431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:33.275856 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ae2b7f8-226b-4dfc-820b-1530252a5ae5 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.237970 Oct 15 21:50:33.276801 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1981/6205] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:33 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:50:34.301076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:34.301781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:34.304590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:34.311080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:34.311581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:34.312033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:34.312749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:34.313251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:34.313691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:34.450087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:34.458505 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ddd5b7f2-c45e-4169-ad20-77f8d3a15840 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.161821 Oct 15 21:50:34.459082 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4225/6206] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:34 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:34.926761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "name": "network-vif-plugged", "status": "completed", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:34.932128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=35 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:34.938355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:34.938858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:34.939318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:34.942305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:34.955787 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] Creating event network-vif-plugged:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch1-master-1 Oct 15 21:50:34.962110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:50:34.974545 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-cb2b7a48-650d-434f-bbde-17dcda6e2e3c req-76750051-c509-4486-80c8-a7991e740e0f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051508 Oct 15 21:50:34.975157 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1982/6207] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:35.482926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:35.483773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:35.491155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:35.491822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:35.492268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:35.493092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:35.493568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:35.494565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:35.670546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:35.683131 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e48a838-7b94-4ee7-9b10-1828a70ca150 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.205025 Oct 15 21:50:35.683946 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4226/6208] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:35 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:36.709437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:36.710163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:36.717208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:36.717727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:36.718243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:36.718905 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:36.719382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:36.719830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:36.753808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "name": "network-vif-unplugged", "status": "completed", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:36.771793 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:36.772378 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:36.772885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:36.785985 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] Creating event network-vif-unplugged:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:50:36.791997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:50:36.803642 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-83f3dd57-9256-41cb-acba-cb0a1395b13a req-48d860ed-5803-4ded-86fc-18537ec594b7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.055547 Oct 15 21:50:36.804533 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4227/6209] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:36.947740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:36.957429 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-27b20196-0872-4818-aa75-91c2f1ba7486 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.253120 Oct 15 21:50:36.958303 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1983/6210] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:36 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:37.982973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:37.983688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:37.990122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:37.990594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:37.991037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:37.991742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:37.992207 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:37.992639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:38.126320 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:38.135365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bca0b836-d8f6-49ff-bb61-bd9d9c4a4d94 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.156936 Oct 15 21:50:38.135973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4228/6211] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:37 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:38.826187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:38.836692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:38.837336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:38.838355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:38.850069 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] Creating event network-changed:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:50:38.855377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 21:50:38.866087 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-34949b50-c36d-4446-9d03-0692178a2d34 req-7152f031-02ae-4f9c-ac5b-9aba5cd27b1a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044330 Oct 15 21:50:38.866672 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1984/6212] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:39.159021 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:39.159640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:39.166662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:39.167186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:39.167636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:39.168382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:39.168958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:39.169293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:39.692082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:39.700530 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0de9ec58-4027-4c23-bd1a-fdda46b529c6 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.545385 Oct 15 21:50:39.701170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4229/6213] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:39 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:40.727094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:40.727721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:40.734546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:40.735166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:40.735630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:40.736348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:40.736817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:40.737304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:40.935178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:40.943890 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d017dbff-7a25-4495-9133-e3125bae5671 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.222460 Oct 15 21:50:40.945108 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1985/6214] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:40 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:41.968919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:41.969573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:41.976389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:41.976879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:41.977359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:41.978059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:41.978520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:41.978952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:42.109844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:42.118091 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f72fcf29-8467-4052-bf06-7b7f5d01bb01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1723 microversion: 2.1 time: 0.153083 Oct 15 21:50:42.118682 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4230/6215] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:41 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:43.141660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:43.142289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:43.149146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.149679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.150134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.150842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.152230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.152987 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.288169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:43.297430 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b9772eae-7ea9-4cd5-b5b4-6a5086be555e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1716 microversion: 2.1 time: 0.159880 Oct 15 21:50:43.298009 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1986/6216] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:43 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1716 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:43.319886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:43.320504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:43.322617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:43.329280 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.329853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.330334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.331054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.331525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.332012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.468990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:43.477365 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2407a293-2c12-430f-86a7-4923fbd3813a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1716 microversion: 2.1 time: 0.161313 Oct 15 21:50:43.477943 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4231/6217] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:43 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1716 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:43.500805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:43.501466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:43.507483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.507989 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.508450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.509192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.509678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.510130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.642205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:43.650656 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-63a15393-f84a-4c0a-be29-558054514e93 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1716 microversion: 2.1 time: 0.154171 Oct 15 21:50:43.651802 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1987/6218] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:43 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1716 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:43.673633 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:43.674052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:43.680172 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.680697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.681167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.681874 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.682354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.682785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.708604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:50:43.788835 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21684c3e-1034-409e-a400-f4d0fd114d92 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 204 len: 0 microversion: 2.1 time: 0.118844 Oct 15 21:50:43.789741 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4232/6219] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:50:43 2022] DELETE /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:50:43.799510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:43.800473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:43.806520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.807034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.808038 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.808915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:43.809434 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:43.809889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:43.945372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:43.954771 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-58005beb-2680-46a1-acd7-3245fe054c01 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1722 microversion: 2.1 time: 0.159920 Oct 15 21:50:43.955353 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1988/6220] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:43 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:50:44.980155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:44.980927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:44.987745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:44.988233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:44.988706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:44.989433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:44.989897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:44.990328 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:45.122911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:45.131227 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d8c75462-4f66-49df-ba61-651321c0cb10 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1722 microversion: 2.1 time: 0.155963 Oct 15 21:50:45.131802 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4233/6221] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:44 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:45.804338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e4af97cb-0081-4643-bb67-31fc26bac40e req-117ee10c-9f4a-47df-942f-32b5f2403e1d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "name": "network-vif-plugged", "status": "completed", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:45.815635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e4af97cb-0081-4643-bb67-31fc26bac40e req-117ee10c-9f4a-47df-942f-32b5f2403e1d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:45.816131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e4af97cb-0081-4643-bb67-31fc26bac40e req-117ee10c-9f4a-47df-942f-32b5f2403e1d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:45.816781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e4af97cb-0081-4643-bb67-31fc26bac40e req-117ee10c-9f4a-47df-942f-32b5f2403e1d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:45.829243 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e4af97cb-0081-4643-bb67-31fc26bac40e req-117ee10c-9f4a-47df-942f-32b5f2403e1d service nova] Creating event network-vif-plugged:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:50:45.834239 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e4af97cb-0081-4643-bb67-31fc26bac40e req-117ee10c-9f4a-47df-942f-32b5f2403e1d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034042 Oct 15 21:50:45.835254 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1989/6222] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:46.156540 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:46.158055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:46.165640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:46.166427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:46.167525 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:46.168473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:46.169150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:46.169595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:46.256256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4165cc6b781b4df3abd6a227170d4e46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd42197537d0b407882fc410a5d359e1b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:46.268016 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ad4f8431-0ed8-441a-a912-c38b4e4ddf2e tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 200 len: 1654 microversion: 2.1 time: 0.116272 Oct 15 21:50:46.269094 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4234/6223] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:46 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 1654 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:50:46.433946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e2f48de0-a7f1-4c8d-969f-006b8f1e409f req-e5f05ba4-889a-4d86-9597-6351e5c88e85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b1a6a116-a85f-4080-8ac6-7bdfb84a426c", "name": "network-vif-deleted", "tag": "fd90f1fc-e550-461f-96f0-c557cff06229"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:46.445802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e2f48de0-a7f1-4c8d-969f-006b8f1e409f req-e5f05ba4-889a-4d86-9597-6351e5c88e85 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:46.446318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e2f48de0-a7f1-4c8d-969f-006b8f1e409f req-e5f05ba4-889a-4d86-9597-6351e5c88e85 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:46.446769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e2f48de0-a7f1-4c8d-969f-006b8f1e409f req-e5f05ba4-889a-4d86-9597-6351e5c88e85 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:46.459526 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e2f48de0-a7f1-4c8d-969f-006b8f1e409f req-e5f05ba4-889a-4d86-9597-6351e5c88e85 service nova] Creating event network-vif-deleted:fd90f1fc-e550-461f-96f0-c557cff06229 for instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c on nch2-master-1 Oct 15 21:50:46.464778 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e2f48de0-a7f1-4c8d-969f-006b8f1e409f req-e5f05ba4-889a-4d86-9597-6351e5c88e85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036050 Oct 15 21:50:46.465443 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1990/6224] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:50:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:50:47.294379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:47.294996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:47.302284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:47.302770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:47.303209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:47.303916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:47.304372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:47.304797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:47.318979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c HTTP/1.1 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b1a6a116-a85f-4080-8ac6-7bdfb84a426c'} Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:47.320667 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:47.326261 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-721ab289-e599-46be-a23f-4bd71710da6a tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 404 len: 111 microversion: 2.1 time: 0.031629 Oct 15 21:50:47.327027 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4235/6225] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:47 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:47.391539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:47.393870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:47.400237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:47.400779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:47.402104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:47.402915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:47.403399 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:47.403865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:47.418004 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-volume_attachments HTTP/1.1 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'server_id': 'b1a6a116-a85f-4080-8ac6-7bdfb84a426c'} Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:47.419773 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:47.424169 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7df591a6-911a-40de-8d15-5e91d7ea16ad tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.036064 Oct 15 21:50:47.425281 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1991/6226] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:50:47 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c/os-volume_attachments => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.638157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.638644 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.645521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.646011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.646454 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.647150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.647613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.648044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.663323 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c HTTP/1.1 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'b1a6a116-a85f-4080-8ac6-7bdfb84a426c'} Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.664878 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a9ff710f-952a-4d4c-9435-81a87e61d254 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 404 len: 111 microversion: 2.1 time: 0.033149 Oct 15 21:50:48.669851 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4236/6227] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:50:48 2022] DELETE /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.674918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.675579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: b1a6a116-a85f-4080-8ac6-7bdfb84a426c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.682491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.683270 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.683956 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.684731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.685246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.685692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.699760 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c HTTP/1.1 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'b1a6a116-a85f-4080-8ac6-7bdfb84a426c'} Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.701351 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance b1a6a116-a85f-4080-8ac6-7bdfb84a426c could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6f143fa7-82b6-42fb-affb-92dba13a7031 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c" status: 404 len: 111 microversion: 2.1 time: 0.031795 Oct 15 21:50:48.706531 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1992/6228] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:48 2022] GET /compute/v2.1/servers/b1a6a116-a85f-4080-8ac6-7bdfb84a426c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.711253 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.711763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.718571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.719059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.719504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.720200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.720695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.721158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.737730 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee HTTP/1.1 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'fbf96a37-4ef8-4ad6-ab74-2972d0b214ee'} Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.739269 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de078f11-5830-4905-841f-efe0b8058adb tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 404 len: 111 microversion: 2.1 time: 0.033299 Oct 15 21:50:48.744093 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4237/6229] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:50:48 2022] DELETE /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.749131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.749783 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: fbf96a37-4ef8-4ad6-ab74-2972d0b214ee] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.755668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.756168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.756621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.757351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.757824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.758267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.772768 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee HTTP/1.1 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'fbf96a37-4ef8-4ad6-ab74-2972d0b214ee'} Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.774394 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance fbf96a37-4ef8-4ad6-ab74-2972d0b214ee could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d3d40d27-ecc1-490f-bdd3-edebd2a7e884 tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee" status: 404 len: 111 microversion: 2.1 time: 0.030562 Oct 15 21:50:48.779742 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1993/6230] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:48 2022] GET /compute/v2.1/servers/fbf96a37-4ef8-4ad6-ab74-2972d0b214ee => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.785522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.786020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.792121 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.792868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.793324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.794013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.794799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.795260 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.809102 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 HTTP/1.1 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '309ca9c6-85ac-47ae-bda8-3c8391ac41b5'} Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.810620 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09c5b547-3cd3-4a25-83eb-c9e0cf419b5d tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 404 len: 111 microversion: 2.1 time: 0.030651 Oct 15 21:50:48.815550 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4238/6231] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:50:48 2022] DELETE /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:50:48.820468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.821125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: 309ca9c6-85ac-47ae-bda8-3c8391ac41b5] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.827998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.828508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.828959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.829764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.830241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.830686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.845191 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 HTTP/1.1 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '309ca9c6-85ac-47ae-bda8-3c8391ac41b5'} Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.846732 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance 309ca9c6-85ac-47ae-bda8-3c8391ac41b5 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6bbc66fd-90ed-4550-b0f6-a08cd419ce6b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5" status: 404 len: 111 microversion: 2.1 time: 0.031540 Oct 15 21:50:48.851958 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1994/6232] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:48 2022] GET /compute/v2.1/servers/309ca9c6-85ac-47ae-bda8-3c8391ac41b5 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.856478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.856977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.863148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.863592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.864034 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.864761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.865246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.865678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.879488 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 HTTP/1.1 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'cc2752f4-8611-45a9-be62-a077e31fdde8'} Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.881023 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-48ae186e-5f8c-41c6-840c-bd025a4d970f tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 404 len: 111 microversion: 2.1 time: 0.029825 Oct 15 21:50:48.886475 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4239/6233] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:50:48 2022] DELETE /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:50:48.891695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.892317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] [instance: cc2752f4-8611-45a9-be62-a077e31fdde8] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:48.898535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.899030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.899481 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.900185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:48.900651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:48.901705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:48.915213 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] HTTP exception thrown: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Request method failure captured: Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 HTTP/1.1 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSymOmNguAmrFZ9AXYnOrbDsLkFEdysfWUG6OZXKN6hFjbgAbWQDy2d3OiOuETWNICUFBh6Z49Qcunf8yOwCb4YeB5NVFrZ2yf_IaJYwKYAqkx-SO-7TkeAaIWgyDBizYW9-TuYrFU-UaPUuSyZsi9QRl70MR3rv5u1Yvn-q3z_G7IjU Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d42197537d0b407882fc410a5d359e1b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d42197537d0b407882fc410a5d359e1b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: d42197537d0b407882fc410a5d359e1b Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServerActionsTestJSON-390599151 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4165cc6b781b4df3abd6a227170d4e46 Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServerActionsTestJSON-390599151-project Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'cc2752f4-8611-45a9-be62-a077e31fdde8'} Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:50:48.916747 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Returning 404 to user: Instance cc2752f4-8611-45a9-be62-a077e31fdde8 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5fd0dea8-ddc4-4dd4-a12e-4d6a1837038b tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8" status: 404 len: 111 microversion: 2.1 time: 0.030590 Oct 15 21:50:48.922248 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1995/6234] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:48 2022] GET /compute/v2.1/servers/cc2752f4-8611-45a9-be62-a077e31fdde8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 15 21:50:48.926619 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0c2fcbf0-d265-4c12-a8ba-ef85e26add7c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] Calling method '.version_select at 0x7fe758517430>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:48.949989 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0c2fcbf0-d265-4c12-a8ba-ef85e26add7c tempest-ServerActionsTestJSON-390599151 tempest-ServerActionsTestJSON-390599151-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-350156625" status: 202 len: 0 microversion: 2.1 time: 0.027121 Oct 15 21:50:48.950710 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4240/6235] 10.222.0.16 () {58 vars in 1218 bytes} [Sat Oct 15 21:50:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-350156625 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:50:58.438099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-902325300", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "11d3c406-85de-4e5f-8e28-62bdf1f25df8"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:50:58.496509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] validate_networks() for [('11d3c406-85de-4e5f-8e28-62bdf1f25df8', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:50:58.903944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:50:58.904398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:50:58.904903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:50:58.905394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:50:58.905832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:50:58.908003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=51 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:58.917417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:58.917855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:58.918554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:58.919492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:50:58.919974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:50:58.920427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:50:58.939267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Getting quotas for project b75b65002a4e4116945a865160f9ecd5. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:50:58.944744 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Getting quotas for user 1e44ab28be024022bf6c8eaa84239494 and project b75b65002a4e4116945a865160f9ecd5. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:50:58.958376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:50:58.960757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:50:59.009247 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b385e6b-4b6b-425a-969a-c8c7427bb60b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.731912 Oct 15 21:50:59.016398 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1996/6236] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:50:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 740 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:50:59.020014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:59.020766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=37,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:50:59.025242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3857238c-8526-428d-9b1d-cecd825c44c7 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:50:59.026268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3857238c-8526-428d-9b1d-cecd825c44c7 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:50:59.094798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3857238c-8526-428d-9b1d-cecd825c44c7 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:50:59.102570 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3857238c-8526-428d-9b1d-cecd825c44c7 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1216 microversion: 2.1 time: 0.084535 Oct 15 21:50:59.103152 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4241/6237] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:50:59 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1216 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:00.130383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:00.131408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:00.139664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:00.140682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:00.141167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:00.144029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:00.145238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:00.145580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:00.258329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:00.267451 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0ff3290e-33d0-44ee-b4b7-25dcc1bb609d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1286 microversion: 2.1 time: 0.144788 Oct 15 21:51:00.267964 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1997/6238] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:00 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1286 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:01.295885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:01.297027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:01.309699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:01.310745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:01.311499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:01.312413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:01.313115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:01.313977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:01.519298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:01.527897 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-055c186e-7b41-48d5-9549-657aeada67f2 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.240316 Oct 15 21:51:01.541573 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4242/6239] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:01 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:01.598092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d0c926ce-1497-4f75-a1d1-f3f267dc9bf8 req-92a30ac6-4258-4da6-9135-01c97af362d1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb52c112-8bd7-432c-b4a8-48c520dbd790", "tag": "cdd159b8-fa42-413f-8498-724f60d7290a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:01.609411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d0c926ce-1497-4f75-a1d1-f3f267dc9bf8 req-92a30ac6-4258-4da6-9135-01c97af362d1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:01.609895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d0c926ce-1497-4f75-a1d1-f3f267dc9bf8 req-92a30ac6-4258-4da6-9135-01c97af362d1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:01.610435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d0c926ce-1497-4f75-a1d1-f3f267dc9bf8 req-92a30ac6-4258-4da6-9135-01c97af362d1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:01.623235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d0c926ce-1497-4f75-a1d1-f3f267dc9bf8 req-92a30ac6-4258-4da6-9135-01c97af362d1 service nova] Creating event network-changed:cdd159b8-fa42-413f-8498-724f60d7290a for instance cb52c112-8bd7-432c-b4a8-48c520dbd790 on nch2-master-1 Oct 15 21:51:01.628486 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d0c926ce-1497-4f75-a1d1-f3f267dc9bf8 req-92a30ac6-4258-4da6-9135-01c97af362d1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033962 Oct 15 21:51:01.629237 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1998/6240] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:51:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:51:02.556701 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:02.557326 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:02.564196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:02.564680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:02.565155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:02.565853 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:02.566308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:02.566735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:02.701324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:02.709012 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1ea5086-0bae-40b1-89cf-6a0bc2f6352a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.161036 Oct 15 21:51:02.709596 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4243/6241] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:02 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:03.733628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:03.734273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:03.740453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:03.740957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:03.741440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:03.742153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:03.742628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:03.743076 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:03.874302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:03.882170 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-af720171-e455-40d4-aa1d-49e257436b1b tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.154143 Oct 15 21:51:03.882741 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 1999/6242] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:03 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:04.906065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:04.906683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:04.913814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:04.914312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:04.914748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:04.915450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:04.915913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:04.916347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:05.048052 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:05.055858 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f113f256-ea0d-4de4-976b-19394d215761 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.153877 Oct 15 21:51:05.056425 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4244/6243] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:04 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:06.080640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:06.081313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:06.083619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:06.090180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:06.090863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:06.091320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:06.092032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:06.092502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:06.092946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:06.227411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:06.235333 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e1d2e6a4-3408-421d-b771-82719d9651c5 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.158632 Oct 15 21:51:06.235903 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2000/6244] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:06 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:07.259006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:07.259640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:07.270770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:07.271387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:07.271830 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:07.272539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:07.272998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:07.273459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:07.404684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:07.412408 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-515eb1dd-3ddf-4545-9561-87930e53d42c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.157408 Oct 15 21:51:07.412980 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4245/6245] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:07 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:07.898703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e93d9c84-b9ad-4f81-90db-073fb25a38e5 req-be20fe38-8f10-4a4f-ba87-bd5b4612055a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb52c112-8bd7-432c-b4a8-48c520dbd790", "name": "network-vif-plugged", "status": "completed", "tag": "cdd159b8-fa42-413f-8498-724f60d7290a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:07.911093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e93d9c84-b9ad-4f81-90db-073fb25a38e5 req-be20fe38-8f10-4a4f-ba87-bd5b4612055a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:07.911803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e93d9c84-b9ad-4f81-90db-073fb25a38e5 req-be20fe38-8f10-4a4f-ba87-bd5b4612055a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:07.912264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e93d9c84-b9ad-4f81-90db-073fb25a38e5 req-be20fe38-8f10-4a4f-ba87-bd5b4612055a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:07.924363 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e93d9c84-b9ad-4f81-90db-073fb25a38e5 req-be20fe38-8f10-4a4f-ba87-bd5b4612055a service nova] Creating event network-vif-plugged:cdd159b8-fa42-413f-8498-724f60d7290a for instance cb52c112-8bd7-432c-b4a8-48c520dbd790 on nch2-master-1 Oct 15 21:51:07.929490 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e93d9c84-b9ad-4f81-90db-073fb25a38e5 req-be20fe38-8f10-4a4f-ba87-bd5b4612055a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034723 Oct 15 21:51:07.930248 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2001/6246] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:51:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:51:08.435629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:08.436283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:08.443361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:08.443863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:08.444287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:08.444992 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:08.445494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:08.445927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:08.577477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:08.585211 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b6c49d8-ba1f-4d85-8397-6c56e5a25258 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1316 microversion: 2.1 time: 0.154486 Oct 15 21:51:08.585794 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4246/6247] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:08 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1316 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:09.609484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:09.610243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:09.613256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:09.613980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:09.619865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:09.620368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:09.620825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:09.621569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:09.622042 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:09.622490 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:09.754051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:09.761904 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-edcb04eb-e9fb-49c7-8b34-0a450cc45b34 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.156863 Oct 15 21:51:09.763010 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2002/6248] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:09 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:09.824192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-244564253", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "11d3c406-85de-4e5f-8e28-62bdf1f25df8"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:09.833776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:09.880287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] validate_networks() for [('11d3c406-85de-4e5f-8e28-62bdf1f25df8', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:51:10.333877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:51:10.334307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:51:10.334783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:51:10.335228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:51:10.335755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:51:10.345364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:10.345872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:10.346318 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:10.347042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:10.347499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:10.347924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:10.364637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Getting quotas for project b75b65002a4e4116945a865160f9ecd5. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:51:10.369828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Getting quotas for user 1e44ab28be024022bf6c8eaa84239494 and project b75b65002a4e4116945a865160f9ecd5. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:51:10.382013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:51:10.384295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:51:10.430974 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-97925a7a-b314-498f-b84c-e26abab44d6c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.610446 Oct 15 21:51:10.431632 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4247/6249] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:51:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 612 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:51:10.443365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3d7a636c-1a12-4ab7-b84e-043baa6a11c8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:10.444025 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3d7a636c-1a12-4ab7-b84e-043baa6a11c8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:10.501233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3d7a636c-1a12-4ab7-b84e-043baa6a11c8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:10.508329 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3d7a636c-1a12-4ab7-b84e-043baa6a11c8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1216 microversion: 2.1 time: 0.068924 Oct 15 21:51:10.508963 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2003/6250] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:10 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1216 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:11.532761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:11.533411 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:11.540298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:11.540784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:11.541252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:11.541949 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:11.542493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:11.542924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:11.613581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:11.622062 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-279bffbb-0d26-4ab1-b2ed-b14c308b6971 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1274 microversion: 2.1 time: 0.093507 Oct 15 21:51:11.622951 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4248/6251] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:11 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1274 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:12.647329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:12.648086 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:12.655697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:12.656217 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:12.656677 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:12.657433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:12.657909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:12.658355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:12.797592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ad3fa016-1639-4a45-80b1-9d70d888eae2 req-1fc54c5b-67eb-4695-972c-2f38a85e26f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1fce011f-38ff-4703-b3fa-a0dc87251994", "tag": "917a5b24-905a-4d4f-b25d-6e945fe75c8d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:12.809438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad3fa016-1639-4a45-80b1-9d70d888eae2 req-1fc54c5b-67eb-4695-972c-2f38a85e26f8 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:12.810248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad3fa016-1639-4a45-80b1-9d70d888eae2 req-1fc54c5b-67eb-4695-972c-2f38a85e26f8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:12.810771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ad3fa016-1639-4a45-80b1-9d70d888eae2 req-1fc54c5b-67eb-4695-972c-2f38a85e26f8 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:12.823171 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ad3fa016-1639-4a45-80b1-9d70d888eae2 req-1fc54c5b-67eb-4695-972c-2f38a85e26f8 service nova] Creating event network-changed:917a5b24-905a-4d4f-b25d-6e945fe75c8d for instance 1fce011f-38ff-4703-b3fa-a0dc87251994 on nch1-master-1 Oct 15 21:51:12.828182 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ad3fa016-1639-4a45-80b1-9d70d888eae2 req-1fc54c5b-67eb-4695-972c-2f38a85e26f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034989 Oct 15 21:51:12.828798 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4249/6252] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:51:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:51:12.909487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:12.917371 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-03f2dd50-a7d4-4d1e-a0bf-0ad5c4f2d496 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.273948 Oct 15 21:51:12.917943 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2004/6253] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:12 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:13.940646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:13.941379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:13.947606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:13.948088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:13.948526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:13.949248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:13.949766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:13.950191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:14.224099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:14.232122 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2f0ce152-5581-4408-9c75-6019bd72f90f tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.296511 Oct 15 21:51:14.232705 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4250/6254] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:13 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:15.255133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:15.255778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:15.263613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:15.264147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:15.264602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:15.265623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:15.266107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:15.266559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:15.398207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:15.406085 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-327046f7-21c8-431e-876d-184a2aacc0ca tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.154887 Oct 15 21:51:15.406653 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2005/6255] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:15 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:16.428836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:16.429510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:16.437147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:16.437672 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:16.438123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:16.438829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:16.439295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:16.439732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:16.568047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:16.576213 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c644c9ee-b19b-439f-a00f-fd13558c100d tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.151260 Oct 15 21:51:16.576791 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4251/6256] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:16 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:17.598495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:17.599060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:17.605966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:17.606472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:17.606927 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:17.607638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:17.608115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:17.608583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:17.740407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:17.748780 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1859ef7c-3246-4170-bb18-31a9fc526a70 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.154413 Oct 15 21:51:17.749404 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2006/6257] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:17 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:18.772242 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:18.772868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:18.779817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:18.780314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:18.780763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:18.781497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:18.781972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:18.782404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:18.915437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:18.923871 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bbb26302-75db-4c1e-b3d3-80b6fe20f098 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.155767 Oct 15 21:51:18.924458 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4252/6258] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:18 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:19.047517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-4a2fee80-6d2b-43b3-8fb2-abb33ee83f48 req-b6a84686-0ea0-4a83-afd3-13b0638d1f0d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1fce011f-38ff-4703-b3fa-a0dc87251994", "name": "network-vif-plugged", "status": "completed", "tag": "917a5b24-905a-4d4f-b25d-6e945fe75c8d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:19.058382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a2fee80-6d2b-43b3-8fb2-abb33ee83f48 req-b6a84686-0ea0-4a83-afd3-13b0638d1f0d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:19.059678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a2fee80-6d2b-43b3-8fb2-abb33ee83f48 req-b6a84686-0ea0-4a83-afd3-13b0638d1f0d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:19.060214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-4a2fee80-6d2b-43b3-8fb2-abb33ee83f48 req-b6a84686-0ea0-4a83-afd3-13b0638d1f0d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:19.072506 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-4a2fee80-6d2b-43b3-8fb2-abb33ee83f48 req-b6a84686-0ea0-4a83-afd3-13b0638d1f0d service nova] Creating event network-vif-plugged:917a5b24-905a-4d4f-b25d-6e945fe75c8d for instance 1fce011f-38ff-4703-b3fa-a0dc87251994 on nch1-master-1 Oct 15 21:51:19.077767 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-4a2fee80-6d2b-43b3-8fb2-abb33ee83f48 req-b6a84686-0ea0-4a83-afd3-13b0638d1f0d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034643 Oct 15 21:51:19.078377 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2007/6259] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:51:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:51:19.946887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:19.947500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:19.954859 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:19.955344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:19.955783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:19.956474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:19.956930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:19.957386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:20.092842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:20.100550 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8847d3e4-d246-475d-9587-7233eed2e4ef tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1316 microversion: 2.1 time: 0.157494 Oct 15 21:51:20.101167 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4253/6260] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:19 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1316 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:21.125024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:21.125774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:21.128679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:21.134186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:21.134689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:21.135141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:21.135849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:21.136319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:21.136759 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:21.270438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:21.280271 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bcb2baf1-9165-4cfc-9e3e-4f5e723f7493 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1504 microversion: 2.1 time: 0.160081 Oct 15 21:51:21.280870 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2008/6261] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:21 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1504 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:21.302931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:21.303475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:21.305445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:21.306215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:21.313603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:21.314119 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:21.314613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:21.315364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:21.315828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:21.316265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:21.342408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:51:21.417141 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bb79a37b-8edd-428b-b0ad-274a7dcbb0ba tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 204 len: 0 microversion: 2.1 time: 0.117915 Oct 15 21:51:21.417740 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4254/6262] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:51:21 2022] DELETE /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:51:21.426246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:21.426879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:21.432983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:21.433520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:21.434651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:21.435368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:21.435845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:21.436291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:21.566381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:21.574236 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-10ce46f0-5029-4cc6-976a-cfb4747dfba3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1510 microversion: 2.1 time: 0.152076 Oct 15 21:51:21.575028 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2009/6263] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:21 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1510 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:22.598148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:22.598871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:22.606374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:22.607107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:22.607546 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:22.608244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:22.608700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:22.609160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:22.815998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:22.823884 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-477343a5-8893-48b9-94f8-7d8f54ec65d3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 200 len: 1510 microversion: 2.1 time: 0.229988 Oct 15 21:51:22.824470 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4255/6264] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:22 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 1510 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:23.174211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b290da5d-c06f-4110-95aa-84dd4a56ccae req-ec4c5cfe-b7c9-4bf6-8f4e-10da505d28df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1fce011f-38ff-4703-b3fa-a0dc87251994", "name": "network-vif-deleted", "tag": "917a5b24-905a-4d4f-b25d-6e945fe75c8d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:23.186342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b290da5d-c06f-4110-95aa-84dd4a56ccae req-ec4c5cfe-b7c9-4bf6-8f4e-10da505d28df service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:23.187107 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b290da5d-c06f-4110-95aa-84dd4a56ccae req-ec4c5cfe-b7c9-4bf6-8f4e-10da505d28df service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:23.187878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b290da5d-c06f-4110-95aa-84dd4a56ccae req-ec4c5cfe-b7c9-4bf6-8f4e-10da505d28df service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:23.204600 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b290da5d-c06f-4110-95aa-84dd4a56ccae req-ec4c5cfe-b7c9-4bf6-8f4e-10da505d28df service nova] Creating event network-vif-deleted:917a5b24-905a-4d4f-b25d-6e945fe75c8d for instance 1fce011f-38ff-4703-b3fa-a0dc87251994 on nch1-master-1 Oct 15 21:51:23.210717 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b290da5d-c06f-4110-95aa-84dd4a56ccae req-ec4c5cfe-b7c9-4bf6-8f4e-10da505d28df service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040177 Oct 15 21:51:23.211647 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2010/6265] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:51:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:51:23.847512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:23.848053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:23.854971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:23.855453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:23.855892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:23.856667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:23.857161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:23.857592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:23.870869 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] HTTP exception thrown: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 HTTP/1.1 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1fce011f-38ff-4703-b3fa-a0dc87251994'} Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:23.872623 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 404 to user: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-851a7f78-98f2-44eb-babe-fa1ab6be027c tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 404 len: 111 microversion: 2.1 time: 0.030747 Oct 15 21:51:23.878088 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4256/6266] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:23 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:51:23.883591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:23.884311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:23.891445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:23.891940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:23.892391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:23.893126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:23.893600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:23.894041 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.023074 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:24.031041 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7dbd7e5f-dc85-4578-8b2c-cd47bba4fc62 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.151616 Oct 15 21:51:24.031611 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2011/6267] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:23 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:24.053664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06477cf4-855b-4c30-a95d-0b568871ecdc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "452"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:24.055987 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-06477cf4-855b-4c30-a95d-0b568871ecdc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 0dda2970-33cf-42ab-b1c6-f7cd3385f141] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.061639 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-06477cf4-855b-4c30-a95d-0b568871ecdc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] HTTP exception thrown: Instance 0dda2970-33cf-42ab-b1c6-f7cd3385f141 could not be found. Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06477cf4-855b-4c30-a95d-0b568871ecdc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/0dda2970-33cf-42ab-b1c6-f7cd3385f141/action HTTP/1.1 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 32 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: {"resize": {"flavorRef": "452"}} Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 0dda2970-33cf-42ab-b1c6-f7cd3385f141 could not be found. Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '0dda2970-33cf-42ab-b1c6-f7cd3385f141', 'body': {'resize': {'flavorRef': '452'}}} Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0dda2970-33cf-42ab-b1c6-f7cd3385f141 could not be found. Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1046, in _resize Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_id, Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:51:24.063179 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0dda2970-33cf-42ab-b1c6-f7cd3385f141 could not be found. Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0dda2970-33cf-42ab-b1c6-f7cd3385f141 could not be found. Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-06477cf4-855b-4c30-a95d-0b568871ecdc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 404 to user: Instance 0dda2970-33cf-42ab-b1c6-f7cd3385f141 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-06477cf4-855b-4c30-a95d-0b568871ecdc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "POST /compute/v2.1/servers/0dda2970-33cf-42ab-b1c6-f7cd3385f141/action" status: 404 len: 111 microversion: 2.1 time: 0.014818 Oct 15 21:51:24.067774 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4257/6268] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:51:24 2022] POST /compute/v2.1/servers/0dda2970-33cf-42ab-b1c6-f7cd3385f141/action => generated 111 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:51:24.072819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:24.073524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.080408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.080926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.081412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.082134 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.082617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.083070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.214554 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:24.222339 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-db772475-269f-4eb2-a8dd-d2fae3716024 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.153358 Oct 15 21:51:24.222919 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2012/6269] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:24 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:51:24.244729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:24.245366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.252815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.253324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.253766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.254457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.254912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.255341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.385728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:24.394921 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-49d0c236-fe86-4e83-a521-27a96ec66409 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.153916 Oct 15 21:51:24.395506 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4258/6270] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:24 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:24.417824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "c1d3ee97-1221-44b4-8993-827402122589"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:24.420065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.426195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.426694 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.427144 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.427852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.428323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.428761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.472171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:24.477484 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] HTTP exception thrown: Flavor c1d3ee97-1221-44b4-8993-827402122589 could not be found. Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: request: POST /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790/action HTTP/1.1 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 65 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: {"resize": {"flavorRef": "c1d3ee97-1221-44b4-8993-827402122589"}} Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: exception: Flavor c1d3ee97-1221-44b4-8993-827402122589 could not be found. Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'cb52c112-8bd7-432c-b4a8-48c520dbd790', 'body': {'resize': {'flavorRef': 'c1d3ee97-1221-44b4-8993-827402122589'}}} Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4200, in resize Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi new_flavor = flavors.get_flavor_by_flavor_id( Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:24.490240 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor c1d3ee97-1221-44b4-8993-827402122589 could not be found. Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1083, in _resize Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor c1d3ee97-1221-44b4-8993-827402122589 could not be found. Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor c1d3ee97-1221-44b4-8993-827402122589 could not be found. Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 400 to user: Flavor c1d3ee97-1221-44b4-8993-827402122589 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52de0f2f-bc97-41e6-a3f2-22d922277173 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "POST /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790/action" status: 400 len: 107 microversion: 2.1 time: 0.078695 Oct 15 21:51:24.495309 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2013/6271] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:51:24 2022] POST /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790/action => generated 107 bytes in 80 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:51:24.502163 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:24.503006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.509204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.509695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.510168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.510870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.511337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.512373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.641173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:24.644742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:24.649979 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b4071532-b8fc-4851-811d-f9660179d244 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.151675 Oct 15 21:51:24.650561 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4259/6272] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:24 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:24.673713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:24.674272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.680168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.680665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.681150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.681852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.682319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.682756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.816327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:24.824400 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-22b27eeb-66f9-487d-8d62-fa27997c8ed9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.154487 Oct 15 21:51:24.825424 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2014/6273] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:24 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:24.848086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30d7167f-6898-4b5a-b0b4-2507ff103db3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": ""}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30d7167f-6898-4b5a-b0b4-2507ff103db3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: request: POST /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790/action HTTP/1.1 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 29 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: {"resize": {"flavorRef": ""}} Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: exception: Invalid input for field/attribute flavorRef. Value: . '' is too short Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'cb52c112-8bd7-432c-b4a8-48c520dbd790', 'body': {'resize': {'flavorRef': ''}}} Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise error Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['resize']['properties']['flavorRef']: Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi {'minLength': 1, 'type': ['string', 'integer']} Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi On instance['resize']: Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi '' Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute flavorRef. Value: . '' is too short Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.851695 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute flavorRef. Value: . '' is too short Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30d7167f-6898-4b5a-b0b4-2507ff103db3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 400 to user: Invalid input for field/attribute flavorRef. Value: . '' is too short {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30d7167f-6898-4b5a-b0b4-2507ff103db3 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "POST /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790/action" status: 400 len: 113 microversion: 2.1 time: 0.009536 Oct 15 21:51:24.857389 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4260/6274] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:51:24 2022] POST /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790/action => generated 113 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 15 21:51:24.862218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:24.862849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:24.868720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.869512 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.869701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:24.870401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:24.870868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:24.871879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.000064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:25.007824 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f4cba221-8fc2-471d-b8b1-b14112cfb1c9 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1505 microversion: 2.1 time: 0.149547 Oct 15 21:51:25.008393 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2015/6275] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:24 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1505 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:25.035724 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:25.036251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:25.046880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.047366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.047803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.048496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.048951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.049429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.063262 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] HTTP exception thrown: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 HTTP/1.1 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1fce011f-38ff-4703-b3fa-a0dc87251994'} Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:51:25.064801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 404 to user: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8cc6ed63-8f17-421e-b34a-87ef1c8f3ab6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 404 len: 111 microversion: 2.1 time: 0.035637 Oct 15 21:51:25.069444 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4261/6276] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:51:25 2022] DELETE /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:51:25.076137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:25.076766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: 1fce011f-38ff-4703-b3fa-a0dc87251994] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:25.082784 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.083570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.084020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.084728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.085259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.085701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.099440 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] HTTP exception thrown: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 HTTP/1.1 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '1fce011f-38ff-4703-b3fa-a0dc87251994'} Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:25.100991 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 404 to user: Instance 1fce011f-38ff-4703-b3fa-a0dc87251994 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-930892a9-8199-4754-a5cd-6b72addf719a tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994" status: 404 len: 111 microversion: 2.1 time: 0.030784 Oct 15 21:51:25.105752 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2016/6277] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:25 2022] GET /compute/v2.1/servers/1fce011f-38ff-4703-b3fa-a0dc87251994 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:51:25.111069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:25.111573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:25.117737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.118227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.118666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.119362 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.119826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.120259 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.146148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:51:25.222544 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-84a10911-cb2a-4098-b74f-c24617285adc tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 204 len: 0 microversion: 2.1 time: 0.115068 Oct 15 21:51:25.223208 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4262/6278] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:51:25 2022] DELETE /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:51:25.232448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:25.233127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:25.239957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.240461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.240914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.241648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:25.242121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:25.242564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:25.372235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:25.375800 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=100 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:51:25.381029 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-873b164e-266a-485a-a8fa-f3ebf2a4a2fd tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1511 microversion: 2.1 time: 0.153294 Oct 15 21:51:25.381614 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2017/6279] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:25 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1511 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:26.406113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:26.406855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:26.413536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:26.414024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:26.414466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:26.415160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:26.415620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:26.416051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:26.552753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:26.560850 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a45b0d41-1f5d-4342-affa-ce555677a5b8 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1511 microversion: 2.1 time: 0.159775 Oct 15 21:51:26.561463 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4263/6280] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:26 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1511 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:27.351491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c04400a2-f5e3-40c6-86a8-4d08044dfe6d req-0e1fc88a-b39b-4227-9937-a7a0cf9c687a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb52c112-8bd7-432c-b4a8-48c520dbd790", "name": "network-vif-deleted", "tag": "cdd159b8-fa42-413f-8498-724f60d7290a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:51:27.370565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c04400a2-f5e3-40c6-86a8-4d08044dfe6d req-0e1fc88a-b39b-4227-9937-a7a0cf9c687a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:27.371430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c04400a2-f5e3-40c6-86a8-4d08044dfe6d req-0e1fc88a-b39b-4227-9937-a7a0cf9c687a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:27.372522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c04400a2-f5e3-40c6-86a8-4d08044dfe6d req-0e1fc88a-b39b-4227-9937-a7a0cf9c687a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:27.391144 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c04400a2-f5e3-40c6-86a8-4d08044dfe6d req-0e1fc88a-b39b-4227-9937-a7a0cf9c687a service nova] Creating event network-vif-deleted:cdd159b8-fa42-413f-8498-724f60d7290a for instance cb52c112-8bd7-432c-b4a8-48c520dbd790 on nch2-master-1 Oct 15 21:51:27.399204 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c04400a2-f5e3-40c6-86a8-4d08044dfe6d req-0e1fc88a-b39b-4227-9937-a7a0cf9c687a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054257 Oct 15 21:51:27.400294 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2018/6281] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:51:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:51:27.583400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:27.583952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:27.591648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:27.592134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:27.592571 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:27.593306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:27.593769 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:27.594202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:27.624680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:27.625196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:27.625637 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:27.626086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lazy-loading 'fault' on Instance uuid cb52c112-8bd7-432c-b4a8-48c520dbd790 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:51:27.667901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e44ab28be024022bf6c8eaa84239494', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b75b65002a4e4116945a865160f9ecd5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:51:27.675976 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-75a1f2b9-801e-468b-99fc-270eea445996 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 200 len: 1306 microversion: 2.1 time: 0.096583 Oct 15 21:51:27.676543 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4264/6282] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:27 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 1306 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:51:28.701568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:51:28.702369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] [instance: cb52c112-8bd7-432c-b4a8-48c520dbd790] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:51:28.708564 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:28.709085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:28.709537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:28.710240 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:51:28.710708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:51:28.711145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:51:28.724713 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] HTTP exception thrown: Instance cb52c112-8bd7-432c-b4a8-48c520dbd790 could not be found. Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Request method failure captured: Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 HTTP/1.1 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSytC3SCmx-WVy2PEbAkaJsux-eQNBoyDvYiNSXqXIVzs8iAMRTjRGBwHPE88e0uu5KTLD5N_J4Pi1MwM6RG3Rkpw_hwWD788y13MvmZRwHrx1uqS63mje7r-pU9bAGyFoI-jroGn6HsD7ZHEezg08fMrB4wjAstVmOlFNCgYP_HYjk4 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b75b65002a4e4116945a865160f9ecd5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: b75b65002a4e4116945a865160f9ecd5 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-ServersNegativeTestJSON-190385960 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 1e44ab28be024022bf6c8eaa84239494 Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-ServersNegativeTestJSON-190385960-project Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance cb52c112-8bd7-432c-b4a8-48c520dbd790 could not be found. Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'cb52c112-8bd7-432c-b4a8-48c520dbd790'} Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb52c112-8bd7-432c-b4a8-48c520dbd790 could not be found. Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb52c112-8bd7-432c-b4a8-48c520dbd790 could not be found. Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:51:28.726373 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb52c112-8bd7-432c-b4a8-48c520dbd790 could not be found. Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] Returning 404 to user: Instance cb52c112-8bd7-432c-b4a8-48c520dbd790 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6b32b923-9cfa-4f5f-b152-7e9c670fabb6 tempest-ServersNegativeTestJSON-190385960 tempest-ServersNegativeTestJSON-190385960-project] 10.222.0.16 "GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790" status: 404 len: 111 microversion: 2.1 time: 0.032132 Oct 15 21:51:28.731915 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2019/6283] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:51:28 2022] GET /compute/v2.1/servers/cb52c112-8bd7-432c-b4a8-48c520dbd790 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:52:10.510613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-387ee995-67bf-4d96-9db9-3b9943b20ffb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'create', calling method: .version_select at 0x7fe74fb3b430>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-453138580", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD6An32iHSaaDBXLJRio1E1dz+euLJmnWwafOOUCRpUnW1Dkt6bU3x6urktAJfkd4EKV8Fr589vpEzzskmK/E196yWhEr0pCNCWyssondvHRtx09NQK/L5qDPeYDAsuR9w=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:10.517552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:10.518247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:10.536434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-387ee995-67bf-4d96-9db9-3b9943b20ffb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for project cd7071741b2e4f35954da54683a2b55c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:52:10.544261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-387ee995-67bf-4d96-9db9-3b9943b20ffb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for user 4768c0948f1b44689210529a25211aa7 and project cd7071741b2e4f35954da54683a2b55c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:52:10.570605 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-387ee995-67bf-4d96-9db9-3b9943b20ffb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.064040 Oct 15 21:52:10.571225 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4265/6284] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:52:10 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:52:10.623884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2105953069", "imageRef": "d7c7c8ba-4245-43f6-a647-f8936596c70b", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-453138580", "networks": [{"uuid": "75bdcce5-e24a-477d-a243-69942be01590"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:10.633013 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=43 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:10.633741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:10.683703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] validate_networks() for [('75bdcce5-e24a-477d-a243-69942be01590', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:52:11.149479 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:52:11.149925 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:52:11.150447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:52:11.150879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:52:11.151309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:52:11.160972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:11.161502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:11.161953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:11.162702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:11.163175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:11.163618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:11.180672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for project cd7071741b2e4f35954da54683a2b55c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:52:11.186492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for user 4768c0948f1b44689210529a25211aa7 and project cd7071741b2e4f35954da54683a2b55c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:52:11.200593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:52:11.202908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d7c7c8ba-4245-43f6-a647-f8936596c70b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:52:11.248506 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-65847aa6-38ce-4638-b6c8-3cf8efb93c1f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.629737 Oct 15 21:52:11.249189 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2020/6285] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:52:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 631 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:52:11.260113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-64d9dfb7-0ab0-479b-bf3a-83bf158a8689 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:11.260741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-64d9dfb7-0ab0-479b-bf3a-83bf158a8689 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:11.316810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-64d9dfb7-0ab0-479b-bf3a-83bf158a8689 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:11.324452 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-64d9dfb7-0ab0-479b-bf3a-83bf158a8689 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1258 microversion: 2.1 time: 0.067702 Oct 15 21:52:11.325053 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4266/6286] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:11 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1258 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:12.348443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:12.349842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:12.357385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:12.358128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:12.358592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:12.359354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:12.360019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:12.360466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:12.448297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:12.458695 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d792f6b1-51c6-42b2-8cdb-313c2a3b0f97 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1316 microversion: 2.1 time: 0.115400 Oct 15 21:52:12.459380 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2021/6287] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:12 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1316 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:13.484008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:13.484604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:13.494151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:13.494705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:13.495201 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:13.496594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:13.497071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:13.497519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:13.646153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:13.654892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d4e489df-db03-4c56-bccf-d144b2cd9144 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.176330 Oct 15 21:52:13.655591 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4267/6288] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:13 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:14.679562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:14.680295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:14.688364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:14.688950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:14.689563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:14.690349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:14.690893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:14.691421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:14.833427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:14.842986 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-08d428c1-329b-4a24-9e38-46b67d449dac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.167390 Oct 15 21:52:14.843655 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2022/6289] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:14 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:15.867920 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:15.868715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:15.877407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:15.878028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:15.878709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:15.879497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:15.879964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:15.880397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:16.027393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:16.036434 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5c9d046b-8274-4546-9ef4-7b83655730cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.172952 Oct 15 21:52:16.040081 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4268/6290] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:15 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:17.066995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:17.068382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:17.076317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:17.076817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:17.077344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:17.078101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:17.078574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:17.079012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:17.227522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:17.236019 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-60cd0e21-e5b2-4036-b8e1-6cf776b0c20c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.176627 Oct 15 21:52:17.236898 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2023/6291] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:17 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:18.260168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:18.267475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:18.276040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:18.276839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:18.277385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:18.278189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:18.278663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:18.279099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:18.434278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:18.443800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-39db32b3-2543-4bd4-a7fa-d0149322d23d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.187625 Oct 15 21:52:18.446155 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4269/6292] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:18 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:19.470208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:19.470208 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:19.493016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:19.493016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:19.493016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:19.494015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:19.494502 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:19.496401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:19.650319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:19.659637 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-606bc327-3dcf-4330-81c3-69907f2841f5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.195572 Oct 15 21:52:19.660226 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2024/6293] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:19 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:20.684211 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:20.684772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:20.691839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:20.692334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:20.692781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:20.693511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:20.693977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:20.694414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:20.827702 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:20.835403 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5f9d20ff-4888-4e37-b758-27a180efba6b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.155417 Oct 15 21:52:20.835978 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4270/6294] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:20 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:21.860158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:21.860737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:21.863474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:21.864153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:21.870151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:21.871201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:21.871650 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:21.872356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:21.872821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:21.873287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:22.007775 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:22.016182 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5b2fa3d3-230b-4836-b066-82d188b81e44 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.160032 Oct 15 21:52:22.016752 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2025/6295] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:21 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:23.039332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:23.039897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:23.041778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:23.048445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:23.048936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:23.049413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:23.050124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:23.050616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:23.051055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:23.288289 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:23.297812 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f4c48824-6d3c-4797-83a6-11ef8297a96a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.262400 Oct 15 21:52:23.298325 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4271/6296] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:23 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:23.633602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-b5a7a98d-94f7-4385-bbfa-ce06d2117009 req-095a5497-8998-41af-a27e-11b504bd0fed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:23.643469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b5a7a98d-94f7-4385-bbfa-ce06d2117009 req-095a5497-8998-41af-a27e-11b504bd0fed service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:23.644206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b5a7a98d-94f7-4385-bbfa-ce06d2117009 req-095a5497-8998-41af-a27e-11b504bd0fed service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:23.644666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-b5a7a98d-94f7-4385-bbfa-ce06d2117009 req-095a5497-8998-41af-a27e-11b504bd0fed service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:23.656364 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-b5a7a98d-94f7-4385-bbfa-ce06d2117009 req-095a5497-8998-41af-a27e-11b504bd0fed service nova] Creating event network-changed:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:52:23.661346 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-b5a7a98d-94f7-4385-bbfa-ce06d2117009 req-095a5497-8998-41af-a27e-11b504bd0fed service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.154132 Oct 15 21:52:23.662104 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2026/6297] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:52:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:52:24.321745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:24.322397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:24.330111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:24.330602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:24.331047 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:24.331752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:24.332219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:24.332652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:24.461171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:24.469331 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-137a80ba-9c13-437e-9b2e-905e98d193f9 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.151470 Oct 15 21:52:24.469935 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4272/6298] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:24 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:25.493630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:25.494452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:25.502625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:25.503116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:25.503560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:25.504261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:25.504727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:25.505195 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:25.643051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:25.650859 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-076b7816-8a5f-4da9-ae91-ada6c36786de tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.162470 Oct 15 21:52:25.651914 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2027/6299] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:25 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:26.673997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:26.674647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:26.681458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:26.681946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:26.682406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:26.683584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:26.684039 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:26.684468 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:26.812470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:26.820301 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f47a2e5-a62c-4981-a7dc-aca7f34b8275 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.150233 Oct 15 21:52:26.820877 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4273/6300] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:26 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:27.844032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:27.844661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:27.851618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:27.852118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:27.852574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:27.853315 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:27.853789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:27.854256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:27.984600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:27.992478 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d66f6396-989f-47d6-80f7-a63315db7025 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.152396 Oct 15 21:52:27.993111 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2028/6301] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:27 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:29.016728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:29.017433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:29.024337 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:29.024840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:29.025294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:29.025986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:29.027053 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:29.027481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:29.164291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:29.172074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-19f52836-9213-4019-9dcc-f44e78d8ef18 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.159475 Oct 15 21:52:29.172642 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4274/6302] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:29 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:30.195159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:30.196746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:30.206985 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:30.207604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:30.208124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:30.208848 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:30.209409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:30.209872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:30.344432 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:30.352641 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a8e1ced7-599f-4352-99ad-618da5a62c51 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.161381 Oct 15 21:52:30.353427 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2029/6303] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:30 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:30.395771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-399353b4-29d7-4ebb-a851-1855f59da006 req-83e5697e-2881-49a8-b3d0-d1090167d678 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "name": "network-vif-plugged", "status": "completed", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:30.405741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-399353b4-29d7-4ebb-a851-1855f59da006 req-83e5697e-2881-49a8-b3d0-d1090167d678 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:30.406220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-399353b4-29d7-4ebb-a851-1855f59da006 req-83e5697e-2881-49a8-b3d0-d1090167d678 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:30.406653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-399353b4-29d7-4ebb-a851-1855f59da006 req-83e5697e-2881-49a8-b3d0-d1090167d678 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:30.420275 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-399353b4-29d7-4ebb-a851-1855f59da006 req-83e5697e-2881-49a8-b3d0-d1090167d678 service nova] Creating event network-vif-plugged:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:52:30.425389 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-399353b4-29d7-4ebb-a851-1855f59da006 req-83e5697e-2881-49a8-b3d0-d1090167d678 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105536 Oct 15 21:52:30.426037 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4275/6304] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:52:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:52:31.378761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:31.379381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:31.386715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:31.387209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:31.387659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:31.388367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:31.388833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:31.389299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:31.521109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:31.529494 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-892d97b9-60bc-40c7-9112-ec94ca6b73d2 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1358 microversion: 2.1 time: 0.154988 Oct 15 21:52:31.530058 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2030/6305] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:31 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1358 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:32.552235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:32.552941 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:32.560521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.561010 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.561474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.562306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.562858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.563290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.693349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:32.701274 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d855906d-94eb-43d3-b459-27db5f4e8335 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1549 microversion: 2.1 time: 0.153325 Oct 15 21:52:32.701847 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4276/6306] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:32 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1549 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:32.724350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:32.724916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:32.733128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.733633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.734093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.734803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.735280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.735729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.872674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:32.880483 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c16cd213-6f87-45f5-a8d7-474a423952f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1549 microversion: 2.1 time: 0.160144 Oct 15 21:52:32.881082 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2031/6307] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:32 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1549 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:32.904095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:32.908812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:32.909604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Searching by: {'deleted': False, 'project_id': 'cd7071741b2e4f35954da54683a2b55c'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:52:32.914886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.915361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.915795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.916521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.916982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.917434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.941218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:52:32.968479 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-71c0bb3b-90b1-446a-afbe-845c7813e8ef tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.068940 Oct 15 21:52:32.969071 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4277/6308] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:52:32 2022] GET /compute/v2.1/servers => generated 343 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:52:32.980869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:32.981483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:32.987435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.987924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.988371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:32.989094 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:32.989567 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:32.990006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:33.118401 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:33.127199 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ad08f427-2944-4bfd-9905-a818845fb409 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1549 microversion: 2.1 time: 0.150303 Oct 15 21:52:33.127776 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2032/6309] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:32 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1549 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:34.677482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "18c4c53b-8b10-4985-aa75-a4962ba04680"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:34.679887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:34.686902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:34.687395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:34.687864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:34.688569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:34.689064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:34.689507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:34.927346 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6f4d609c-544c-4c27-b6e8-7243cb69bcac tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.253778 Oct 15 21:52:34.927963 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4278/6310] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:52:34 2022] POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments => generated 194 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:52:39.530861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:39.531486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:39.537703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:39.538204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:39.538654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:39.539353 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:39.539820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:39.540285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:39.673897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:39.682782 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0f8fddfa-3e4a-4dfc-b211-91af66098a0e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1595 microversion: 2.1 time: 0.155308 Oct 15 21:52:39.683412 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2033/6311] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:39 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1595 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:41.677150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-175d3e1f-c3a9-43da-b61e-f642b19af34e req-76466200-bfa5-40e1-9bbd-71771f92b39f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:41.687852 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:41.690262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-175d3e1f-c3a9-43da-b61e-f642b19af34e req-76466200-bfa5-40e1-9bbd-71771f92b39f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:41.690396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-175d3e1f-c3a9-43da-b61e-f642b19af34e req-76466200-bfa5-40e1-9bbd-71771f92b39f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:41.690812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-175d3e1f-c3a9-43da-b61e-f642b19af34e req-76466200-bfa5-40e1-9bbd-71771f92b39f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:41.695290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=76,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:41.706158 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-175d3e1f-c3a9-43da-b61e-f642b19af34e req-76466200-bfa5-40e1-9bbd-71771f92b39f service nova] Creating event network-changed:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:52:41.712351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-175d3e1f-c3a9-43da-b61e-f642b19af34e req-76466200-bfa5-40e1-9bbd-71771f92b39f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039208 Oct 15 21:52:41.716206 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4279/6312] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:52:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:52:42.782213 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-2116448610"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:42.782950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:42.786084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:42.791711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:42.792302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:42.792855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:42.793724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:42.794278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:42.794980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:42.798430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=77 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:42.927695 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.network.security_group_api [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Adding security group 71286ace-63b2-45f2-b7d0-7586bf043311 to port 9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 Oct 15 21:52:43.241370 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-06de1155-aee1-41a2-af44-59751526ec49 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action" status: 202 len: 0 microversion: 2.1 time: 0.463124 Oct 15 21:52:43.242034 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2034/6313] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:52:42 2022] POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action => generated 0 bytes in 465 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:52:43.250993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:52:43.251783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:52:43.254903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:52:43.276561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:43.277110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:43.277635 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:43.278650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:43.279128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:43.279567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:43.445246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:52:43.454199 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-03d197b8-e273-4557-82b2-f66c33eb6d5a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1761 microversion: 2.1 time: 0.207574 Oct 15 21:52:43.454788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4280/6314] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:52:43 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1761 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:52:43.733266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-20f89df2-b6d7-47b3-b779-45da4e31b129 req-e36265b8-8116-415f-8119-171d32d7bd7c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:52:43.743207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-20f89df2-b6d7-47b3-b779-45da4e31b129 req-e36265b8-8116-415f-8119-171d32d7bd7c service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:52:43.743779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-20f89df2-b6d7-47b3-b779-45da4e31b129 req-e36265b8-8116-415f-8119-171d32d7bd7c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:52:43.744293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-20f89df2-b6d7-47b3-b779-45da4e31b129 req-e36265b8-8116-415f-8119-171d32d7bd7c service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:52:43.755832 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-20f89df2-b6d7-47b3-b779-45da4e31b129 req-e36265b8-8116-415f-8119-171d32d7bd7c service nova] Creating event network-changed:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:52:43.761069 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-20f89df2-b6d7-47b3-b779-45da4e31b129 req-e36265b8-8116-415f-8119-171d32d7bd7c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037860 Oct 15 21:52:43.761681 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2035/6315] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:52:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:55:36.218722 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [8a327d79-84a5-4eee-8d31-7df1f6cabc7a] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 21:55:36.223436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:55:36.225679 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:55:36.228185 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 21:55:36.228634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:55:36.229374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=7 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:55:36.230386 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:55:36.230528 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:55:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action (ip 10.222.0.16) !!! Oct 15 21:55:36.233171 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 21:55:36.233314 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:55:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action (ip 10.222.0.16) !!! Oct 15 21:55:36.235168 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 21:55:36.235337 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:55:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action (ip 10.222.0.16) !!! Oct 15 21:55:36.245255 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:36.245751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:36.246189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:36.246886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:36.247370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:36.247798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:36.325863 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-21d42582-f225-4f9c-af99-370befa3f500 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action" status: 202 len: 0 microversion: 2.1 time: 0.108778 Oct 15 21:55:36.326517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4281/6316] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:55:36 2022] POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:55:36.334937 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:55:36.336726 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:55:36.337579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:55:36.338351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=5 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:55:36.340094 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 21:55:36.340363 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:55:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab (ip 10.222.0.16) !!! Oct 15 21:55:36.342024 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:55:36.342181 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 21:55:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab (ip 10.222.0.16) !!! Oct 15 21:55:36.351235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:55:36.351950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:55:36.365482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:36.365991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:36.366444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:36.367188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:36.367660 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:36.368099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:36.690815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:55:36.699315 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2e74290e-6fab-4281-ad01-cc683f69c6e0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1763 microversion: 2.1 time: 0.367744 Oct 15 21:55:36.699841 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2036/6317] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:55:36 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1763 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:55:37.725753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:55:37.726462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:55:37.735422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:37.735912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:37.736480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:37.737228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:37.737698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:37.738131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:37.742475 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [8a327d79-84a5-4eee-8d31-7df1f6cabc7a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 45706. Oct 15 21:55:37.745532 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 45708. Oct 15 21:55:37.884690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:55:37.893973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eafd02c2-eea4-42c5-82b5-5f366cbb8d20 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1768 microversion: 2.1 time: 0.174288 Oct 15 21:55:37.894534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4282/6318] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:55:37 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1768 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:55:38.918325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:55:38.918955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:55:38.931689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:38.932190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:38.932637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:38.933385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:55:38.933855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:55:38.934295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:55:38.938554 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 45712. Oct 15 21:55:38.942096 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 45714. Oct 15 21:55:39.084588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:55:39.092931 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6a7c8d45-29ae-495e-8d6e-ba320cb1637b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1761 microversion: 2.1 time: 0.178600 Oct 15 21:55:39.093539 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2037/6319] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:55:38 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1761 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:09.209240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:09.210004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=16,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:09.260500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-2116448610"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:09.261071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:09.267327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:09.267818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:09.268261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:09.268960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:09.269449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:09.269882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:09.452041 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.network.security_group_api [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Removing security group 71286ace-63b2-45f2-b7d0-7586bf043311 from port 9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 Oct 15 21:57:09.755412 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed781bde-ca28-426c-afff-277a23a3f5d5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action" status: 202 len: 0 microversion: 2.1 time: 0.548150 Oct 15 21:57:09.759063 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4283/6320] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:57:09 2022] POST /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/action => generated 0 bytes in 550 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:57:09.782230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:09.796131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:09.797805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:09.800601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:09.802284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:09.803095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:09.823481 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] Creating event network-changed:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:57:09.828025 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] [51a6695d-5500-4af2-b864-07ca9f7c1fbf] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 21:57:10.834875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:10.835508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:10.836608 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] [51a6695d-5500-4af2-b864-07ca9f7c1fbf] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 45784. Oct 15 21:57:10.841297 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-51ed8720-0a98-4fb2-b5f9-21e5b4b9fbe0 req-dde1fb01-b0fc-46ec-a858-a03e869d9d35 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 1.064117 Oct 15 21:57:10.842337 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2038/6321] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 1066 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:10.843086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:10.843704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:10.844288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:10.845274 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:10.845803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:10.846294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:10.932886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lazy-loading 'services' on Instance uuid 688d79b6-9ae1-4618-b48d-f076ec7744ab {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:57:11.017532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e6004bf-8f33-46aa-b290-61dd2eea0c9d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments/18c4c53b-8b10-4985-aa75-a4962ba04680" status: 202 len: 0 microversion: 2.1 time: 0.186822 Oct 15 21:57:11.018147 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4284/6322] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:57:10 2022] DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments/18c4c53b-8b10-4985-aa75-a4962ba04680 => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:57:11.027559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:11.030299 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:11.037655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:11.038458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:11.039258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:11.040238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:11.040695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:11.041330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:11.080237 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef3b10ab-7651-4882-be5c-6b5e8abef9eb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057128 Oct 15 21:57:11.080810 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2039/6323] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:57:11 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:12.094474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:12.096770 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:12.103151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:12.103642 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:12.104084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:12.104780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:12.105265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:12.105698 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:12.135805 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5131b50f-2fa3-4982-a9e3-27f90dd5fd24 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046672 Oct 15 21:57:12.136365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4285/6324] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:57:12 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:57:12.147282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:12.147823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:12.154290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:12.154779 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:12.155229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:12.155926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:12.156394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:12.156833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:12.231569 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] HTTP exception thrown: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab is not attached to volume 18c4c53b-8b10-4985-aa75-a4962ba04680 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Request method failure captured: Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: request: DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments/18c4c53b-8b10-4985-aa75-a4962ba04680 HTTP/1.1 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSyt7U0OhH3xObWcw6SnTJSOfkVFxFjeWlg3ueBoUhcZAvCocXs0-fgfOx7XMTXTu8SgeieBtmOxIVWLLVOH2uUd6y-u6tCs4IQTOUtF6n506wwpUebFOCF4rgEfoHhBdhVLLw1O2pSDR60U7vjr6vFEl69FsNXmlHU9bdUoekftheZc Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4768c0948f1b44689210529a25211aa7 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab is not attached to volume 18c4c53b-8b10-4985-aa75-a4962ba04680 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'server_id': '688d79b6-9ae1-4618-b48d-f076ec7744ab', 'id': '18c4c53b-8b10-4985-aa75-a4962ba04680'} Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 18c4c53b-8b10-4985-aa75-a4962ba04680. Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab is not attached to volume 18c4c53b-8b10-4985-aa75-a4962ba04680 Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:57:12.240902 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab is not attached to volume 18c4c53b-8b10-4985-aa75-a4962ba04680 Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Returning 404 to user: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab is not attached to volume 18c4c53b-8b10-4985-aa75-a4962ba04680 {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-48535637-ab76-43ea-9d85-b582d9da9720 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments/18c4c53b-8b10-4985-aa75-a4962ba04680" status: 404 len: 154 microversion: 2.1 time: 0.099150 Oct 15 21:57:12.246473 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2040/6325] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:57:12 2022] DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab/os-volume_attachments/18c4c53b-8b10-4985-aa75-a4962ba04680 => generated 154 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:57:12.857496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d3c5b506-6e21-4b32-a749-d86f3a23d2c9 req-0a59e85d-27ab-464a-8216-a4efc3506793 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:12.867082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d3c5b506-6e21-4b32-a749-d86f3a23d2c9 req-0a59e85d-27ab-464a-8216-a4efc3506793 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:12.867566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d3c5b506-6e21-4b32-a749-d86f3a23d2c9 req-0a59e85d-27ab-464a-8216-a4efc3506793 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:12.867995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d3c5b506-6e21-4b32-a749-d86f3a23d2c9 req-0a59e85d-27ab-464a-8216-a4efc3506793 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:12.878722 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d3c5b506-6e21-4b32-a749-d86f3a23d2c9 req-0a59e85d-27ab-464a-8216-a4efc3506793 service nova] Creating event network-changed:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:57:12.884100 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d3c5b506-6e21-4b32-a749-d86f3a23d2c9 req-0a59e85d-27ab-464a-8216-a4efc3506793 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030250 Oct 15 21:57:12.884688 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4286/6326] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:13.388146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:13.388590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:13.396550 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:13.397099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:13.397555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:13.398262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:13.398761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:13.399199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:13.423438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:57:13.500266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d2ffb54f-4ce6-43b2-8047-f930941b8edd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 204 len: 0 microversion: 2.1 time: 0.115672 Oct 15 21:57:13.501005 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2041/6327] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:57:13 2022] DELETE /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:57:13.511340 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:13.512048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:13.518576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:13.519068 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:13.519517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:13.520219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:13.520681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:13.521144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:13.655126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:13.663197 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b48c2a2-ebfb-4f67-9b09-b08b4f3d8a8a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1555 microversion: 2.1 time: 0.156904 Oct 15 21:57:13.663812 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4287/6328] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:13 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1555 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:14.688287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:14.689008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:14.697344 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:14.697849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:14.698319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:14.699030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:14.699497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:14.699938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:14.835579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:14.843871 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28663e21-baf4-4999-a7f1-173d98ef1473 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1555 microversion: 2.1 time: 0.159948 Oct 15 21:57:14.844445 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2042/6329] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:14 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1555 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:15.669854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-333035f9-a5aa-4a9e-b6c7-fe0785db4abf req-f5e7cadc-5583-4fa5-953f-f883d3ab1ad1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "688d79b6-9ae1-4618-b48d-f076ec7744ab", "name": "network-vif-deleted", "tag": "9a66f78e-649f-4fd5-bd05-79fbcf7e31f6"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:15.680929 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-333035f9-a5aa-4a9e-b6c7-fe0785db4abf req-f5e7cadc-5583-4fa5-953f-f883d3ab1ad1 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:15.681450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-333035f9-a5aa-4a9e-b6c7-fe0785db4abf req-f5e7cadc-5583-4fa5-953f-f883d3ab1ad1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:15.682090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-333035f9-a5aa-4a9e-b6c7-fe0785db4abf req-f5e7cadc-5583-4fa5-953f-f883d3ab1ad1 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:15.694222 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-333035f9-a5aa-4a9e-b6c7-fe0785db4abf req-f5e7cadc-5583-4fa5-953f-f883d3ab1ad1 service nova] Creating event network-vif-deleted:9a66f78e-649f-4fd5-bd05-79fbcf7e31f6 for instance 688d79b6-9ae1-4618-b48d-f076ec7744ab on nch2-master-1 Oct 15 21:57:15.699295 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-333035f9-a5aa-4a9e-b6c7-fe0785db4abf req-f5e7cadc-5583-4fa5-953f-f883d3ab1ad1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033216 Oct 15 21:57:15.699894 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4288/6330] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:15.868007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:15.868640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:15.875592 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:15.876039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:15.876497 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:15.877237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:15.877711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:15.878182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:15.910541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:15.912064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:15.912524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:15.913004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lazy-loading 'fault' on Instance uuid 688d79b6-9ae1-4618-b48d-f076ec7744ab {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:57:15.958202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:15.966379 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-53b6a9ca-60e6-44ea-bf2c-480581fffb9f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 200 len: 1348 microversion: 2.1 time: 0.103207 Oct 15 21:57:15.966964 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2043/6331] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:15 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 1348 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:16.994565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:16.995269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 688d79b6-9ae1-4618-b48d-f076ec7744ab] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:17.001851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:17.002346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:17.002784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:17.003482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:17.003943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:17.004415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:17.018592 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] HTTP exception thrown: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab could not be found. Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Request method failure captured: Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab HTTP/1.1 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyt7U0OhH3xObWcw6SnTJSOfkVFxFjeWlg3ueBoUhcZAvCocXs0-fgfOx7XMTXTu8SgeieBtmOxIVWLLVOH2uUd6y-u6tCs4IQTOUtF6n506wwpUebFOCF4rgEfoHhBdhVLLw1O2pSDR60U7vjr6vFEl69FsNXmlHU9bdUoekftheZc Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4768c0948f1b44689210529a25211aa7 Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab could not be found. Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '688d79b6-9ae1-4618-b48d-f076ec7744ab'} Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab could not be found. Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab could not be found. Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:57:17.020349 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab could not be found. Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Returning 404 to user: Instance 688d79b6-9ae1-4618-b48d-f076ec7744ab could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c5e753af-1bc3-4e15-939e-c53ffc8cabcb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab" status: 404 len: 111 microversion: 2.1 time: 0.034103 Oct 15 21:57:17.025613 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4289/6332] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:16 2022] GET /compute/v2.1/servers/688d79b6-9ae1-4618-b48d-f076ec7744ab => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:57:17.030975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9a210f58-c9a5-4a2e-8815-5d22528e434a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '.version_select at 0x7fe74ef85b80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:17.056268 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9a210f58-c9a5-4a2e-8815-5d22528e434a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-453138580" status: 202 len: 0 microversion: 2.1 time: 0.029149 Oct 15 21:57:17.056839 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2044/6333] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:57:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-453138580 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:57:26.257885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df430866-9b07-4fba-a627-ff8a28c26617 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'create', calling method: .version_select at 0x7fe74f848dc0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-190349489", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDf10P14qzDH+K44g2gM0FJ20K6W1LLQdsLoViuomWMsIo9Sb4AMGT3yOKYQjwq8GN0guVzHgHGSQf3C4Sno4sul3124vd5v3G9L4HUdVPl7gy2smWSL3ja4GIXoPW8wZQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:26.275626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-df430866-9b07-4fba-a627-ff8a28c26617 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for project cd7071741b2e4f35954da54683a2b55c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:57:26.281159 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-df430866-9b07-4fba-a627-ff8a28c26617 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for user 4768c0948f1b44689210529a25211aa7 and project cd7071741b2e4f35954da54683a2b55c. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:57:26.307892 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df430866-9b07-4fba-a627-ff8a28c26617 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.053922 Oct 15 21:57:26.308497 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4290/6334] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:57:26 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:26.358942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:26.363808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2084908731", "imageRef": "e8524381-13ca-469c-a5b9-f1635df8cb1b", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-190349489", "networks": [{"uuid": "75bdcce5-e24a-477d-a243-69942be01590"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:26.416839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] validate_networks() for [('75bdcce5-e24a-477d-a243-69942be01590', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:57:26.837617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:57:26.838060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:57:26.838551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:57:26.838977 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:57:26.839404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:57:26.848290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:26.848786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:26.849285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:26.850031 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:26.850529 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:26.850972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:26.869211 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for project cd7071741b2e4f35954da54683a2b55c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:57:26.875006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Getting quotas for user 4768c0948f1b44689210529a25211aa7 and project cd7071741b2e4f35954da54683a2b55c. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:57:26.888557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:57:26.891150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e8524381-13ca-469c-a5b9-f1635df8cb1b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:57:26.937012 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9fc35551-b89a-444e-8b21-d683f46af200 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.579783 Oct 15 21:57:26.937743 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2045/6335] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:57:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 581 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 21:57:26.949221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f014963c-eaf9-4ed4-91e4-1b82ac3eeb02 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:26.949845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f014963c-eaf9-4ed4-91e4-1b82ac3eeb02 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:27.005032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f014963c-eaf9-4ed4-91e4-1b82ac3eeb02 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:27.011418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=31,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:27.013287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f014963c-eaf9-4ed4-91e4-1b82ac3eeb02 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1258 microversion: 2.1 time: 0.067884 Oct 15 21:57:27.013888 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4291/6336] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:26 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1258 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:28.037931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:28.038508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:28.044995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:28.045541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:28.045992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:28.046974 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:28.047557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:28.048101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:28.131907 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:28.140169 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-830473e0-ff64-46c1-aeb7-f122aa0dc4ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1318 microversion: 2.1 time: 0.108163 Oct 15 21:57:28.140744 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2046/6337] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:28 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1318 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:29.165375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:29.166173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:29.175754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:29.176252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:29.176697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:29.177515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:29.177961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:29.178397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:29.329967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:29.341751 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0676c381-9b5d-4719-bb81-ee06d89f0f5b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.180032 Oct 15 21:57:29.342350 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4292/6338] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:29 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:30.366692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:30.367504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:30.374923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:30.375645 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:30.376348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:30.377342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:30.378073 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:30.378595 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:30.524825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:30.533100 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fe28749a-5da2-4045-9a3b-1d8fd9e08dbb tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.171098 Oct 15 21:57:30.538966 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2047/6339] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:30 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:31.565447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:31.566054 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:31.573877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:31.574392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:31.574795 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:31.575503 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:31.575967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:31.576401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:31.719928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:31.728344 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2562f334-82ae-4d43-8815-70fc655c16e1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.166711 Oct 15 21:57:31.735241 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4293/6340] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:31 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:32.753376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:32.754149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:32.761185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:32.761698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:32.762264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:32.762976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:32.763646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:32.764359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:32.902183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:32.910969 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a217f18d-bfca-4a3a-a136-b6df68d278cd tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.161403 Oct 15 21:57:32.911553 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2048/6341] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:32 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:33.936333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:33.936945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:33.945626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:33.946112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:33.946634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:33.947293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:33.947753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:33.948176 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:34.087662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:34.095933 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bff7bd52-9847-4a36-845d-4cbc9cfa7973 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.163466 Oct 15 21:57:34.096616 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4294/6342] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:33 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:35.120081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:35.121673 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:35.135004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:35.135516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:35.135966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:35.136689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:35.137183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:35.137625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:35.287395 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:35.295596 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef453f88-58cd-4229-bd70-5598863d6eb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.179589 Oct 15 21:57:35.296573 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2049/6343] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:35 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:36.320114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:36.320930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:36.328096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:36.328590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:36.329069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:36.330030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:36.330497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:36.330927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:36.469519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:36.477661 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed8abe51-426e-4468-b20b-b6f8ca92837c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.161871 Oct 15 21:57:36.478465 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4295/6344] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:36 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:37.501177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:37.501841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:37.504721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:37.505423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:37.511659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:37.512163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:37.512615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:37.513354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:37.513824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:37.514519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:38.063937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:38.072285 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ae56c5b2-2d16-4337-bc62-c2cf14bbf3bf tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.575068 Oct 15 21:57:38.072875 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2050/6345] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:37 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:38.266729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:38.319091 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-85c7295f-8780-48cd-aadd-ce61bc575429 req-2da279ea-bee7-4af8-af88-85b9c1396ee6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:38.329086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-85c7295f-8780-48cd-aadd-ce61bc575429 req-2da279ea-bee7-4af8-af88-85b9c1396ee6 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:38.329842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-85c7295f-8780-48cd-aadd-ce61bc575429 req-2da279ea-bee7-4af8-af88-85b9c1396ee6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:38.330283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-85c7295f-8780-48cd-aadd-ce61bc575429 req-2da279ea-bee7-4af8-af88-85b9c1396ee6 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:38.343884 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-85c7295f-8780-48cd-aadd-ce61bc575429 req-2da279ea-bee7-4af8-af88-85b9c1396ee6 service nova] Creating event network-changed:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:57:38.349074 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-85c7295f-8780-48cd-aadd-ce61bc575429 req-2da279ea-bee7-4af8-af88-85b9c1396ee6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087673 Oct 15 21:57:38.349686 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4296/6346] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:39.096324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:39.096921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:39.104040 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:39.104547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:39.105003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:39.105754 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:39.106226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:39.106671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:39.311998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:39.320370 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6373283f-9dbc-4203-930c-a949a4b8f421 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.228406 Oct 15 21:57:39.320946 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2051/6347] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:39 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:40.344085 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:40.344854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:40.347761 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:40.353995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:40.354486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:40.354928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:40.355629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:40.356092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:40.356522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:40.492180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:40.499860 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9fd04014-59c9-43b1-bd58-1a45848bac54 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.159900 Oct 15 21:57:40.500436 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4297/6348] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:40 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:41.524276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:41.524901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:41.527648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:57:41.533641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:41.534174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:41.534659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:41.535371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:41.535839 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:41.536776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:41.666718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:41.674365 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f8ad381b-5a3f-433d-b5b6-01e1fba12d79 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.154782 Oct 15 21:57:41.674943 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2052/6349] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:41 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:42.698376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:42.699042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:42.706042 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:42.706543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:42.706990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:42.707700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:42.708715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:42.709666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:42.844575 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:42.853419 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-aa8a07ad-2cef-425f-896b-2bd1e2870a2d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.159182 Oct 15 21:57:42.854192 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4298/6350] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:42 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:43.876889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:43.877555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:43.884566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:43.885154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:43.885611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:43.886379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:43.886855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:43.887304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:44.016881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:44.024978 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5f8ad2d0-9555-48c1-8f53-e09293c077f8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.151952 Oct 15 21:57:44.025582 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2053/6351] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:43 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:44.912335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5a5c93cb-ee7a-443d-84a3-bfa9d9def3c2 req-71006c5e-0274-4f8b-9b43-f590a8a1e2d9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "name": "network-vif-plugged", "status": "completed", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:44.922576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5a5c93cb-ee7a-443d-84a3-bfa9d9def3c2 req-71006c5e-0274-4f8b-9b43-f590a8a1e2d9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:44.923074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5a5c93cb-ee7a-443d-84a3-bfa9d9def3c2 req-71006c5e-0274-4f8b-9b43-f590a8a1e2d9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:44.923520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5a5c93cb-ee7a-443d-84a3-bfa9d9def3c2 req-71006c5e-0274-4f8b-9b43-f590a8a1e2d9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:44.935764 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5a5c93cb-ee7a-443d-84a3-bfa9d9def3c2 req-71006c5e-0274-4f8b-9b43-f590a8a1e2d9 service nova] Creating event network-vif-plugged:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:57:44.941567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5a5c93cb-ee7a-443d-84a3-bfa9d9def3c2 req-71006c5e-0274-4f8b-9b43-f590a8a1e2d9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094051 Oct 15 21:57:44.942319 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4299/6352] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:45.050464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:45.051106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:45.057918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:45.058712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:45.059335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:45.060147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:45.060622 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:45.061089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:45.219248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:45.228260 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a7a55d55-71e3-4b0d-ba2c-3963583710f1 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1358 microversion: 2.1 time: 0.182987 Oct 15 21:57:45.228934 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2054/6353] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:45 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1358 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:46.253359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:46.254067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:46.260735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.261272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.261697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.262397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.262857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.263286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.393414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:46.401456 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df8ee0fe-a3a6-411d-93e1-557862a73a28 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1549 microversion: 2.1 time: 0.153180 Oct 15 21:57:46.402035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4300/6354] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:46 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1549 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:46.424236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:46.424873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:46.432415 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.432996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.433483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.434205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.434681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.435129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.567132 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:46.574995 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-346dfa6d-443f-4387-bd44-886bd61c903e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1549 microversion: 2.1 time: 0.154609 Oct 15 21:57:46.575569 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2055/6355] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:46 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1549 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:46.597622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:46.603064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:46.603843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Searching by: {'deleted': False, 'project_id': 'cd7071741b2e4f35954da54683a2b55c'} {{(pid=138688) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 15 21:57:46.609730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.610218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.610659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.611397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.611862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.612294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.635691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.multi_cell_list [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138688) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 15 21:57:46.663990 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf2b6f13-c2c8-4e01-a4ae-cb56afadf3d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.070249 Oct 15 21:57:46.664579 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4301/6356] 10.222.0.16 () {58 vars in 1125 bytes} [Sat Oct 15 21:57:46 2022] GET /compute/v2.1/servers => generated 343 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:46.677118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:46.678194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:46.684222 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.684721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.685207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.685921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:46.686391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:46.686832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:46.819485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:46.827292 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-80d3ab62-baaa-46cc-9498-ee9c439f6f30 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1549 microversion: 2.1 time: 0.154197 Oct 15 21:57:46.827863 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2056/6357] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:46 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1549 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:48.350854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:48.353232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:48.360101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:48.360576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:48.361020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:48.361757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:48.362217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:48.362654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:48.600180 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58f82c32-0030-44ab-a157-6dfa6bb7b4b5 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.252246 Oct 15 21:57:48.600967 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4302/6358] 10.222.0.16 () {60 vars in 1323 bytes} [Sat Oct 15 21:57:48 2022] POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments => generated 194 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:51.968244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:51.968885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:51.975965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:51.976472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:51.976929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:51.977674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:51.978181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:51.978630 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:52.111024 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:52.119936 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f548d8d9-19cb-483a-a09f-1c977dae915f tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1595 microversion: 2.1 time: 0.155686 Oct 15 21:57:52.120541 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2057/6359] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:51 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1595 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:53.755972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-9985de6e-c930-4a81-9753-af6b7d2c5925 req-f10134ea-4c8c-458d-abb5-f7d2d648b8e7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:53.767185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9985de6e-c930-4a81-9753-af6b7d2c5925 req-f10134ea-4c8c-458d-abb5-f7d2d648b8e7 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:53.767715 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9985de6e-c930-4a81-9753-af6b7d2c5925 req-f10134ea-4c8c-458d-abb5-f7d2d648b8e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:53.768232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-9985de6e-c930-4a81-9753-af6b7d2c5925 req-f10134ea-4c8c-458d-abb5-f7d2d648b8e7 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:53.780488 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-9985de6e-c930-4a81-9753-af6b7d2c5925 req-f10134ea-4c8c-458d-abb5-f7d2d648b8e7 service nova] Creating event network-changed:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:57:53.785606 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-9985de6e-c930-4a81-9753-af6b7d2c5925 req-f10134ea-4c8c-458d-abb5-f7d2d648b8e7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035254 Oct 15 21:57:53.789689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4303/6360] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:57:54.756443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-2004460526"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:54.757009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:54.764643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:54.765199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:54.765661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:54.766422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:54.766904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:54.767351 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:54.907148 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.network.security_group_api [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Adding security group c1e224bd-025d-4283-a30c-f930ee97bb50 to port cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 Oct 15 21:57:55.278994 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0892e97d-c80e-448d-afab-037656f1cbb8 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action" status: 202 len: 0 microversion: 2.1 time: 0.527753 Oct 15 21:57:55.282658 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2058/6361] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:57:54 2022] POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action => generated 0 bytes in 530 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:57:55.290151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:57:55.290857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:57:55.310476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:55.311518 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:55.312020 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:55.312748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:55.313249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:55.314648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:55.562576 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:57:55.571974 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4877e9ca-f313-4786-b1ca-f4535b2bf45a tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1759 microversion: 2.1 time: 0.287787 Oct 15 21:57:55.572964 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4304/6362] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:57:55 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1759 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:57:55.806558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-ff8cfc7e-29d0-43d0-bc59-9bc356f19870 req-7ab7ac19-05b1-4768-8d07-f2724ff14e7a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:57:55.819625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ff8cfc7e-29d0-43d0-bc59-9bc356f19870 req-7ab7ac19-05b1-4768-8d07-f2724ff14e7a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:57:55.820126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ff8cfc7e-29d0-43d0-bc59-9bc356f19870 req-7ab7ac19-05b1-4768-8d07-f2724ff14e7a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:57:55.820570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-ff8cfc7e-29d0-43d0-bc59-9bc356f19870 req-7ab7ac19-05b1-4768-8d07-f2724ff14e7a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:57:55.834045 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-ff8cfc7e-29d0-43d0-bc59-9bc356f19870 req-7ab7ac19-05b1-4768-8d07-f2724ff14e7a service nova] Creating event network-changed:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:57:55.839285 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-ff8cfc7e-29d0-43d0-bc59-9bc356f19870 req-7ab7ac19-05b1-4768-8d07-f2724ff14e7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037825 Oct 15 21:57:55.839885 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2059/6363] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:57:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:58:19.932624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:58:19.934875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:19.937376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:58:19.938143 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=15 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:58:19.939014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:58:19.945731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:19.946218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:19.946659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:19.947382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:19.947836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:19.948261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:20.020873 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4f577fd6-7970-4a65-bfe9-3d3de866a9f0 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action" status: 202 len: 0 microversion: 2.1 time: 0.091591 Oct 15 21:58:20.021785 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4305/6364] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:58:19 2022] POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action => generated 0 bytes in 94 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:58:20.028593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=15 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:58:20.029715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=80 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:58:20.033767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:20.034492 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:20.042349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:20.042932 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:20.043358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:20.044071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:20.044617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:20.045178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:20.289269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:20.297582 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e4431303-05aa-45c7-a0fa-d304096ac867 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1751 microversion: 2.1 time: 0.271312 Oct 15 21:58:20.298158 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2060/6365] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:20 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1751 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:21.323438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:21.324165 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:21.330304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:21.330800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:21.331222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:21.331908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:21.332373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:21.332801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:21.475097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:21.483418 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c602d5b9-3e65-4c23-b903-bc16007850d4 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.165211 Oct 15 21:58:21.483989 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4306/6366] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:21 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:22.508180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:22.508840 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:22.515969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:22.516474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:22.516923 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:22.517662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:22.518131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:22.518572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:22.665311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:22.674014 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-233ae1d3-408d-4923-b565-223d17c7d850 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.170303 Oct 15 21:58:22.674594 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2061/6367] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:22 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:23.698216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:23.698829 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:23.705790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:23.706270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:23.706706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:23.707890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:23.708603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:23.709060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:23.851480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:23.859623 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de4e89a1-e256-4b8e-80ad-fe2d0b41173e tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.165316 Oct 15 21:58:23.860187 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4307/6368] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:23 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:24.883643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:24.884259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:24.891513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:24.892039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:24.892487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:24.893224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:24.893696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:24.894139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:25.042317 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:25.051795 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b80085e0-67ad-474d-8bcf-dd17c9b42763 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.171863 Oct 15 21:58:25.052415 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2062/6369] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:24 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:26.075800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:26.076491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:26.084008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:26.084492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:26.084937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:26.085663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:26.086123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:26.086556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:26.233972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:26.242626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-567631fb-fc10-47b4-973d-1f62f20edbfa tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.170488 Oct 15 21:58:26.243215 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4308/6370] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:26 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:27.265949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:27.266575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:27.274060 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:27.274557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:27.275006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:27.275712 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:27.276183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:27.276623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:27.424612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:27.433304 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4ac30eb9-6d6f-4344-b3c1-b262d18cd423 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.171441 Oct 15 21:58:27.433890 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2063/6371] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:27 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:28.455777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:28.456307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:28.464399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:28.464884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:28.465357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:28.466055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:28.466515 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:28.466947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:28.612474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:28.620876 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a3d8054e-5e13-433a-92f0-3869169411ca tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.169077 Oct 15 21:58:28.621486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4309/6372] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:28 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:29.645245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:29.645873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:29.652710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:29.653250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:29.653703 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:29.654406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:29.654870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:29.655307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:29.794864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:29.803735 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2283e9e3-d3bc-465a-8fd0-75771c1c807b tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.162621 Oct 15 21:58:29.804306 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2064/6373] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:29 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:30.827339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:30.827965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:30.838562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:30.839056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:30.839500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:30.840206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:30.840794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:30.841313 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:30.985773 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:30.994086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-55b2ffb9-7e34-4f0e-adc2-6f71245f64d6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.170611 Oct 15 21:58:30.994668 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4310/6374] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:30 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:32.018679 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:32.019300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:32.026746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:32.027245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:32.027698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:32.028412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:32.028881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:32.029354 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:32.173471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:32.182394 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4d3724f8-8c19-4eec-b763-993a1f688327 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.167679 Oct 15 21:58:32.182960 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2065/6375] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:32 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:33.206306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:33.207027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:33.214187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:33.214676 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:33.215120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:33.215828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:33.216291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:33.216753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:33.362818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:33.371797 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-446338d4-5ff3-48b9-ab5d-7c4c3f1421a6 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.169362 Oct 15 21:58:33.372486 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4311/6376] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:33 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:34.395778 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:34.396433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:34.403350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:34.403845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:34.404291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:34.404993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:34.405517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:34.405958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:34.551611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:34.560038 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-edf3b4e6-10f8-41ce-a55a-d12e42135e2c tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.168219 Oct 15 21:58:34.560607 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2066/6377] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:34 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:35.584169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:35.584711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:35.591631 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:35.592124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:35.592573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:35.593311 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:35.593780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:35.594216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:35.739004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:35.747406 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6681c5f4-131f-49ab-af4b-c61dd9146d52 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1756 microversion: 2.1 time: 0.167759 Oct 15 21:58:35.747979 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4312/6378] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:35 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1756 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:58:36.771417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:58:36.772049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:58:36.778970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:36.779474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:36.779929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:36.780640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:58:36.781143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:58:36.781588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:58:36.932356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:58:36.940773 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28b1b727-9b2e-4ae2-b53d-4a20b956e1ae tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1759 microversion: 2.1 time: 0.173892 Oct 15 21:58:36.941885 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2067/6379] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:58:36 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1759 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:09.106790 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-2004460526"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:09.107465 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:09.109794 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=16 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:09.110507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:09.111651 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 21:59:09.111846 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 21:59:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action (ip 10.222.0.16) !!! Oct 15 21:59:09.120491 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:09.120977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:09.121444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:09.122153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:09.122611 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:09.123043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:09.266192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.network.security_group_api [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Removing security group c1e224bd-025d-4283-a30c-f930ee97bb50 from port cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 Oct 15 21:59:09.568168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4c9eeb3d-e0e5-499c-a2a1-63fff0a53fea tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action" status: 202 len: 0 microversion: 2.1 time: 0.465559 Oct 15 21:59:09.571919 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4313/6380] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 21:59:09 2022] POST /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/action => generated 0 bytes in 467 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 21:59:09.579709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-20317f0b-c2e5-431f-a69a-3ee00425abbd req-eb8c994d-f501-40c7-9b4e-277edf42f15e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:09.588769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:09.589507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=48 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:09.607525 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-20317f0b-c2e5-431f-a69a-3ee00425abbd req-eb8c994d-f501-40c7-9b4e-277edf42f15e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:09.608034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-20317f0b-c2e5-431f-a69a-3ee00425abbd req-eb8c994d-f501-40c7-9b4e-277edf42f15e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:09.608536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-20317f0b-c2e5-431f-a69a-3ee00425abbd req-eb8c994d-f501-40c7-9b4e-277edf42f15e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:09.632318 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-20317f0b-c2e5-431f-a69a-3ee00425abbd req-eb8c994d-f501-40c7-9b4e-277edf42f15e service nova] Creating event network-changed:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:59:09.641754 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-20317f0b-c2e5-431f-a69a-3ee00425abbd req-eb8c994d-f501-40c7-9b4e-277edf42f15e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068003 Oct 15 21:59:09.642514 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2068/6381] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:59:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:59:10.898213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:10.898721 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:10.904777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:10.905302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:10.905771 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:10.906494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:10.906959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:10.907397 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:10.991593 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lazy-loading 'services' on Instance uuid 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:59:11.066972 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8df89fb0-e995-4454-8825-5cb4316ab437 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments/c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6" status: 202 len: 0 microversion: 2.1 time: 0.173769 Oct 15 21:59:11.067565 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4314/6382] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:59:10 2022] DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments/c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:59:11.077606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:11.080069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:11.087952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:11.088466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:11.088931 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:11.089697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:11.090169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:11.090608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:11.124122 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c8b979d3-70e6-4e2e-ba5f-8598ad24ec67 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051665 Oct 15 21:59:11.124973 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2069/6383] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:59:11 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:59:11.666568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-07fc761e-0b42-45f8-8d12-9286f77ee311 req-08a0cc85-48db-4a6a-bd18-ac12ce759591 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:11.677728 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-07fc761e-0b42-45f8-8d12-9286f77ee311 req-08a0cc85-48db-4a6a-bd18-ac12ce759591 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:11.678675 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-07fc761e-0b42-45f8-8d12-9286f77ee311 req-08a0cc85-48db-4a6a-bd18-ac12ce759591 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:11.679169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-07fc761e-0b42-45f8-8d12-9286f77ee311 req-08a0cc85-48db-4a6a-bd18-ac12ce759591 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:11.693699 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-07fc761e-0b42-45f8-8d12-9286f77ee311 req-08a0cc85-48db-4a6a-bd18-ac12ce759591 service nova] Creating event network-changed:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:59:11.698763 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-07fc761e-0b42-45f8-8d12-9286f77ee311 req-08a0cc85-48db-4a6a-bd18-ac12ce759591 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036755 Oct 15 21:59:11.699375 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4315/6384] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:59:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:59:12.137590 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:12.140122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:12.147793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:12.148416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:12.148870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:12.150293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:12.150293 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:12.150686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:12.183213 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce5bb6df-52ca-4020-998f-5f1a1fc0ac61 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049667 Oct 15 21:59:12.183790 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2070/6385] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Oct 15 21:59:12 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 15 21:59:12.197594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:12.197594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:12.206710 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:12.207208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:12.207653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:12.208371 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:12.208833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:12.209314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:12.285606 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] HTTP exception thrown: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 is not attached to volume c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Request method failure captured: Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: request: DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments/c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 HTTP/1.1 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyt7U0OhH3xObWcw6SnTJSOfkVFxFjeWlg3ueBoUhcZAvCocXs0-fgfOx7XMTXTu8SgeieBtmOxIVWLLVOH2uUd6y-u6tCs4IQTOUtF6n506wwpUebFOCF4rgEfoHhBdhVLLw1O2pSDR60U7vjr6vFEl69FsNXmlHU9bdUoekftheZc Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4768c0948f1b44689210529a25211aa7 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 is not attached to volume c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'server_id': '1b8041e4-2306-46fe-9c3e-8d6edc9869c6', 'id': 'c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6'} Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6. Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 is not attached to volume c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:59:12.287563 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 is not attached to volume c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Returning 404 to user: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 is not attached to volume c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0c49e97-c709-446b-b98e-33aaf63b0271 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments/c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6" status: 404 len: 154 microversion: 2.1 time: 0.097368 Oct 15 21:59:12.292407 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4316/6386] 10.222.0.16 () {58 vars in 1416 bytes} [Sat Oct 15 21:59:12 2022] DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6/os-volume_attachments/c97bd3c5-9e5f-41b5-8c8b-055459c8ffd6 => generated 154 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:59:13.432454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:13.432996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:13.440339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:13.441137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:13.441599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:13.442323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:13.442831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:13.443282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:13.469200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 21:59:13.543606 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e422704b-52b0-44ab-95d0-0fa3223c379d tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 204 len: 0 microversion: 2.1 time: 0.114696 Oct 15 21:59:13.544304 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2071/6387] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 21:59:13 2022] DELETE /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 21:59:13.553098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:13.553729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:13.561346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:13.561837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:13.562286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:13.562989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:13.563459 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:13.563899 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:13.696837 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:13.704755 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67ca720b-a899-4824-b6e1-ba028cc37536 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1555 microversion: 2.1 time: 0.155821 Oct 15 21:59:13.705357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4317/6388] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:13 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1555 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:14.729251 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:14.729952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:14.737692 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:14.738190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:14.738643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:14.739350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:14.739821 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:14.740266 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:14.866948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:14.875019 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8a4071ff-9bc9-41ad-bbfa-98ea1c532374 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1555 microversion: 2.1 time: 0.150155 Oct 15 21:59:14.875614 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2072/6389] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:14 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1555 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:15.614372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-cee64871-3c08-4207-9896-6e3b3652fa5d req-7e7aa26d-1149-4534-a95e-a7a7fab15d01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b8041e4-2306-46fe-9c3e-8d6edc9869c6", "name": "network-vif-deleted", "tag": "cd2e32c1-3388-4fd6-b7ef-8fea4081abf9"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:15.624951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cee64871-3c08-4207-9896-6e3b3652fa5d req-7e7aa26d-1149-4534-a95e-a7a7fab15d01 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:15.626015 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cee64871-3c08-4207-9896-6e3b3652fa5d req-7e7aa26d-1149-4534-a95e-a7a7fab15d01 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:15.626799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-cee64871-3c08-4207-9896-6e3b3652fa5d req-7e7aa26d-1149-4534-a95e-a7a7fab15d01 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:15.644402 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-cee64871-3c08-4207-9896-6e3b3652fa5d req-7e7aa26d-1149-4534-a95e-a7a7fab15d01 service nova] Creating event network-vif-deleted:cd2e32c1-3388-4fd6-b7ef-8fea4081abf9 for instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 on nch1-master-1 Oct 15 21:59:15.649197 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-cee64871-3c08-4207-9896-6e3b3652fa5d req-7e7aa26d-1149-4534-a95e-a7a7fab15d01 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039102 Oct 15 21:59:15.650007 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4318/6390] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:59:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:59:15.900167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:15.900902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:15.908129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:15.908657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:15.909410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:15.910191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:15.910676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:15.911128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:15.944820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:15.944980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:15.945465 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:15.945947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lazy-loading 'fault' on Instance uuid 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 21:59:15.987857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4768c0948f1b44689210529a25211aa7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd7071741b2e4f35954da54683a2b55c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:15.996266 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bd4ae330-d27b-4d04-b2b1-ef36808ec895 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 200 len: 1348 microversion: 2.1 time: 0.100040 Oct 15 21:59:15.997065 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2073/6391] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:15 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 1348 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:17.022150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:17.022854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] [instance: 1b8041e4-2306-46fe-9c3e-8d6edc9869c6] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:17.029276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:17.029798 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:17.030244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:17.030944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:17.031408 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:17.031840 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:17.045535 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] HTTP exception thrown: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 could not be found. Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Request method failure captured: Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 HTTP/1.1 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSyt7U0OhH3xObWcw6SnTJSOfkVFxFjeWlg3ueBoUhcZAvCocXs0-fgfOx7XMTXTu8SgeieBtmOxIVWLLVOH2uUd6y-u6tCs4IQTOUtF6n506wwpUebFOCF4rgEfoHhBdhVLLw1O2pSDR60U7vjr6vFEl69FsNXmlHU9bdUoekftheZc Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cd7071741b2e4f35954da54683a2b55c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: cd7071741b2e4f35954da54683a2b55c Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestMinimumBasicScenario-195392836 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4768c0948f1b44689210529a25211aa7 Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestMinimumBasicScenario-195392836-project Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 could not be found. Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '1b8041e4-2306-46fe-9c3e-8d6edc9869c6'} Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 could not be found. Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 could not be found. Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 21:59:17.047140 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 could not be found. Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Returning 404 to user: Instance 1b8041e4-2306-46fe-9c3e-8d6edc9869c6 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0f49da11-8828-47c3-bf51-6971ffb53a78 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6" status: 404 len: 111 microversion: 2.1 time: 0.032151 Oct 15 21:59:17.056193 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4319/6392] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:17 2022] GET /compute/v2.1/servers/1b8041e4-2306-46fe-9c3e-8d6edc9869c6 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 21:59:17.057955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7f547930-c77b-4f14-af81-d7c62d3fc957 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] Calling method '.version_select at 0x7fe74ecff1f0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:17.078284 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7f547930-c77b-4f14-af81-d7c62d3fc957 tempest-TestMinimumBasicScenario-195392836 tempest-TestMinimumBasicScenario-195392836-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-190349489" status: 202 len: 0 microversion: 2.1 time: 0.024118 Oct 15 21:59:17.078869 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2074/6393] 10.222.0.16 () {58 vars in 1269 bytes} [Sat Oct 15 21:59:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-190349489 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 21:59:26.158078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-32102f07-4e2e-4c4e-b055-50ffb53a69ca tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: .version_select at 0x7fe74e12e670>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1236102919", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBESuGWs931keu5TrgKU1ry2Be5rXGEKBlakB5e1jBacoH8/4qOEca+KPpL5wvCK6pj+UaJLzaH40AIsMZO+EBTQWcwN6jBV7AjYcAJ22Mq294GvmkH142dYXM0rXaxJJvQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:26.174310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-32102f07-4e2e-4c4e-b055-50ffb53a69ca tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:59:26.179489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-32102f07-4e2e-4c4e-b055-50ffb53a69ca tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:59:26.205329 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-32102f07-4e2e-4c4e-b055-50ffb53a69ca tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.214595 Oct 15 21:59:26.205973 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4320/6394] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 21:59:25 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:59:32.187587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:32.188566 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:32.195411 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1889066653", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "28cff4a9-3f6d-48f2-9ad1-2aa0c87f15e8"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1236102919", "security_groups": [{"name": "tempest-secgroup-smoke-675236494"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:32.336120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] validate_networks() for [('28cff4a9-3f6d-48f2-9ad1-2aa0c87f15e8', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 21:59:32.761559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 21:59:32.762006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 21:59:32.762560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 21:59:32.763179 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 21:59:32.764158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 21:59:32.775065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:32.775584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:32.776045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:32.776847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:32.777320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:32.777767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:32.799305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 21:59:32.805825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 21:59:32.819868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 21:59:32.822448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 21:59:32.879649 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-29ef5517-8dce-4345-811a-e420ff28def3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.694878 Oct 15 21:59:32.880318 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2075/6395] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 21:59:32 2022] POST /compute/v2.1/servers => generated 405 bytes in 696 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 15 21:59:32.889757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=31,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:32.895205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1f9185c9-4b2e-4a45-b36c-59d989b473e3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:32.896067 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1f9185c9-4b2e-4a45-b36c-59d989b473e3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:32.962386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1f9185c9-4b2e-4a45-b36c-59d989b473e3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:32.969588 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1f9185c9-4b2e-4a45-b36c-59d989b473e3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1267 microversion: 2.1 time: 0.081647 Oct 15 21:59:32.970182 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4321/6396] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:32 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1267 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:34.002340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:34.002980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:34.020765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:34.021707 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:34.022364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:34.023261 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:34.025331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:34.027991 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:34.129204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:34.138911 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-af7ad57b-0eb3-45c2-b43c-1e085f913cd9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1319 microversion: 2.1 time: 0.148115 Oct 15 21:59:34.140334 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2076/6397] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:33 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:35.168177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:35.168641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:35.180535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:35.181075 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:35.181764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:35.182498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:35.182962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:35.183393 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:35.392884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:35.401169 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8682ec15-6d56-4a6d-a54e-511cd7665031 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1392 microversion: 2.1 time: 0.237105 Oct 15 21:59:35.418585 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4322/6398] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:35 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1392 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:35.737807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c4fa017b-923d-4b1d-a290-a8e6e89d9640 req-4c537914-4af3-4859-907d-8e8b64575438 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43b238d4-ded4-4985-95bd-2b8b215b5d9c", "tag": "8d8d9ea7-c216-4d99-9d29-50bcd2129111"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:35.750356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c4fa017b-923d-4b1d-a290-a8e6e89d9640 req-4c537914-4af3-4859-907d-8e8b64575438 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:35.751235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c4fa017b-923d-4b1d-a290-a8e6e89d9640 req-4c537914-4af3-4859-907d-8e8b64575438 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:35.752508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c4fa017b-923d-4b1d-a290-a8e6e89d9640 req-4c537914-4af3-4859-907d-8e8b64575438 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:35.773482 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c4fa017b-923d-4b1d-a290-a8e6e89d9640 req-4c537914-4af3-4859-907d-8e8b64575438 service nova] Creating event network-changed:8d8d9ea7-c216-4d99-9d29-50bcd2129111 for instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c on nch2-master-1 Oct 15 21:59:35.778742 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c4fa017b-923d-4b1d-a290-a8e6e89d9640 req-4c537914-4af3-4859-907d-8e8b64575438 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047180 Oct 15 21:59:35.779358 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2077/6399] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:59:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:59:36.426545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:36.427153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:36.434099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:36.434577 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:36.435011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:36.435700 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:36.436161 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:36.436612 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:36.574594 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:36.583049 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-58fc639b-7e27-413e-83c7-96b62e55d122 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1392 microversion: 2.1 time: 0.160392 Oct 15 21:59:36.583620 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4323/6400] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:36 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1392 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:37.607433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:37.608158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:37.614437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:37.614940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:37.615397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:37.616115 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:37.616705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:37.617184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:37.750105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:37.758535 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b3996400-4d9e-4599-9125-6fcd348c85f1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1392 microversion: 2.1 time: 0.156161 Oct 15 21:59:37.759101 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2078/6401] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:37 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1392 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:38.782270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:38.782887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:38.789186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:38.789674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:38.790111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:38.790808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:38.791267 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:38.791691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:38.926269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:38.934081 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-2aaee078-1b52-4474-9520-98af02c89573 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1392 microversion: 2.1 time: 0.155832 Oct 15 21:59:38.934673 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4324/6402] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:38 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1392 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:39.960996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:39.961657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:39.967808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:39.968301 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:39.968753 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:39.969491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:39.969963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:39.970398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:40.106726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:40.114662 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-f60aee0a-bafe-4521-a81f-44ab975e7044 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1392 microversion: 2.1 time: 0.157842 Oct 15 21:59:40.115231 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2079/6403] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:39 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1392 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:40.372919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-6ca1432a-60a6-44c4-88f0-1314d615e3cd req-bc701b7f-03c9-4029-8e04-61f1983d2ec5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43b238d4-ded4-4985-95bd-2b8b215b5d9c", "name": "network-vif-plugged", "status": "completed", "tag": "8d8d9ea7-c216-4d99-9d29-50bcd2129111"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:40.379620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:40.385923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ca1432a-60a6-44c4-88f0-1314d615e3cd req-bc701b7f-03c9-4029-8e04-61f1983d2ec5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:40.386424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ca1432a-60a6-44c4-88f0-1314d615e3cd req-bc701b7f-03c9-4029-8e04-61f1983d2ec5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:40.386864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-6ca1432a-60a6-44c4-88f0-1314d615e3cd req-bc701b7f-03c9-4029-8e04-61f1983d2ec5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:40.400315 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-6ca1432a-60a6-44c4-88f0-1314d615e3cd req-bc701b7f-03c9-4029-8e04-61f1983d2ec5 service nova] Creating event network-vif-plugged:8d8d9ea7-c216-4d99-9d29-50bcd2129111 for instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c on nch2-master-1 Oct 15 21:59:40.405437 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-6ca1432a-60a6-44c4-88f0-1314d615e3cd req-bc701b7f-03c9-4029-8e04-61f1983d2ec5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036268 Oct 15 21:59:40.406040 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4325/6404] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:59:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 21:59:41.137156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:41.137793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:41.145386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:41.145893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:41.146350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:41.147062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:41.147534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:41.148010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:41.286728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:41.294554 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70950a9e-9ea1-4746-8629-141e7c514373 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1392 microversion: 2.1 time: 0.161397 Oct 15 21:59:41.295134 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2080/6405] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:41 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1392 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:42.319496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:42.320198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:42.326395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:42.326880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:42.327321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:42.328013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:42.328472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:42.328898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:42.468720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:42.476741 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ef0aa4b1-d6b1-42f6-8b7a-3d414289d131 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1564 microversion: 2.1 time: 0.162893 Oct 15 21:59:42.477503 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4326/6406] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:42 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1564 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 21:59:42.501345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:42.501983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:42.508700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:42.509279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:42.509750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:42.510516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:42.511000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:42.511454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:42.645659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:42.653333 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5bd35742-f896-441b-a975-609b5533124a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1564 microversion: 2.1 time: 0.156751 Oct 15 21:59:42.653897 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2081/6407] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:42 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1564 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 21:59:44.572464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 21:59:44.573819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 21:59:44.575733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8e82dac2-1b05-48b3-93a8-3035b93025ce req-1ad16c2a-391b-4c96-ba36-26fc17a1907d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43b238d4-ded4-4985-95bd-2b8b215b5d9c", "tag": "8d8d9ea7-c216-4d99-9d29-50bcd2129111"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 21:59:44.577130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:44.583499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:44.583733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 21:59:44.583996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:44.584538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:44.585007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:44.585902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:44.586390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:44.586858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:44.589776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8e82dac2-1b05-48b3-93a8-3035b93025ce req-1ad16c2a-391b-4c96-ba36-26fc17a1907d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 21:59:44.590280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8e82dac2-1b05-48b3-93a8-3035b93025ce req-1ad16c2a-391b-4c96-ba36-26fc17a1907d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 21:59:44.590730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8e82dac2-1b05-48b3-93a8-3035b93025ce req-1ad16c2a-391b-4c96-ba36-26fc17a1907d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 21:59:44.603917 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8e82dac2-1b05-48b3-93a8-3035b93025ce req-1ad16c2a-391b-4c96-ba36-26fc17a1907d service nova] Creating event network-changed:8d8d9ea7-c216-4d99-9d29-50bcd2129111 for instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c on nch2-master-1 Oct 15 21:59:44.609294 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8e82dac2-1b05-48b3-93a8-3035b93025ce req-1ad16c2a-391b-4c96-ba36-26fc17a1907d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037084 Oct 15 21:59:44.610173 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2082/6408] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 21:59:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 21:59:44.837286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 21:59:44.845994 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52af38bb-4a82-4f9e-a12c-a01e3b6002cb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1564 microversion: 2.1 time: 0.277471 Oct 15 21:59:44.846589 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4327/6409] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 21:59:44 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1564 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:16.137635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:16.138711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=36 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:16.143368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:16.146489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:16.154272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:16.154771 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:16.155224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:16.155926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:16.156392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:16.156831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:16.209203 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-005b2166-d5dd-401e-9a17-c4f2fba1694d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action" status: 200 len: 24567 microversion: 2.1 time: 0.073161 Oct 15 22:00:16.210112 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2083/6410] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:00:16 2022] POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action => generated 24567 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:00:46.655586 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=5 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:46.656396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:46.661377 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:46.661966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:46.668865 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:46.669386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:46.669836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:46.670553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:46.671023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:46.671609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:46.757095 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bc98d125-1e3a-44fc-a6a0-3e2a2b8b2596 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action" status: 202 len: 0 microversion: 2.1 time: 0.103136 Oct 15 22:00:46.757930 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4328/6411] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:00:46 2022] POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:00:46.771570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:46.772477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:46.775799 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:46.776664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:00:46.783258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:46.783770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:46.784228 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:46.784963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:46.785491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:46.786205 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:47.029428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:47.037716 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4e532c5a-41b6-437c-9959-23f3c939ab6f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1686 microversion: 2.1 time: 0.274299 Oct 15 22:00:47.038396 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2084/6412] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:46 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1686 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:48.067188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:48.067862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:48.074220 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:48.074706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:48.075145 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:48.075845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:48.076300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:48.076726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:48.220245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:48.228509 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-78b4c4e8-bcc5-4a55-8a5f-563b3ea6ebaf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1666 microversion: 2.1 time: 0.166957 Oct 15 22:00:48.229090 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4329/6413] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:48 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1666 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:49.327920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:49.330177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:49.338946 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:49.339448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:49.339919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:49.340644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:49.341176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:49.341627 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:49.400006 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4c9aede-0763-4582-a444-8e9eac94b88a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action" status: 200 len: 32634 microversion: 2.1 time: 0.075856 Oct 15 22:00:49.400651 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2085/6414] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:00:49 2022] POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action => generated 32634 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 15 22:00:49.412462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:49.413473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:49.421361 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:49.421856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:49.422300 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:49.423019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:49.423481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:49.423912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:49.495326 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eeca167a-99e4-4383-8cbf-f6465950207b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action" status: 202 len: 0 microversion: 2.1 time: 0.086640 Oct 15 22:00:49.495861 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4330/6415] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:00:49 2022] POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:00:49.504574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:49.505332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:49.512335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:49.512835 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:49.513319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:49.514027 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:49.514498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:49.514940 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:49.649002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:49.656800 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aff3ea5a-a952-4707-84e4-fb24b753e36d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1673 microversion: 2.1 time: 0.156516 Oct 15 22:00:49.657391 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2086/6416] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:49 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1673 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:50.683556 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:50.684262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:50.690580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:50.691063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:50.691505 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:50.692697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:50.693180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:50.693608 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:50.823111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:50.830799 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ec5766e-9a99-4085-971f-5e0989160b1b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1681 microversion: 2.1 time: 0.152413 Oct 15 22:00:50.831367 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4331/6417] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:50 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:00:50.932941 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:50.935158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:50.943002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:50.943499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:50.943951 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:50.944659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:50.945157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:50.945600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:50.996742 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-03f1ca5e-be57-4fcc-b913-b42adafa3d86 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action" status: 200 len: 37980 microversion: 2.1 time: 0.067483 Oct 15 22:00:50.997413 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2087/6418] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:00:50 2022] POST /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c/action => generated 37980 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:00:52.023566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:52.024671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:52.025906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-6826360d-9e94-4ae7-b849-78d59c6f2148 req-94fa3878-6693-4b82-a320-c7fccf2d6c56 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43b238d4-ded4-4985-95bd-2b8b215b5d9c", "tag": "8d8d9ea7-c216-4d99-9d29-50bcd2129111"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:52.032921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:52.033868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:52.034494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:52.036092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:52.036692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:52.037167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6826360d-9e94-4ae7-b849-78d59c6f2148 req-94fa3878-6693-4b82-a320-c7fccf2d6c56 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:52.037565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:52.037702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6826360d-9e94-4ae7-b849-78d59c6f2148 req-94fa3878-6693-4b82-a320-c7fccf2d6c56 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:52.038110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-6826360d-9e94-4ae7-b849-78d59c6f2148 req-94fa3878-6693-4b82-a320-c7fccf2d6c56 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:52.050372 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-6826360d-9e94-4ae7-b849-78d59c6f2148 req-94fa3878-6693-4b82-a320-c7fccf2d6c56 service nova] Creating event network-changed:8d8d9ea7-c216-4d99-9d29-50bcd2129111 for instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c on nch2-master-1 Oct 15 22:00:52.056463 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-6826360d-9e94-4ae7-b849-78d59c6f2148 req-94fa3878-6693-4b82-a320-c7fccf2d6c56 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034890 Oct 15 22:00:52.057304 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2088/6419] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:00:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:00:52.066884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:00:52.173591 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95365994-1d75-4dc5-839f-09382ec4eaf0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 204 len: 0 microversion: 2.1 time: 0.154342 Oct 15 22:00:52.174666 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4332/6420] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:00:52 2022] DELETE /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:00:52.185009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:52.186080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:52.193826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:52.194666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:52.195120 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:52.196059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:52.196708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:52.197372 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:52.343310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:52.351899 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-53428963-0c08-422e-9034-cc2f9e59dfb0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1687 microversion: 2.1 time: 0.172404 Oct 15 22:00:52.352481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2089/6421] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:52 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1687 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:53.377012 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:53.378678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:53.389607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:53.390174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:53.390974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:53.391844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:53.392603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:53.393339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:53.613437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:53.622073 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecc48129-43dc-4861-b8d8-38c3240a90d2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1687 microversion: 2.1 time: 0.249554 Oct 15 22:00:53.622657 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4333/6422] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:53 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1687 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:54.288993 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-1d0f31eb-46bb-40d6-801f-1d7efc868468 req-c7e14a5e-6fad-4bb8-bc4b-5d0b00fc7f3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43b238d4-ded4-4985-95bd-2b8b215b5d9c", "name": "network-vif-deleted", "tag": "8d8d9ea7-c216-4d99-9d29-50bcd2129111"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:54.299428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1d0f31eb-46bb-40d6-801f-1d7efc868468 req-c7e14a5e-6fad-4bb8-bc4b-5d0b00fc7f3b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:54.299928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1d0f31eb-46bb-40d6-801f-1d7efc868468 req-c7e14a5e-6fad-4bb8-bc4b-5d0b00fc7f3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:54.300380 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-1d0f31eb-46bb-40d6-801f-1d7efc868468 req-c7e14a5e-6fad-4bb8-bc4b-5d0b00fc7f3b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:54.312235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-1d0f31eb-46bb-40d6-801f-1d7efc868468 req-c7e14a5e-6fad-4bb8-bc4b-5d0b00fc7f3b service nova] Creating event network-vif-deleted:8d8d9ea7-c216-4d99-9d29-50bcd2129111 for instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c on nch2-master-1 Oct 15 22:00:54.318223 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-1d0f31eb-46bb-40d6-801f-1d7efc868468 req-c7e14a5e-6fad-4bb8-bc4b-5d0b00fc7f3b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033470 Oct 15 22:00:54.319360 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2090/6423] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:00:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:00:54.646444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:54.647065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:54.654004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:54.654501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:54.654950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:54.655745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:54.656215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:54.656652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:54.690139 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:54.690638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:54.691088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:54.691553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lazy-loading 'fault' on Instance uuid 43b238d4-ded4-4985-95bd-2b8b215b5d9c {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:00:54.733360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:00:54.742061 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d0082c7-a6c5-4ef2-83ff-f4e6c5d27c06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 200 len: 1357 microversion: 2.1 time: 0.099670 Oct 15 22:00:54.742647 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4334/6424] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:54 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 1357 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:00:55.768916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:55.769599 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 43b238d4-ded4-4985-95bd-2b8b215b5d9c] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:00:55.776578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:55.777106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:55.777613 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:55.778273 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:00:55.778755 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:00:55.779204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:00:55.792786 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] HTTP exception thrown: Instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c could not be found. Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Request method failure captured: Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c HTTP/1.1 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSy09-QfPZ-9CNDIIFxEAKgOvMyzS_BGx2YAooBKnz2qBEmy5yraK8uUQYc8ul_plXjHeNR8-6n07eBm4u9lCM3KGAMnrogPl4-d9DQbXxkTgXgwzrU8kkhbIZtFBlypgzMCEjP9tyrjP6od2c2gsCfU2f50omy2qrKGIa7h0T3j2Xp0 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 823ad7d2979042c188525f5341febdff Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c could not be found. Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': '43b238d4-ded4-4985-95bd-2b8b215b5d9c'} Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c could not be found. Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c could not be found. Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:00:55.794484 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c could not be found. Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Returning 404 to user: Instance 43b238d4-ded4-4985-95bd-2b8b215b5d9c could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28e5ce45-2a1b-449a-bc7d-36e00935dbe2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c" status: 404 len: 111 microversion: 2.1 time: 0.032496 Oct 15 22:00:55.801509 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2091/6425] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:00:55 2022] GET /compute/v2.1/servers/43b238d4-ded4-4985-95bd-2b8b215b5d9c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:00:59.173507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-722d6557-5358-4668-bbb7-2489b63e6bc0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '.version_select at 0x7fe7584d18b0>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:00:59.203287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-722d6557-5358-4668-bbb7-2489b63e6bc0 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1236102919" status: 202 len: 0 microversion: 2.1 time: 0.034008 Oct 15 22:00:59.204184 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4335/6426] 10.222.0.16 () {58 vars in 1284 bytes} [Sat Oct 15 22:00:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1236102919 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:00:59.236456 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df5d7de4-5652-4b24-bfd2-e1c234e8a1d6 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: .version_select at 0x7fe74ed301f0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-429211487", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP8RkuU28FHIHw9q4IXe7JKFehHjk/AuCBxJiWH4XmuDJyU9sMBw5/uH9tCxNrBC/z8ef7YXSFGYGwQVVNuzpTESGSkazmAY71czQzYXCU/Lc2KjuD1mAMvXr2/ZwGfoGA=="}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:00:59.258391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-df5d7de4-5652-4b24-bfd2-e1c234e8a1d6 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:00:59.263723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-df5d7de4-5652-4b24-bfd2-e1c234e8a1d6 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:00:59.288724 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df5d7de4-5652-4b24-bfd2-e1c234e8a1d6 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.056425 Oct 15 22:00:59.289553 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2092/6427] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:00:59 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:01:04.673146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-509855813", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "ba6edbe7-b15f-4a9c-9fe6-cb6c4a8416aa"}], "key_name": "tempest-TestNetworkAdvancedServerOps-429211487", "security_groups": [{"name": "tempest-secgroup-smoke-1732409899"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:01:04.737845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:04.787115 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] validate_networks() for [('ba6edbe7-b15f-4a9c-9fe6-cb6c4a8416aa', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:01:05.292582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:01:05.293101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:01:05.293591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:01:05.294215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:01:05.294645 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:01:05.311666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:05.312555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:05.313516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:05.314544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:05.315225 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:05.315887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:05.339363 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:01:05.346097 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:01:05.365622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:01:05.368204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:01:05.422934 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b5020e6e-69be-407f-8d41-201b1b465e3c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.754509 Oct 15 22:01:05.430771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4336/6428] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:01:04 2022] POST /compute/v2.1/servers => generated 406 bytes in 756 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:01:05.436249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c1b473f6-3616-4327-b473-b18667b57b04 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:05.436890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c1b473f6-3616-4327-b473-b18667b57b04 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:05.551083 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c1b473f6-3616-4327-b473-b18667b57b04 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:05.567844 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c1b473f6-3616-4327-b473-b18667b57b04 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1265 microversion: 2.1 time: 0.135333 Oct 15 22:01:05.583646 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2093/6429] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:05 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1265 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:06.591956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:06.593496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:06.601993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:06.602975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:06.603657 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:06.604553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:06.605269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:06.606029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:06.692036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:06.704218 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9e41a98c-25ee-474a-a73d-33a55c3cd71a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1325 microversion: 2.1 time: 0.116770 Oct 15 22:01:06.705019 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4337/6430] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:06 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1325 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:07.736970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:07.737749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:07.748383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=32 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:07.753319 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:07.755291 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:07.755822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:07.756616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:07.757149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:07.757610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:08.042856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:08.052050 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7b926e24-f876-486e-b191-d401e5c00694 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.327517 Oct 15 22:01:08.052945 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2094/6431] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:07 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:08.223785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-8831827a-4ea7-4ed4-83a5-221c9a6b00f5 req-03216b96-cd16-407d-913e-bb6aab975281 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "57bd4963-31b6-4ae0-ada6-7beaeb6c4566", "tag": "2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:01:08.236768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8831827a-4ea7-4ed4-83a5-221c9a6b00f5 req-03216b96-cd16-407d-913e-bb6aab975281 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:08.237551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8831827a-4ea7-4ed4-83a5-221c9a6b00f5 req-03216b96-cd16-407d-913e-bb6aab975281 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:08.238071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-8831827a-4ea7-4ed4-83a5-221c9a6b00f5 req-03216b96-cd16-407d-913e-bb6aab975281 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:08.251226 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-8831827a-4ea7-4ed4-83a5-221c9a6b00f5 req-03216b96-cd16-407d-913e-bb6aab975281 service nova] Creating event network-changed:2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a for instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 on nch2-master-1 Oct 15 22:01:08.257303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-8831827a-4ea7-4ed4-83a5-221c9a6b00f5 req-03216b96-cd16-407d-913e-bb6aab975281 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037517 Oct 15 22:01:08.257924 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4338/6432] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:01:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:01:09.076350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:09.077141 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:09.084139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:09.084643 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:09.085133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:09.085843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:09.086316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:09.086760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:09.218104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:09.226839 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-88f64a08-124f-41b0-aa19-276b69f33f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.154370 Oct 15 22:01:09.227477 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2095/6433] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:09 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:10.251754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:10.252650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:10.260264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:10.260743 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:10.261210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:10.261902 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:10.262531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:10.262965 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:10.397094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:10.404811 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0490fc2b-6bd3-4ed7-bbd7-ad2543a8e392 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.157459 Oct 15 22:01:10.405413 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4339/6434] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:10 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:11.427972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:11.428628 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:11.435854 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:11.436362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:11.436823 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:11.437575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:11.438059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:11.438511 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:11.578229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:11.586753 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df58390b-b5e0-4546-a8e9-67597935079a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.162867 Oct 15 22:01:11.587352 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2096/6435] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:11 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:01:12.609943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:12.610494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:12.617989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:12.618488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:12.618933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:12.619646 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:12.620109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:12.620543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:12.754072 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:12.762709 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5d694b6c-87e2-4e54-ac98-98241111ed44 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.156778 Oct 15 22:01:12.763300 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4340/6436] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:12 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:13.786811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:13.787349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:13.794307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:13.794818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:13.795276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:13.795995 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:13.796477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:13.796926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:13.931406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:13.939182 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-26487402-e3aa-46f3-bbdd-b3a14c3511a2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.156556 Oct 15 22:01:13.939746 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2097/6437] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:13 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:14.963520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:14.964141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:14.971029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:14.971523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:14.971972 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:14.972682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:14.973330 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:14.974497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:14.995349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e102dac4-1f7a-4418-88d0-a4525021bf44 req-6b483c34-ad72-45a4-8221-50f6822a7436 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "57bd4963-31b6-4ae0-ada6-7beaeb6c4566", "name": "network-vif-plugged", "status": "completed", "tag": "2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:01:15.006797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e102dac4-1f7a-4418-88d0-a4525021bf44 req-6b483c34-ad72-45a4-8221-50f6822a7436 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:15.007611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e102dac4-1f7a-4418-88d0-a4525021bf44 req-6b483c34-ad72-45a4-8221-50f6822a7436 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:15.008070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e102dac4-1f7a-4418-88d0-a4525021bf44 req-6b483c34-ad72-45a4-8221-50f6822a7436 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:15.023601 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e102dac4-1f7a-4418-88d0-a4525021bf44 req-6b483c34-ad72-45a4-8221-50f6822a7436 service nova] Creating event network-vif-plugged:2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a for instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 on nch2-master-1 Oct 15 22:01:15.028963 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e102dac4-1f7a-4418-88d0-a4525021bf44 req-6b483c34-ad72-45a4-8221-50f6822a7436 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038021 Oct 15 22:01:15.029765 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2098/6438] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:01:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:01:15.158921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:15.167642 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e509fb51-b3bc-4e13-8f89-22d2c4e23421 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.208093 Oct 15 22:01:15.168275 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4341/6439] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:14 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:16.188638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:16.194367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:16.195090 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:16.201671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:16.202173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:16.202872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:16.203322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:16.203798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:16.204257 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:16.341480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:16.349426 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9442c474-9147-4871-a511-81cbc109cbc2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1391 microversion: 2.1 time: 0.161724 Oct 15 22:01:16.349990 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2099/6440] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:16 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1391 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:17.371997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:17.372655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:17.374751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:17.375561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:17.376919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:17.383285 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:17.383782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:17.384234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:17.384942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:17.385438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:17.385880 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:17.522739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:17.530918 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-69893909-dbe8-4342-9d20-0e6e2be0d04b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1563 microversion: 2.1 time: 0.162687 Oct 15 22:01:17.531172 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4342/6441] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:17 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1563 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:17.554127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:17.554765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:17.557496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=44,nova_api:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:17.563379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:17.563888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:17.564352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:17.565114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:17.565582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:17.566034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:17.704494 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:17.712972 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4c77b3e1-6e4f-4a44-9cf3-1ff38eef43c3 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1563 microversion: 2.1 time: 0.162767 Oct 15 22:01:17.713577 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2100/6442] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:17 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1563 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:19.305379 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:01:19.306133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:19.312958 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:19.313603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:19.314302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:19.315122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:19.315695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:19.316424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:19.453828 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8fe2c812-0468-434b-8207-003667368a9a req-2a7c4abb-a2aa-4f1c-a439-156aac6fd9bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "57bd4963-31b6-4ae0-ada6-7beaeb6c4566", "tag": "2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:01:19.465068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8fe2c812-0468-434b-8207-003667368a9a req-2a7c4abb-a2aa-4f1c-a439-156aac6fd9bd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:19.465571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8fe2c812-0468-434b-8207-003667368a9a req-2a7c4abb-a2aa-4f1c-a439-156aac6fd9bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:19.466034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8fe2c812-0468-434b-8207-003667368a9a req-2a7c4abb-a2aa-4f1c-a439-156aac6fd9bd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:19.477168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:01:19.478812 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8fe2c812-0468-434b-8207-003667368a9a req-2a7c4abb-a2aa-4f1c-a439-156aac6fd9bd service nova] Creating event network-changed:2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a for instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 on nch2-master-1 Oct 15 22:01:19.484157 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8fe2c812-0468-434b-8207-003667368a9a req-2a7c4abb-a2aa-4f1c-a439-156aac6fd9bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094135 Oct 15 22:01:19.485156 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2101/6443] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:01:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:01:19.490574 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e4d27931-1e79-4bbd-8a98-9b4187225bdc tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1563 microversion: 2.1 time: 0.194409 Oct 15 22:01:19.491176 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4343/6444] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:01:19 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1563 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:01:51.046043 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:01:51.048342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:01:51.050798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:51.051611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=40 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:01:51.058084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:51.058584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:51.059035 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:51.059740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:01:51.060207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:01:51.060646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:01:51.120061 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d6a2e1f3-f676-4c6d-b121-baa900e4a24b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566/action" status: 200 len: 24574 microversion: 2.1 time: 0.078110 Oct 15 22:01:51.120727 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2102/6445] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:01:51 2022] POST /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566/action => generated 24574 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:02:21.586655 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:02:21.587402 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=40 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:02:21.592460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:02:21.594729 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:21.601434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:21.601917 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:21.602356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:21.603046 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:21.603508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:21.603934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:21.676821 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a7690af-4f79-43df-88ed-0c584e4146a4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566/action" status: 202 len: 0 microversion: 2.1 time: 0.091749 Oct 15 22:02:21.677558 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4344/6446] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:02:21 2022] POST /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566/action => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:02:21.687170 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:21.687915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:21.690789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:02:21.691608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:02:21.699150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:21.699676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:21.700347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:21.701124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:21.701608 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:21.702057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:21.933751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:21.941630 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a8d4629b-d4db-4692-8851-66de05b648d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1674 microversion: 2.1 time: 0.258991 Oct 15 22:02:21.942206 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2103/6447] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:21 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1674 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:22.968555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:22.969294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:22.975599 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:22.976092 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:22.976539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:22.977278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:22.977745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:22.978181 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:23.159290 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:23.167646 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c5337320-44a7-45ef-ba64-3a4d5d663191 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.203964 Oct 15 22:02:23.168228 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4345/6448] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:22 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:24.191338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:24.191978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:24.199849 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:24.200362 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:24.200824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:24.201582 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:24.202066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:24.202521 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:24.343984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:24.352041 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-45fbc9be-d969-47a9-a618-1a7e8aad2bf7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.164724 Oct 15 22:02:24.352615 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2104/6449] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:24 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:25.376164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:25.376781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:25.383860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:25.384352 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:25.384797 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:25.385528 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:25.386602 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:25.387209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:25.518448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:25.526507 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3a6ad796-0713-4619-ab05-d00534ffe4ed tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.154264 Oct 15 22:02:25.527087 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4346/6450] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:25 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:26.550345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:26.550916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:26.557957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:26.558561 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:26.559020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:26.559732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:26.560207 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:26.560651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:26.693786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:26.702487 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9a865bfc-3674-498f-9d23-ed96e4be7465 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.156201 Oct 15 22:02:26.703059 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2105/6451] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:26 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:27.726922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:27.727537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:27.735081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:27.735566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:27.736005 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:27.736695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:27.737185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:27.737614 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:27.869175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:27.877192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-31282298-dd45-48e2-9ae7-8fad95278a87 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.154102 Oct 15 22:02:27.877756 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4347/6452] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:27 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:28.902135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:28.902793 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:28.909659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:28.910161 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:28.910612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:28.911316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:28.911785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:28.912225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:29.043347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:29.051164 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8043bb9b-5362-4adf-b37c-3846b2dee9ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.153024 Oct 15 22:02:29.051757 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2106/6453] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:28 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:30.075532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:30.076153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:30.084025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:30.084520 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:30.084966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:30.085707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:30.086173 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:30.086639 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:30.223983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:30.233521 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cf7c8a42-63dc-4279-9cf0-01537ae512ef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.162235 Oct 15 22:02:30.234202 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4348/6454] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:30 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:31.257536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:31.258203 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:31.260401 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:02:31.260543 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 22:02:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 (ip 10.222.0.16) !!! Oct 15 22:02:31.271647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:31.272150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:31.272603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:31.273348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:31.273826 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:31.274275 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:31.407671 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:31.415536 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-571d84b8-4f60-4f36-b38b-b1200274bde1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.161948 Oct 15 22:02:31.416177 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2107/6455] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:31 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:32.440040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:32.440653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:32.447695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:32.448107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:32.449171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:32.449886 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:32.450345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:32.450776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:32.588846 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:32.597210 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8a6dc1d1-504d-4180-8314-02cc1e737e85 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1679 microversion: 2.1 time: 0.161033 Oct 15 22:02:32.597787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4349/6456] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:32 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1679 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:02:33.619914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:02:33.620560 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:02:33.627652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:33.628151 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:33.628607 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:33.629350 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:02:33.629824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:02:33.630269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:02:33.760172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:02:33.767956 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-11361623-9ac3-4a2a-84d0-34725b442f5f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1682 microversion: 2.1 time: 0.152063 Oct 15 22:02:33.768534 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2108/6457] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:02:33 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1682 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:05.992430 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:05.994632 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:05.997400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:05.998134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:06.004284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:06.004777 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:06.005281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:06.005973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:06.006435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:06.006868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:06.069376 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0b2a081e-f7af-49d5-a101-584371d925cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566/action" status: 200 len: 69116 microversion: 2.1 time: 0.081012 Oct 15 22:03:06.070038 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4350/6458] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:03:05 2022] POST /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566/action => generated 69116 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:03:23.318200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:23.320664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:23.321070 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:23.323482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:23.327390 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=12 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:23.328902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=36 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:23.333756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:23.334276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:23.334865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:23.335602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:23.336088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:23.336811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:23.365625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:03:23.390031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-344620d1-a734-41bf-a568-4c072d969172 req-fa495584-bb3d-43d6-a306-3146bd6c5862 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "57bd4963-31b6-4ae0-ada6-7beaeb6c4566", "tag": "2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:23.401952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-344620d1-a734-41bf-a568-4c072d969172 req-fa495584-bb3d-43d6-a306-3146bd6c5862 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:23.402490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-344620d1-a734-41bf-a568-4c072d969172 req-fa495584-bb3d-43d6-a306-3146bd6c5862 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:23.402989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-344620d1-a734-41bf-a568-4c072d969172 req-fa495584-bb3d-43d6-a306-3146bd6c5862 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:23.415464 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-344620d1-a734-41bf-a568-4c072d969172 req-fa495584-bb3d-43d6-a306-3146bd6c5862 service nova] Creating event network-changed:2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a for instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 on nch2-master-1 Oct 15 22:03:23.422496 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-344620d1-a734-41bf-a568-4c072d969172 req-fa495584-bb3d-43d6-a306-3146bd6c5862 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110260 Oct 15 22:03:23.423770 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4351/6459] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:03:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:03:23.459134 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e987e350-7448-4a09-bc09-286dadfbb4b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 204 len: 0 microversion: 2.1 time: 0.151298 Oct 15 22:03:23.461982 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2109/6460] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:03:23 2022] DELETE /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:03:23.471568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:23.472629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:23.479108 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:23.479601 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:23.480071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:23.480774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:23.482003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:23.482531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:23.621244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:23.629341 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c0db90e9-f9b1-4f0b-979b-ae3842efdf5d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1688 microversion: 2.1 time: 0.164004 Oct 15 22:03:23.629924 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4352/6461] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:23 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1688 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:24.655176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:24.655910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:24.662738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:24.663247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:24.663696 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:24.664412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:24.664879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:24.665346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:24.876052 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:24.883977 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c0f6ad67-5fcf-4ec5-b952-8a4d2d2cdab1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1688 microversion: 2.1 time: 0.233410 Oct 15 22:03:24.884554 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2110/6462] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:24 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1688 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:25.684258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-82d106f3-d632-4084-85cb-e233a88831ef req-a13dd946-44bf-47ab-b097-051554c64030 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "57bd4963-31b6-4ae0-ada6-7beaeb6c4566", "name": "network-vif-deleted", "tag": "2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:25.695627 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-82d106f3-d632-4084-85cb-e233a88831ef req-a13dd946-44bf-47ab-b097-051554c64030 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:25.696192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-82d106f3-d632-4084-85cb-e233a88831ef req-a13dd946-44bf-47ab-b097-051554c64030 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:25.696722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-82d106f3-d632-4084-85cb-e233a88831ef req-a13dd946-44bf-47ab-b097-051554c64030 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:25.709686 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-82d106f3-d632-4084-85cb-e233a88831ef req-a13dd946-44bf-47ab-b097-051554c64030 service nova] Creating event network-vif-deleted:2c5d06c9-69fa-43a7-a5bb-7c18b0cb4a2a for instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 on nch2-master-1 Oct 15 22:03:25.714624 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-82d106f3-d632-4084-85cb-e233a88831ef req-a13dd946-44bf-47ab-b097-051554c64030 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034233 Oct 15 22:03:25.716142 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4353/6463] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:03:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:03:25.910172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:25.910798 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:25.917726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:25.918226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:25.919233 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:25.920032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:25.920508 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:25.920952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:25.954082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:25.954603 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:25.955095 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:25.955575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lazy-loading 'fault' on Instance uuid 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:03:25.998185 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:26.007530 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fb29c067-4933-4d2d-a7df-fb9d1f77f64b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 200 len: 1355 microversion: 2.1 time: 0.101199 Oct 15 22:03:26.008116 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2111/6464] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:25 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 1355 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:27.033390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:27.034095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 57bd4963-31b6-4ae0-ada6-7beaeb6c4566] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:27.040342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:27.040835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:27.041309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:27.042008 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:27.042471 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:27.042906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:27.057225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] HTTP exception thrown: Instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 could not be found. Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Request method failure captured: Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 HTTP/1.1 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSy09-QfPZ-9CNDIIFxEAKgOvMyzS_BGx2YAooBKnz2qBEmy5yraK8uUQYc8ul_plXjHeNR8-6n07eBm4u9lCM3KGAMnrogPl4-d9DQbXxkTgXgwzrU8kkhbIZtFBlypgzMCEjP9tyrjP6od2c2gsCfU2f50omy2qrKGIa7h0T3j2Xp0 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 823ad7d2979042c188525f5341febdff Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 could not be found. Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '57bd4963-31b6-4ae0-ada6-7beaeb6c4566'} Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 could not be found. Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 could not be found. Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:03:27.058885 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 could not be found. Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Returning 404 to user: Instance 57bd4963-31b6-4ae0-ada6-7beaeb6c4566 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd97cd37-2f67-4c11-b86f-51e2fb5af9fd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566" status: 404 len: 111 microversion: 2.1 time: 0.032965 Oct 15 22:03:27.068089 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4354/6465] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:27 2022] GET /compute/v2.1/servers/57bd4963-31b6-4ae0-ada6-7beaeb6c4566 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:03:30.117067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-956ddd7a-23a9-440a-b409-c3bdcb7b3109 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '.version_select at 0x7fe74ed30b80>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:30.142632 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-956ddd7a-23a9-440a-b409-c3bdcb7b3109 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-429211487" status: 202 len: 0 microversion: 2.1 time: 0.028973 Oct 15 22:03:30.143584 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2112/6466] 10.222.0.16 () {58 vars in 1281 bytes} [Sat Oct 15 22:03:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-429211487 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:03:30.173738 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-feddfa84-e2c0-49dd-a562-dcef404d60d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: .version_select at 0x7fe74ee5cb80>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1051697365", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHhoT4xBZhESLhzArQVawqA3jbTnED3jg5RaerZJi8mAH+Fh8CLYtTLsnyESsRBN6rj/m1W/DCFoOBf8O/eKnKb10l62PWbtCZoqC0nr6tgUwBlG5GieDxOJfRDppJDOtQ=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:30.192457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-feddfa84-e2c0-49dd-a562-dcef404d60d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:03:30.199359 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-feddfa84-e2c0-49dd-a562-dcef404d60d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:03:30.227384 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-feddfa84-e2c0-49dd-a562-dcef404d60d5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.057764 Oct 15 22:03:30.228022 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4355/6467] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:03:30 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:03:35.702468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-2130969643", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "883f5051-8082-4375-9f77-715ab51b95c3"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1051697365", "security_groups": [{"name": "tempest-secgroup-smoke-1341555379"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:35.848552 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] validate_networks() for [('883f5051-8082-4375-9f77-715ab51b95c3', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:03:36.002212 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:36.315624 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:03:36.316219 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:03:36.316888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:03:36.317664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:03:36.318114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:03:36.327600 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:36.328106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:36.328569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:36.329427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:36.329909 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:36.330359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:36.348658 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:03:36.353975 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:03:36.367453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:03:36.370164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:03:36.420665 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b2e4a824-71a6-4b73-a3fb-d0a9878bb3a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.722886 Oct 15 22:03:36.425208 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2113/6468] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:03:35 2022] POST /compute/v2.1/servers => generated 406 bytes in 728 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:03:36.434399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3af0f59e-5192-4a89-96aa-97189f0d969c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:36.435156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3af0f59e-5192-4a89-96aa-97189f0d969c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:36.493445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3af0f59e-5192-4a89-96aa-97189f0d969c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:36.501363 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3af0f59e-5192-4a89-96aa-97189f0d969c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1267 microversion: 2.1 time: 0.072266 Oct 15 22:03:36.502222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4356/6469] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:36 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1267 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:03:37.527422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:37.529029 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:37.540239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:37.541176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:37.541714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:37.542473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:37.543023 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:37.543863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:37.614000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:37.622205 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-db09dd50-44fb-47ad-bd45-4fae496ca127 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1337 microversion: 2.1 time: 0.099780 Oct 15 22:03:37.623245 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2114/6470] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:37 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1337 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:38.647027 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:38.648513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:38.653960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:38.658189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:38.658686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:38.659128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:38.659836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:38.664877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:38.665526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:38.814151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:38.822086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-623edaa1-11f3-4260-8505-8ad888cdc303 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.179384 Oct 15 22:03:38.822673 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4357/6471] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:38 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:39.110711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-3116da9f-60a7-4c32-adff-8eb2614eb040 req-c4e4a10c-96d0-46d1-997e-b9c5f78f2239 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:39.127382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3116da9f-60a7-4c32-adff-8eb2614eb040 req-c4e4a10c-96d0-46d1-997e-b9c5f78f2239 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:39.127877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3116da9f-60a7-4c32-adff-8eb2614eb040 req-c4e4a10c-96d0-46d1-997e-b9c5f78f2239 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:39.128324 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3116da9f-60a7-4c32-adff-8eb2614eb040 req-c4e4a10c-96d0-46d1-997e-b9c5f78f2239 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:39.140145 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-3116da9f-60a7-4c32-adff-8eb2614eb040 req-c4e4a10c-96d0-46d1-997e-b9c5f78f2239 service nova] Creating event network-changed:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch2-master-1 Oct 15 22:03:39.145184 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-3116da9f-60a7-4c32-adff-8eb2614eb040 req-c4e4a10c-96d0-46d1-997e-b9c5f78f2239 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038837 Oct 15 22:03:39.145787 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2115/6472] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:03:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:03:39.846617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:39.848058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:39.855697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:39.856748 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:39.857230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:39.858096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:39.858563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:39.859153 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:39.996786 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:40.005632 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-825033b8-164f-4201-b207-89aa57238a78 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.163086 Oct 15 22:03:40.006204 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4358/6473] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:39 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:41.031048 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:41.031803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:41.038032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:41.038532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:41.039022 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:41.039734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:41.040210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:41.040655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:41.171150 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:41.178845 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5628d050-cf48-4ef6-af7f-a976d2781659 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.152953 Oct 15 22:03:41.179411 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2116/6474] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:41 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:42.203265 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:42.203898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:42.211185 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:42.211680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:42.212126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:42.212961 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:42.213474 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:42.213904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:42.354463 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:42.362800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c8b02cb9-092e-48fd-99d4-7cac40e5054e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.163567 Oct 15 22:03:42.363395 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4359/6475] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:42 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:43.386472 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:43.387111 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:43.394500 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:43.394996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:43.395451 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:43.396157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:43.396629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:43.397102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:43.533954 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:43.542037 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c4a77aad-44ff-44ea-9ae2-7dca4c2544b2 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.159499 Oct 15 22:03:43.542610 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2117/6476] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:43 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:44.566826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:44.567464 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:44.574273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:44.574754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:44.575245 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:44.575952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:44.576410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:44.576842 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:44.712093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:44.720909 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4a6f1355-2fbd-4ccb-b154-2df175d69260 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.158038 Oct 15 22:03:44.721507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4360/6477] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:44 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:45.729379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d08fed42-d35e-4aaf-b384-a737b778eee2 req-8b98e02d-5067-42a2-98eb-8c602e9f9118 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "name": "network-vif-plugged", "status": "completed", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:45.739756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d08fed42-d35e-4aaf-b384-a737b778eee2 req-8b98e02d-5067-42a2-98eb-8c602e9f9118 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:45.740256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d08fed42-d35e-4aaf-b384-a737b778eee2 req-8b98e02d-5067-42a2-98eb-8c602e9f9118 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:45.740700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d08fed42-d35e-4aaf-b384-a737b778eee2 req-8b98e02d-5067-42a2-98eb-8c602e9f9118 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:45.745310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:45.745931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:45.752397 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d08fed42-d35e-4aaf-b384-a737b778eee2 req-8b98e02d-5067-42a2-98eb-8c602e9f9118 service nova] Creating event network-vif-plugged:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch2-master-1 Oct 15 22:03:45.752950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:45.753745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:45.754314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:45.755038 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:45.755500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:45.755942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:45.757551 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d08fed42-d35e-4aaf-b384-a737b778eee2 req-8b98e02d-5067-42a2-98eb-8c602e9f9118 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088588 Oct 15 22:03:45.758228 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2118/6478] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:03:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:03:45.898048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:45.906967 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-107053a8-c764-4e39-bfd3-026cfe11e529 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.165638 Oct 15 22:03:45.907589 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4361/6479] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:45 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:03:46.927587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:46.928391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:46.933346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:46.934067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:46.942199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:46.942699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:46.943153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:46.943868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:46.944343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:46.944790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:47.076978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:47.084707 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7e8abb78-cdba-48ae-94af-46d0c782d1ee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1393 microversion: 2.1 time: 0.158080 Oct 15 22:03:47.085374 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2119/6480] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:46 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1393 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:48.109871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:48.110521 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:48.112449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:03:48.118739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:48.119234 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:48.119670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:48.120372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:48.120832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:48.121284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:48.260764 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:48.268663 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c57d6bce-23ae-4834-9d79-59c224a2b6bd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1565 microversion: 2.1 time: 0.163647 Oct 15 22:03:48.269266 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4362/6481] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:48 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1565 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:48.291462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:48.292103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:48.299618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:48.300127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:48.300593 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:48.301378 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:48.301859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:48.302309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:48.432314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:48.441293 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4dc20f69-7649-458b-b814-81bc1846833a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1565 microversion: 2.1 time: 0.153756 Oct 15 22:03:48.441867 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2120/6482] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:48 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:03:50.150957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:03:50.152098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:03:50.156496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c76088be-8371-442c-bed1-49a030c25b80 req-839c657e-72d5-468f-8f40-4d9b592b20af service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:03:50.160250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:50.160753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:50.161331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:50.162924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:50.163930 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:50.164804 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:50.167440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c76088be-8371-442c-bed1-49a030c25b80 req-839c657e-72d5-468f-8f40-4d9b592b20af service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:03:50.168118 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c76088be-8371-442c-bed1-49a030c25b80 req-839c657e-72d5-468f-8f40-4d9b592b20af service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:03:50.168675 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c76088be-8371-442c-bed1-49a030c25b80 req-839c657e-72d5-468f-8f40-4d9b592b20af service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:03:50.189592 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c76088be-8371-442c-bed1-49a030c25b80 req-839c657e-72d5-468f-8f40-4d9b592b20af service nova] Creating event network-changed:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch2-master-1 Oct 15 22:03:50.199385 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c76088be-8371-442c-bed1-49a030c25b80 req-839c657e-72d5-468f-8f40-4d9b592b20af service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047217 Oct 15 22:03:50.201345 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2121/6483] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:03:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:03:50.319911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:03:50.328559 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0e1b034c-09b2-4055-805c-95e4e3449692 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1565 microversion: 2.1 time: 0.182542 Oct 15 22:03:50.329495 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4363/6484] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:03:50 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1565 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:21.609075 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:21.609845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=38 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:21.615124 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:04:21.617442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:21.626017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:21.626721 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:21.627188 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:21.627912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:21.628396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:21.628847 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:21.687584 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d56e811f-891e-432c-99f4-f73df26b7aec tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action" status: 200 len: 24571 microversion: 2.1 time: 0.079829 Oct 15 22:04:21.688217 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2122/6485] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:04:21 2022] POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action => generated 24571 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:04:52.064321 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:52.065131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=42 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:52.066444 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:04:52.066655 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:04:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action (ip 10.222.0.16) !!! Oct 15 22:04:52.159448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:04:52.161824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:52.167924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:52.168418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:52.168866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:52.169600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:52.170063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:52.170494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:52.219697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:52.225871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Old instance type m1.heat2, new instance type m1.heat {{(pid=138688) resize /opt/stack/nova/nova/compute/api.py:4220}} Oct 15 22:04:52.232440 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:04:52.232867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:04:52.233406 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:04:52.233824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:04:52.234247 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:04:52.302368 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-df9b9e4c-bf48-4a0e-87ee-92ce5e8be18b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action" status: 202 len: 0 microversion: 2.1 time: 0.239773 Oct 15 22:04:52.302892 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4364/6486] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:04:52 2022] POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action => generated 0 bytes in 241 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:04:52.311820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:52.313181 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:52.316034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:52.316751 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:04:52.323902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:52.324448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:52.325019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:52.325969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:52.326591 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:52.327037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:52.472077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:52.480292 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-68a972de-f271-4912-99cb-a6ccbd9bec54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1693 microversion: 2.1 time: 0.172844 Oct 15 22:04:52.480879 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2123/6487] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:52 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1693 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:53.508545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:53.509198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:53.515333 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:53.515835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:53.516369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:53.517080 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:53.517543 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:53.517973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:53.658316 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:53.666666 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eac77c85-d3da-48be-8ece-75c3d002be50 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1693 microversion: 2.1 time: 0.163033 Oct 15 22:04:53.667261 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4365/6488] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:53 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1693 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:54.692000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:54.692623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:54.699741 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:54.700238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:54.700689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:54.701422 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:54.701893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:54.702335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:54.832929 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:54.840847 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d75c189e-47c1-4530-b7b4-7f5f64d109ad tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.153242 Oct 15 22:04:54.841436 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2124/6489] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:54 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:55.864193 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:55.864791 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:55.871801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:55.872244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:55.872665 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:55.873383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:55.873851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:55.874279 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:56.008224 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:56.016664 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1944a6f3-795c-41fa-afe1-e0a30dfe619e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.155766 Oct 15 22:04:56.017241 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4366/6490] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:55 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:57.040444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:57.041125 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:57.048218 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:57.048717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:57.049199 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:57.049906 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:57.050376 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:57.050815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:57.181145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:57.189438 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c0a9d90f-9e61-41ea-9f91-4721d2fcc05d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.152858 Oct 15 22:04:57.190259 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2125/6491] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:57 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:58.213752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:58.214373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:58.221816 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:58.222307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:58.222755 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:58.223453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:58.223922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:58.224356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:58.354683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:58.362509 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6a7f5c8e-52c5-4e02-afce-6fa398ee914e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.152724 Oct 15 22:04:58.363085 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4367/6492] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:58 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:04:59.386455 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:04:59.387114 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:04:59.394732 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:59.395245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:59.395708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:59.396431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:04:59.396911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:04:59.397405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:04:59.533088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:04:59.540961 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c06897ab-1bd7-4df5-957c-75d6a92dbf8a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.158416 Oct 15 22:04:59.541564 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2126/6493] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:04:59 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:00.565098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:00.565754 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:00.573475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:00.573974 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:00.574422 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:00.575152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:00.575616 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:00.576051 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:00.707180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:00.714836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9a02ddf-b81e-4adb-ab08-c000cb299ad1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.154357 Oct 15 22:05:00.715408 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4368/6494] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:00 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:01.737616 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:01.738285 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:01.745259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:01.745768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:01.746229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:01.746948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:01.747428 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:01.747875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:01.883802 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:01.892494 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-115aac3e-0654-4945-9004-f6d19771f1ff tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.158870 Oct 15 22:05:01.893089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2127/6495] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:01 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:02.916261 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:02.916876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:02.924711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:02.925238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:02.925680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:02.926373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:02.926833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:02.927263 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:03.056854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:03.064704 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e9ee10f-cbe5-4d2d-bd69-233f0a796b94 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.152290 Oct 15 22:05:03.065310 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4369/6496] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:02 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:04.088766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:04.089374 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:04.097402 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:04.097911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:04.098369 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:04.099085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:04.099562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:04.100010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:04.233323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:04.241939 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b12ca61e-e4b9-4ed7-8fae-b1be4d7346db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.157155 Oct 15 22:05:04.242549 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2128/6497] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:04 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:05.265489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:05.266329 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:05.273414 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:05.273906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:05.274356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:05.275061 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:05.275526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:05.275962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:05.416514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:05.424937 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4658ced6-454e-40f6-8186-8cf9b425b84e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1698 microversion: 2.1 time: 0.163364 Oct 15 22:05:05.425563 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4370/6498] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:05 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1698 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:06.449224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:06.449777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:06.459433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:06.459937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:06.460410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:06.461173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:06.461653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:06.462103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:06.676984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:06.685198 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-68604546-808d-4571-bae2-202118aa0844 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1697 microversion: 2.1 time: 0.239924 Oct 15 22:05:06.685777 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2129/6499] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:06 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1697 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:06.700096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "name": "network-vif-unplugged", "status": "completed", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:06.712216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:06.712915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:06.713547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:06.726787 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] Creating event network-vif-unplugged:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch1-master-1 Oct 15 22:05:06.732506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] Instance e434f121-2231-4124-84e6-f19e9668921b is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138688) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 22:05:06.744249 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-792b0fd8-b023-464b-ba6a-cf0236994827 req-e2ce9441-3d9e-404a-964d-a38d42a7dd1f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.048000 Oct 15 22:05:06.744873 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4371/6500] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:05:06 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:05:07.709731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:07.710364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:07.717360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:07.717864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:07.718322 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:07.719037 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:07.719513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:07.719959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:07.948531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:07.956345 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-14e7662a-64ec-4397-ac13-fdac089e2cee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1697 microversion: 2.1 time: 0.250551 Oct 15 22:05:07.956913 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2130/6501] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:07 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1697 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:08.981622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:08.982323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:08.989277 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:08.989763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:08.990202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:08.990892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:08.991351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:08.991778 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:09.087214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:09.096999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:09.097522 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:09.097967 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:09.109826 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] Creating event network-changed:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch1-master-1 Oct 15 22:05:09.116183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] Instance e434f121-2231-4124-84e6-f19e9668921b is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=138689) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Oct 15 22:05:09.127614 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-2e4df15b-2f5e-415d-99e4-b38bc22c6784 req-0d088849-3e43-441c-9b6e-aa877e07d4bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043763 Oct 15 22:05:09.128228 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2131/6502] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:05:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:05:09.134128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:09.142552 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7057afc5-1c5e-4fcf-8069-5d8c123c3c1c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1695 microversion: 2.1 time: 0.166080 Oct 15 22:05:09.143308 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4372/6503] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:08 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1695 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:10.166268 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:10.167011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:10.174164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.174668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.175119 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.175837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.176310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.176756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.309864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:10.318216 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8c4b4c70-a980-40c8-9507-4f4aa50f6c54 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1692 microversion: 2.1 time: 0.156206 Oct 15 22:05:10.318788 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2132/6504] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:10 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1692 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:10.341281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:10.341800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:10.348857 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.349383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.349824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.350516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.350971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.351424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.409676 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6e66803a-0109-472c-a2a7-064de4243cb1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action" status: 204 len: 0 microversion: 2.1 time: 0.072108 Oct 15 22:05:10.410323 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4373/6505] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:05:10 2022] POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action => generated 0 bytes in 74 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:05:10.418749 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:10.419306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:10.425776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.426290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.426747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.427474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.427949 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.428394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.567865 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:10.575809 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ed6ee695-fe1c-4f94-94d3-63f250fd7bb9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1692 microversion: 2.1 time: 0.161169 Oct 15 22:05:10.576376 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2133/6506] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:10 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1692 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:10.600309 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:10.601024 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:10.607230 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.607726 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.608168 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.608873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:10.609391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:10.609824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:10.754695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:10.762883 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-18dc5635-1dc2-4d18-9b26-f20059b420bf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1692 microversion: 2.1 time: 0.167041 Oct 15 22:05:10.763456 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4374/6507] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:10 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1692 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:11.786397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:11.787034 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:11.794030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:11.794541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:11.795006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:11.795760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:11.796237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:11.796687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:11.929676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:11.937547 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9a24925f-3d4b-4489-a6a6-7aeb7d7157d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1684 microversion: 2.1 time: 0.155142 Oct 15 22:05:11.938120 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2134/6508] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:11 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1684 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:13.281915 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-09c440f7-2a03-40fe-a781-f73750c2658f req-60586f2b-0011-4637-823a-4d1e6102ae3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "name": "network-vif-plugged", "status": "completed", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:13.293835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-09c440f7-2a03-40fe-a781-f73750c2658f req-60586f2b-0011-4637-823a-4d1e6102ae3a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:13.294308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-09c440f7-2a03-40fe-a781-f73750c2658f req-60586f2b-0011-4637-823a-4d1e6102ae3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:13.294757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-09c440f7-2a03-40fe-a781-f73750c2658f req-60586f2b-0011-4637-823a-4d1e6102ae3a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:13.306436 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-09c440f7-2a03-40fe-a781-f73750c2658f req-60586f2b-0011-4637-823a-4d1e6102ae3a service nova] Creating event network-vif-plugged:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch1-master-1 Oct 15 22:05:13.311594 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-09c440f7-2a03-40fe-a781-f73750c2658f req-60586f2b-0011-4637-823a-4d1e6102ae3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033504 Oct 15 22:05:13.312365 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4375/6509] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:05:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:05:34.120412 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:34.122713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:34.125866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=24 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:34.126682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=70 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:34.132770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:34.133314 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:34.133776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:34.134505 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:34.134980 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:34.135431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:34.191803 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-daed9eda-31e2-48f3-8c83-f7ea4e1f436a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action" status: 200 len: 24578 microversion: 2.1 time: 0.075110 Oct 15 22:05:34.192506 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2135/6510] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:05:34 2022] POST /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b/action => generated 24578 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:05:41.213286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:41.214098 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:41.215853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-113a3a31-47de-43bd-8f0e-090d4e563ef0 req-2a15b85d-797b-467c-bc28-57fa1ac7191b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:41.216756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=28 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:41.217524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:41.225705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:41.226001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:41.226624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:41.227410 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:41.227956 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:41.228109 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-113a3a31-47de-43bd-8f0e-090d4e563ef0 req-2a15b85d-797b-467c-bc28-57fa1ac7191b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:41.228478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:41.228637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-113a3a31-47de-43bd-8f0e-090d4e563ef0 req-2a15b85d-797b-467c-bc28-57fa1ac7191b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:41.229081 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-113a3a31-47de-43bd-8f0e-090d4e563ef0 req-2a15b85d-797b-467c-bc28-57fa1ac7191b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:41.242893 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-113a3a31-47de-43bd-8f0e-090d4e563ef0 req-2a15b85d-797b-467c-bc28-57fa1ac7191b service nova] Creating event network-changed:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch1-master-1 Oct 15 22:05:41.249649 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-113a3a31-47de-43bd-8f0e-090d4e563ef0 req-2a15b85d-797b-467c-bc28-57fa1ac7191b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037682 Oct 15 22:05:41.250652 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2136/6511] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:05:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:05:41.256723 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Going to try to terminate instance {{(pid=138688) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:05:41.356457 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-efa16926-51cb-4623-a774-3277ef0eb3a5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 204 len: 0 microversion: 2.1 time: 0.147321 Oct 15 22:05:41.360809 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4376/6512] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:05:41 2022] DELETE /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:05:41.367382 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:41.368069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:41.374352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:41.374857 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:41.375535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:41.376264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:41.376904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:41.377565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:41.535379 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:41.543386 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-804955a2-459e-4a27-a604-328e99595469 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1690 microversion: 2.1 time: 0.181257 Oct 15 22:05:41.544166 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2137/6513] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:41 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1690 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:05:42.569867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:42.570573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:42.577420 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:42.577907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:42.578346 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:42.579040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:42.579498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:42.579923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:42.801134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:42.809940 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5a3424c9-f613-432e-8807-1a0ee3d88c4a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 200 len: 1690 microversion: 2.1 time: 0.244841 Oct 15 22:05:42.810621 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4377/6514] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:42 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 1690 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:43.156019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-3211b994-5b27-4278-bc46-551c61414dad req-5be39e48-3aba-4480-9f28-265b3a05f153 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e434f121-2231-4124-84e6-f19e9668921b", "name": "network-vif-deleted", "tag": "387402a7-bbb0-46a5-99bb-a9c8ac548bad"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:43.168678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3211b994-5b27-4278-bc46-551c61414dad req-5be39e48-3aba-4480-9f28-265b3a05f153 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:43.169202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3211b994-5b27-4278-bc46-551c61414dad req-5be39e48-3aba-4480-9f28-265b3a05f153 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:43.169860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-3211b994-5b27-4278-bc46-551c61414dad req-5be39e48-3aba-4480-9f28-265b3a05f153 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:43.182132 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-3211b994-5b27-4278-bc46-551c61414dad req-5be39e48-3aba-4480-9f28-265b3a05f153 service nova] Creating event network-vif-deleted:387402a7-bbb0-46a5-99bb-a9c8ac548bad for instance e434f121-2231-4124-84e6-f19e9668921b on nch1-master-1 Oct 15 22:05:43.188013 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-3211b994-5b27-4278-bc46-551c61414dad req-5be39e48-3aba-4480-9f28-265b3a05f153 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036152 Oct 15 22:05:43.188703 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2138/6515] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:05:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:05:43.835815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:43.836342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: e434f121-2231-4124-84e6-f19e9668921b] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:43.843148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:43.843629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:43.844069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:43.844758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:43.845251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:43.845680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:43.859135 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] HTTP exception thrown: Instance e434f121-2231-4124-84e6-f19e9668921b could not be found. Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Request method failure captured: Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b HTTP/1.1 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSy09-QfPZ-9CNDIIFxEAKgOvMyzS_BGx2YAooBKnz2qBEmy5yraK8uUQYc8ul_plXjHeNR8-6n07eBm4u9lCM3KGAMnrogPl4-d9DQbXxkTgXgwzrU8kkhbIZtFBlypgzMCEjP9tyrjP6od2c2gsCfU2f50omy2qrKGIa7h0T3j2Xp0 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 823ad7d2979042c188525f5341febdff Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance e434f121-2231-4124-84e6-f19e9668921b could not be found. Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'e434f121-2231-4124-84e6-f19e9668921b'} Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e434f121-2231-4124-84e6-f19e9668921b could not be found. Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e434f121-2231-4124-84e6-f19e9668921b could not be found. Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:05:43.860784 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e434f121-2231-4124-84e6-f19e9668921b could not be found. Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Returning 404 to user: Instance e434f121-2231-4124-84e6-f19e9668921b could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-315354a7-21d5-4d24-a3b6-29b22e4a7565 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b" status: 404 len: 111 microversion: 2.1 time: 0.030979 Oct 15 22:05:43.866766 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4378/6516] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:43 2022] GET /compute/v2.1/servers/e434f121-2231-4124-84e6-f19e9668921b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:05:47.533155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3634cac5-2347-433b-b987-23cd8bbbb7ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '.version_select at 0x7fe74ed5b160>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:47.538302 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [None req-3634cac5-2347-433b-b987-23cd8bbbb7ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [9bd635f8-0214-4cfd-9c77-5a99b26eed51] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:05:48.545634 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [None req-3634cac5-2347-433b-b987-23cd8bbbb7ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [9bd635f8-0214-4cfd-9c77-5a99b26eed51] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 47150. Oct 15 22:05:48.564968 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3634cac5-2347-433b-b987-23cd8bbbb7ba tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1051697365" status: 202 len: 0 microversion: 2.1 time: 1.037285 Oct 15 22:05:48.565588 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2139/6517] 10.222.0.16 () {58 vars in 1284 bytes} [Sat Oct 15 22:05:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1051697365 => generated 0 bytes in 1039 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 15 22:05:48.593734 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d46e21ed-9b8d-4dd2-9922-1c736b963c7d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: .version_select at 0x7fe75846ee50>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-813143157", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNZCo73cWHcsA64e013bTwVYKepdMAGZdy4ehgH6jC0tA1Zh4sh2HN8HPX2alhtLmiSI7t5nCeXa9BfM22jiySovrIJBxCBtxMY4BsZ7Cy4vCl+S+KhjsEf3JbKoUfxp6Q=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:48.610158 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d46e21ed-9b8d-4dd2-9922-1c736b963c7d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:05:48.616089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-d46e21ed-9b8d-4dd2-9922-1c736b963c7d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:05:48.630610 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [None req-d46e21ed-9b8d-4dd2-9922-1c736b963c7d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [383acc1f-300f-4ec6-84a1-9b7307c5fc94] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:05:49.636858 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [None req-d46e21ed-9b8d-4dd2-9922-1c736b963c7d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [383acc1f-300f-4ec6-84a1-9b7307c5fc94] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 47154. Oct 15 22:05:49.651713 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d46e21ed-9b8d-4dd2-9922-1c736b963c7d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 1.061776 Oct 15 22:05:49.652222 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4379/6518] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:05:48 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 1063 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:05:55.186136 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=14 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:55.190367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1260419469", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "5fc17bae-a435-46f2-9de3-5eac04521fed"}], "key_name": "tempest-TestNetworkAdvancedServerOps-813143157", "security_groups": [{"name": "tempest-secgroup-smoke-135818111"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:55.488998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] validate_networks() for [('5fc17bae-a435-46f2-9de3-5eac04521fed', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:05:55.955384 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:05:55.956930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:05:55.958944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:05:55.959638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:05:55.961069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:05:55.978962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:55.981663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:55.982394 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:55.983869 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:55.984605 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:55.985364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:56.011214 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:05:56.018819 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:05:56.039356 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:05:56.042133 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:05:56.090590 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa9163d8-b897-4377-a916-df4326b8a7eb tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.905999 Oct 15 22:05:56.091418 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2140/6519] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:05:55 2022] POST /compute/v2.1/servers => generated 405 bytes in 907 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:05:56.103552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7848910f-5947-44d2-b440-67e48feecbe7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:56.104877 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7848910f-5947-44d2-b440-67e48feecbe7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:56.109434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=22,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:05:56.167685 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7848910f-5947-44d2-b440-67e48feecbe7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:56.175758 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7848910f-5947-44d2-b440-67e48feecbe7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1266 microversion: 2.1 time: 0.075571 Oct 15 22:05:56.191828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4380/6520] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:56 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1266 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:57.200813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:57.201901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:57.209290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:57.209777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:57.210289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:57.211061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:57.211555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:57.212000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:57.310878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:57.327019 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-55d77c6c-c6c4-459f-83f3-e969b8bb022c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1326 microversion: 2.1 time: 0.130996 Oct 15 22:05:57.327989 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2141/6521] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:57 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1326 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:58.353511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:58.354175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:58.363696 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:58.364407 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:58.364884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:58.365719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:58.366184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:58.366666 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:58.518174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:58.526612 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de8146d3-7a5e-46fd-b22d-130666e3e738 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.176919 Oct 15 22:05:58.527449 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4381/6522] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:58 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:05:58.796738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-19d662c0-b024-4947-b2fa-d0e4c0ed647f req-35e01a27-fc0d-421d-b7b2-93f4e8597aab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "21195ba7-7eac-4b80-bba1-3acd875fc258", "tag": "9d3db129-3d5f-429f-833a-52951862509b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:05:58.810895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-19d662c0-b024-4947-b2fa-d0e4c0ed647f req-35e01a27-fc0d-421d-b7b2-93f4e8597aab service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:58.811419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-19d662c0-b024-4947-b2fa-d0e4c0ed647f req-35e01a27-fc0d-421d-b7b2-93f4e8597aab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:58.811866 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-19d662c0-b024-4947-b2fa-d0e4c0ed647f req-35e01a27-fc0d-421d-b7b2-93f4e8597aab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:58.828320 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-19d662c0-b024-4947-b2fa-d0e4c0ed647f req-35e01a27-fc0d-421d-b7b2-93f4e8597aab service nova] Creating event network-changed:9d3db129-3d5f-429f-833a-52951862509b for instance 21195ba7-7eac-4b80-bba1-3acd875fc258 on nch2-master-1 Oct 15 22:05:58.834169 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-19d662c0-b024-4947-b2fa-d0e4c0ed647f req-35e01a27-fc0d-421d-b7b2-93f4e8597aab service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041301 Oct 15 22:05:58.834790 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2142/6523] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:05:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:05:59.551029 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:05:59.551641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:05:59.558707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:59.559409 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:59.559964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:59.561079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:05:59.561549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:05:59.561982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:05:59.695681 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:05:59.703446 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8b01f656-d5ac-45ea-9f4e-6e7e9ebe56db tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.156402 Oct 15 22:05:59.704035 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4382/6524] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:05:59 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:00.727897 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:00.728536 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:00.735596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:00.736129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:00.736584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:00.737321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:00.738361 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:00.738829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:00.869717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:00.877448 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1761597e-0db7-484b-a842-120903af1d09 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.153819 Oct 15 22:06:00.878020 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2143/6525] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:00 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:01.901697 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:01.902349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:01.909673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:01.910179 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:01.910590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:01.911283 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:01.911737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:01.912164 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:02.046324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:02.054664 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c289d33e-b723-4c44-8230-4af2eba78547 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.157272 Oct 15 22:06:02.055233 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4383/6526] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:01 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:03.077143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:03.077718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:03.085410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:03.085920 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:03.086377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:03.087093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:03.087571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:03.088019 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:03.218255 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:03.228038 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fea23439-7dae-4a98-885f-f31e6c9ed599 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.154896 Oct 15 22:06:03.228615 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2144/6527] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:03 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:04.250780 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:04.251524 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:04.259278 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:04.259772 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:04.260238 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:04.260933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:04.261423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:04.261858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:04.403096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:04.411140 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3c7dbf2-5d93-4c91-b30d-d792e0bd9a4e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.164346 Oct 15 22:06:04.411700 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4384/6528] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:04 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:05.434633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:05.435258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:05.438071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=35,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:05.444464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:05.444968 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:05.445450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:05.446163 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:05.446635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:05.447080 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:05.581400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:05.590025 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-74973f40-cc6e-4778-bd48-2e73b8b1a5d4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.159336 Oct 15 22:06:05.590595 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2145/6529] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:05 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:06.279844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b41c2457-c1d3-4516-bdb5-cf5209b5beb3 req-aa990586-8135-4d2d-a0dc-80373807c5f0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "21195ba7-7eac-4b80-bba1-3acd875fc258", "name": "network-vif-plugged", "status": "completed", "tag": "9d3db129-3d5f-429f-833a-52951862509b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:06:06.286276 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:06.294610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b41c2457-c1d3-4516-bdb5-cf5209b5beb3 req-aa990586-8135-4d2d-a0dc-80373807c5f0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:06.295113 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b41c2457-c1d3-4516-bdb5-cf5209b5beb3 req-aa990586-8135-4d2d-a0dc-80373807c5f0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:06.295551 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b41c2457-c1d3-4516-bdb5-cf5209b5beb3 req-aa990586-8135-4d2d-a0dc-80373807c5f0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:06.307202 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b41c2457-c1d3-4516-bdb5-cf5209b5beb3 req-aa990586-8135-4d2d-a0dc-80373807c5f0 service nova] Creating event network-vif-plugged:9d3db129-3d5f-429f-833a-52951862509b for instance 21195ba7-7eac-4b80-bba1-3acd875fc258 on nch2-master-1 Oct 15 22:06:06.312428 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b41c2457-c1d3-4516-bdb5-cf5209b5beb3 req-aa990586-8135-4d2d-a0dc-80373807c5f0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036563 Oct 15 22:06:06.313023 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4385/6530] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:06:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:06:06.614341 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:06.615087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:06.617267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:06.618032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:06.625555 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:06.626062 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:06.626517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:06.627258 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:06.627740 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:06.628186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:06.763928 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:06.772227 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-edbafc08-0a3d-4880-93c6-098b02922a63 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1391 microversion: 2.1 time: 0.162614 Oct 15 22:06:06.772801 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2146/6531] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:06 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1391 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:07.796705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:07.797354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:07.804434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:07.804914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:07.805446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:07.806146 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:07.806605 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:07.807033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:07.945218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:07.952881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ed69abcf-f09d-49bb-81ad-28ab12b616f8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1563 microversion: 2.1 time: 0.160791 Oct 15 22:06:07.953508 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4386/6532] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:07 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1563 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:07.976256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:07.976888 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:07.983777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:07.984279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:07.984731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:07.985469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:07.985945 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:07.986389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:08.115368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:08.123230 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc4066ec-03fb-4186-ba23-e79c2c086b48 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1563 microversion: 2.1 time: 0.151285 Oct 15 22:06:08.123805 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2147/6533] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:07 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1563 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:10.038844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:06:10.039618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:10.042774 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-8f2fcede-2ce6-4311-8eef-269dba9862e9 req-cf17f9b6-39bc-45df-9b34-bbcfe690d702 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "21195ba7-7eac-4b80-bba1-3acd875fc258", "tag": "9d3db129-3d5f-429f-833a-52951862509b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:06:10.046727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:10.047221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:10.047670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:10.048381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:10.048872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:10.049385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:10.054912 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8f2fcede-2ce6-4311-8eef-269dba9862e9 req-cf17f9b6-39bc-45df-9b34-bbcfe690d702 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:10.055715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8f2fcede-2ce6-4311-8eef-269dba9862e9 req-cf17f9b6-39bc-45df-9b34-bbcfe690d702 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:10.056417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-8f2fcede-2ce6-4311-8eef-269dba9862e9 req-cf17f9b6-39bc-45df-9b34-bbcfe690d702 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:10.068139 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-8f2fcede-2ce6-4311-8eef-269dba9862e9 req-cf17f9b6-39bc-45df-9b34-bbcfe690d702 service nova] Creating event network-changed:9d3db129-3d5f-429f-833a-52951862509b for instance 21195ba7-7eac-4b80-bba1-3acd875fc258 on nch2-master-1 Oct 15 22:06:10.073911 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-8f2fcede-2ce6-4311-8eef-269dba9862e9 req-cf17f9b6-39bc-45df-9b34-bbcfe690d702 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036468 Oct 15 22:06:10.074923 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2148/6534] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:06:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:06:10.194815 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:06:10.203734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-765af06d-7b45-401c-bb1e-abcae3487c4b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1563 microversion: 2.1 time: 0.169470 Oct 15 22:06:10.204326 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4387/6535] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:06:10 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1563 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:06:43.496397 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=7 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:43.497174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=42 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:06:43.501486 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:06:43.503769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:06:43.511009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:43.511530 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:43.511986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:43.512676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:06:43.513172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:06:43.513632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:06:43.574040 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-07db742d-d912-4426-80b0-40e624a9f88e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action" status: 200 len: 24562 microversion: 2.1 time: 0.079590 Oct 15 22:06:43.574734 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2149/6536] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:06:43 2022] POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action => generated 24562 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:07:13.775324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=40,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:07:13.776152 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=38 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:07:13.781131 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bf0585a3-af20-4f65-ac6d-078bd64c5176 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:07:13.788994 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf0585a3-af20-4f65-ac6d-078bd64c5176 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:13.790006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf0585a3-af20-4f65-ac6d-078bd64c5176 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:13.790647 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bf0585a3-af20-4f65-ac6d-078bd64c5176 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:13.813967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bf0585a3-af20-4f65-ac6d-078bd64c5176 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Going to try to stop instance {{(pid=138688) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 15 22:07:13.854504 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bf0585a3-af20-4f65-ac6d-078bd64c5176 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action" status: 202 len: 0 microversion: 2.1 time: 0.080966 Oct 15 22:07:13.855130 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4388/6537] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:07:13 2022] POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:07:13.864129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:13.864891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:13.867992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:07:13.868948 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=4 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:07:13.876424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:13.877005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:13.877499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:13.878229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:13.878705 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:13.879153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:14.017147 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:14.025449 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-051cb5f8-1bd5-4a17-854f-ae5e78790af8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.165826 Oct 15 22:07:14.026014 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2150/6538] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:13 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:15.053298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:15.054013 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:15.060919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:15.061434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:15.061873 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:15.062568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:15.063023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:15.063450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:15.198591 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:15.206599 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d82f3843-271f-4b8b-9ac5-fc585656359a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.158705 Oct 15 22:07:15.207172 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4389/6539] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:15 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:16.230176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:16.230831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:16.237788 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:16.238287 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:16.238742 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:16.239445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:16.239916 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:16.240358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:16.372695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:16.380532 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1cc91770-efd5-41e0-a680-a75b9104c1b9 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.154321 Oct 15 22:07:16.381122 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2151/6540] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:16 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:17.404508 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:17.405169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:17.412358 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:17.412845 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:17.413332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:17.414035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:17.415009 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:17.415446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:17.549582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:17.557964 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e14bc75a-dddd-4dd7-b638-a531c7221189 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.157470 Oct 15 22:07:17.558544 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4390/6541] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:17 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:18.581406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:18.582032 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:18.588998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:18.589772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:18.590457 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:18.591332 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:18.591972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:18.592487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:18.724259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:18.732511 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d96fa94a-a590-4c4a-822b-8d82230e9fb5 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.155046 Oct 15 22:07:18.733099 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2152/6542] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:18 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:19.756996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:19.757643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:19.764946 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:19.765475 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:19.765916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:19.766615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:19.767076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:19.767507 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:19.898492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:19.906673 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dd3f4ac6-581c-44af-b234-7179b68e72da tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.153663 Oct 15 22:07:19.907239 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4391/6543] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:19 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:20.930445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:20.931102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:20.938009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:20.938514 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:20.938970 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:20.939686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:20.940159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:20.940602 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:21.076036 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:21.083977 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c2fb78f7-a931-46bd-aaaf-cff10dacf985 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.157413 Oct 15 22:07:21.084555 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2153/6544] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:20 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:22.108680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:22.109336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:22.116462 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:22.116960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:22.118031 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:22.118735 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:22.119198 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:22.119658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:22.253351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:22.261287 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-4ddf5a1e-6e1a-470e-a6a2-3c56d7985591 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.156552 Oct 15 22:07:22.261787 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4392/6545] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:22 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:23.285903 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:23.286531 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:23.294067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:23.294565 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:23.295020 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:23.295725 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:23.296194 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:23.296635 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:23.432445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:23.440393 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b2fe8e02-e85b-427f-84a3-802b4553f643 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1691 microversion: 2.1 time: 0.158427 Oct 15 22:07:23.440968 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2154/6546] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:23 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1691 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:24.465073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:24.466208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:24.472904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:24.473435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:24.473892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:24.474609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:24.475076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:24.475512 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:24.654904 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:24.662970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-04f1b8a3-1dc3-4756-a489-298838f507cd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1668 microversion: 2.1 time: 0.202761 Oct 15 22:07:24.663547 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4393/6547] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:24 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1668 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:25.760431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:07:25.762669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:25.769756 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:25.770263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:25.770723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:25.771438 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:25.771917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:25.772365 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:25.834406 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-692e5179-accd-4f1d-b9a0-686c3f35881f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action" status: 200 len: 38398 microversion: 2.1 time: 0.078608 Oct 15 22:07:25.835082 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2155/6548] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:07:25 2022] POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action => generated 38398 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:07:25.846998 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b69056d6-cf37-4689-9163-66dbfe6a8630 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:07:25.853999 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b69056d6-cf37-4689-9163-66dbfe6a8630 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:25.854488 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b69056d6-cf37-4689-9163-66dbfe6a8630 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:25.854925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b69056d6-cf37-4689-9163-66dbfe6a8630 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:25.877044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b69056d6-cf37-4689-9163-66dbfe6a8630 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Going to try to start instance {{(pid=138688) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 15 22:07:25.918953 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b69056d6-cf37-4689-9163-66dbfe6a8630 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action" status: 202 len: 0 microversion: 2.1 time: 0.075654 Oct 15 22:07:25.919911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4394/6549] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:07:25 2022] POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:07:25.928792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:25.929597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:25.937589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:25.938092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:25.938547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:25.939262 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:25.939728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:25.940169 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:26.081700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:26.089851 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-74d8ad8f-235a-4b2c-98df-c092dfdc3ecf tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1677 microversion: 2.1 time: 0.165256 Oct 15 22:07:26.090543 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2156/6550] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:25 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1677 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:27.114980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:27.115692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:27.123369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:27.123861 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:27.124307 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:27.125096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:27.125568 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:27.126004 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:27.262349 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:27.270547 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0227a70e-47bd-4ff5-a765-b8b189cf5f77 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1677 microversion: 2.1 time: 0.159839 Oct 15 22:07:27.271147 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4395/6551] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:27 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1677 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:07:28.297264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:07:28.297893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:07:28.304713 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:28.305249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:28.306873 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:28.307612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:07:28.308088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:07:28.308535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:07:28.444103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:07:28.452165 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-be0b7e69-87c6-4166-8b8a-09d61596ed68 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1681 microversion: 2.1 time: 0.158866 Oct 15 22:07:28.452673 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2157/6552] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:07:28 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:00.668620 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:00.670898 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:00.673458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=16 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:00.674132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:00.679716 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:00.680324 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:00.680763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:00.681581 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:00.682037 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:00.682499 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:00.747614 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3fb59346-fb06-4ec7-9b86-95929ead537d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action" status: 200 len: 62952 microversion: 2.1 time: 0.083058 Oct 15 22:08:00.755828 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4396/6553] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:08:00 2022] POST /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258/action => generated 62952 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:08:17.961596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:17.962311 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:17.964954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-5484f5df-8aa7-4c1b-a2cc-2e7e762b11db req-164936b3-d184-4d2d-bd56-ed54aa3e2ba4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "21195ba7-7eac-4b80-bba1-3acd875fc258", "tag": "9d3db129-3d5f-429f-833a-52951862509b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:17.966014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=16 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:17.966910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=46 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:17.968085 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 15 22:08:17.969584 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 15 22:08:17.973863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:17.974511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:17.981688 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:17.982537 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:17.983152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:17.984162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:17.984734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:17.985279 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:17.985939 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5484f5df-8aa7-4c1b-a2cc-2e7e762b11db req-164936b3-d184-4d2d-bd56-ed54aa3e2ba4 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:17.986429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5484f5df-8aa7-4c1b-a2cc-2e7e762b11db req-164936b3-d184-4d2d-bd56-ed54aa3e2ba4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:17.986858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-5484f5df-8aa7-4c1b-a2cc-2e7e762b11db req-164936b3-d184-4d2d-bd56-ed54aa3e2ba4 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:17.999563 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-5484f5df-8aa7-4c1b-a2cc-2e7e762b11db req-164936b3-d184-4d2d-bd56-ed54aa3e2ba4 service nova] Creating event network-changed:9d3db129-3d5f-429f-833a-52951862509b for instance 21195ba7-7eac-4b80-bba1-3acd875fc258 on nch2-master-1 Oct 15 22:08:18.004830 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-5484f5df-8aa7-4c1b-a2cc-2e7e762b11db req-164936b3-d184-4d2d-bd56-ed54aa3e2ba4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044890 Oct 15 22:08:18.005790 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4397/6554] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:08:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:08:18.014346 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:08:18.101746 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2ad25a04-5ca1-4f4d-89cd-1794e37d67aa tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 204 len: 0 microversion: 2.1 time: 0.143429 Oct 15 22:08:18.106857 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2158/6555] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:08:17 2022] DELETE /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:08:18.114077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:18.115237 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:18.126066 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:18.126844 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:18.127291 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:18.128215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:18.128690 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:18.129157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:18.282130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:18.290580 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fe39b37b-cdda-47ca-ab6a-162035970853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1687 microversion: 2.1 time: 0.181907 Oct 15 22:08:18.291473 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4398/6556] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:18 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1687 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:08:19.316720 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:19.317468 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:19.325007 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:19.325551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:19.326006 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:19.326728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:19.327210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:19.327657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:19.466710 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:19.474626 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a2e55d36-aa89-4b48-a0c8-04461ecc11b7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1687 microversion: 2.1 time: 0.162240 Oct 15 22:08:19.475197 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2159/6557] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:19 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1687 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:20.290547 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-7514e6f0-9c7b-4ee3-b6d6-cda5b678362c req-9aa9340d-9239-4065-8605-8350c57c5e72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "21195ba7-7eac-4b80-bba1-3acd875fc258", "name": "network-vif-deleted", "tag": "9d3db129-3d5f-429f-833a-52951862509b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:20.301548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7514e6f0-9c7b-4ee3-b6d6-cda5b678362c req-9aa9340d-9239-4065-8605-8350c57c5e72 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:20.302033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7514e6f0-9c7b-4ee3-b6d6-cda5b678362c req-9aa9340d-9239-4065-8605-8350c57c5e72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:20.302656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-7514e6f0-9c7b-4ee3-b6d6-cda5b678362c req-9aa9340d-9239-4065-8605-8350c57c5e72 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:20.313881 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-7514e6f0-9c7b-4ee3-b6d6-cda5b678362c req-9aa9340d-9239-4065-8605-8350c57c5e72 service nova] Creating event network-vif-deleted:9d3db129-3d5f-429f-833a-52951862509b for instance 21195ba7-7eac-4b80-bba1-3acd875fc258 on nch2-master-1 Oct 15 22:08:20.318613 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-7514e6f0-9c7b-4ee3-b6d6-cda5b678362c req-9aa9340d-9239-4065-8605-8350c57c5e72 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087355 Oct 15 22:08:20.319207 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4399/6558] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:08:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:08:20.499015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:20.499662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:20.506863 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:20.507373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:20.507892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:20.508825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:20.509569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:20.510102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:20.545568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:20.546079 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:20.546534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:20.547017 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lazy-loading 'fault' on Instance uuid 21195ba7-7eac-4b80-bba1-3acd875fc258 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:08:20.594323 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:20.602436 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-127fcc3f-0f58-46e9-b6bb-640a2125da5c tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 200 len: 1356 microversion: 2.1 time: 0.107864 Oct 15 22:08:20.603013 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2160/6559] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:20 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 1356 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:21.627984 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:21.628615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 21195ba7-7eac-4b80-bba1-3acd875fc258] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:21.634747 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:21.635722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:21.636157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:21.636847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:21.637334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:21.637763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:21.654426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] HTTP exception thrown: Instance 21195ba7-7eac-4b80-bba1-3acd875fc258 could not be found. Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Request method failure captured: Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 HTTP/1.1 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSy09-QfPZ-9CNDIIFxEAKgOvMyzS_BGx2YAooBKnz2qBEmy5yraK8uUQYc8ul_plXjHeNR8-6n07eBm4u9lCM3KGAMnrogPl4-d9DQbXxkTgXgwzrU8kkhbIZtFBlypgzMCEjP9tyrjP6od2c2gsCfU2f50omy2qrKGIa7h0T3j2Xp0 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 823ad7d2979042c188525f5341febdff Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 21195ba7-7eac-4b80-bba1-3acd875fc258 could not be found. Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '21195ba7-7eac-4b80-bba1-3acd875fc258'} Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 21195ba7-7eac-4b80-bba1-3acd875fc258 could not be found. Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 21195ba7-7eac-4b80-bba1-3acd875fc258 could not be found. Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:08:21.656035 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 21195ba7-7eac-4b80-bba1-3acd875fc258 could not be found. Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Returning 404 to user: Instance 21195ba7-7eac-4b80-bba1-3acd875fc258 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a55861fb-fac5-4a50-ae2b-75204ebcb6a7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258" status: 404 len: 111 microversion: 2.1 time: 0.034594 Oct 15 22:08:21.661905 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4400/6560] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:21 2022] GET /compute/v2.1/servers/21195ba7-7eac-4b80-bba1-3acd875fc258 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:08:24.705957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52946b35-016d-4e9a-b2cc-8530e8b39b05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '.version_select at 0x7fe74ed75820>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:24.736638 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52946b35-016d-4e9a-b2cc-8530e8b39b05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-813143157" status: 202 len: 0 microversion: 2.1 time: 0.035017 Oct 15 22:08:24.737499 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2161/6561] 10.222.0.16 () {58 vars in 1281 bytes} [Sat Oct 15 22:08:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-813143157 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:08:24.774386 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f89e8a51-aa63-4dc7-875a-132d48969d58 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: .version_select at 0x7fe75846ee50>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-815424250", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNwVLu/sRq5zkO1plLpYns2IFs7bRF/ejm/PlpXidaFHAJZL8Zo+qI9AIReKMeZvgpCYUhq/bI1WhJHDhbatr/Uo6KzRqQQs8YWaG5IMW1+b5bcrzGrhS6WmSaSdQQtkQA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:24.792744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f89e8a51-aa63-4dc7-875a-132d48969d58 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:08:24.798872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f89e8a51-aa63-4dc7-875a-132d48969d58 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:08:24.824333 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f89e8a51-aa63-4dc7-875a-132d48969d58 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.053736 Oct 15 22:08:24.824862 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4401/6562] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:08:24 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:08:30.600245 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:30.604981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1104050483", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "networks": [{"uuid": "65afd8cd-5a22-47c8-a950-edec7d520ebe"}], "key_name": "tempest-TestNetworkAdvancedServerOps-815424250", "security_groups": [{"name": "tempest-secgroup-smoke-1903016672"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:30.823360 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] validate_networks() for [('65afd8cd-5a22-47c8-a950-edec7d520ebe', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:08:31.230669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:08:31.231110 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:08:31.231601 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:08:31.232307 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:08:31.232743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:08:31.241706 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:31.242398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:31.242860 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:31.243609 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:31.244082 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:31.245142 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:31.266814 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:08:31.272137 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Getting quotas for user 823ad7d2979042c188525f5341febdff and project ea5345e230164cd6ae5b1195425e7ff0. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:08:31.285127 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:08:31.287470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:08:31.343581 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cd61a5cc-45f0-4535-83d5-6c22085bb650 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.745113 Oct 15 22:08:31.346137 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2162/6563] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:08:30 2022] POST /compute/v2.1/servers => generated 406 bytes in 748 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:08:31.360030 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e9f615f1-ce16-4f41-aaa3-8a69d19abe2d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:31.362078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e9f615f1-ce16-4f41-aaa3-8a69d19abe2d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:31.435228 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e9f615f1-ce16-4f41-aaa3-8a69d19abe2d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:31.443567 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e9f615f1-ce16-4f41-aaa3-8a69d19abe2d tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1266 microversion: 2.1 time: 0.090085 Oct 15 22:08:31.444575 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4402/6564] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:31 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1266 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:32.469818 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:32.471116 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:32.485180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:32.489030 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:32.489780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:32.490646 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:32.491269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:32.494579 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:32.647092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:32.659234 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-235f35ba-ac3a-434e-8b83-45d1b079f124 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1326 microversion: 2.1 time: 0.194697 Oct 15 22:08:32.660489 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2163/6565] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:32 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1326 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:33.683683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:33.684417 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:33.686680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:33.693692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:33.694532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:33.695208 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:33.696157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:33.696692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:33.697281 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:33.869094 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:33.878480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0154c3c6-8996-4d96-90ee-d076219f15c4 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.198533 Oct 15 22:08:33.879457 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4403/6566] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:33 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:34.079499 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-841eeac2-5b80-4036-902a-d6b87eb49a15 req-03faa3c7-168a-411e-ba0a-0d6f8eff2bc0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7c63dd3b-40bd-45ea-924e-8470f841db1e", "tag": "5a398623-e0ed-4c62-91af-9619a592ae0e"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:34.095149 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-841eeac2-5b80-4036-902a-d6b87eb49a15 req-03faa3c7-168a-411e-ba0a-0d6f8eff2bc0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:34.097210 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-841eeac2-5b80-4036-902a-d6b87eb49a15 req-03faa3c7-168a-411e-ba0a-0d6f8eff2bc0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:34.097868 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-841eeac2-5b80-4036-902a-d6b87eb49a15 req-03faa3c7-168a-411e-ba0a-0d6f8eff2bc0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:34.112991 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-841eeac2-5b80-4036-902a-d6b87eb49a15 req-03faa3c7-168a-411e-ba0a-0d6f8eff2bc0 service nova] Creating event network-changed:5a398623-e0ed-4c62-91af-9619a592ae0e for instance 7c63dd3b-40bd-45ea-924e-8470f841db1e on nch2-master-1 Oct 15 22:08:34.118155 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-841eeac2-5b80-4036-902a-d6b87eb49a15 req-03faa3c7-168a-411e-ba0a-0d6f8eff2bc0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042354 Oct 15 22:08:34.118917 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2164/6567] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:08:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:08:34.903154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:34.903718 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:34.911036 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:34.911531 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:34.911970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:34.912684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:34.913174 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:34.913610 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:35.050535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:35.058211 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cffa0378-62db-478e-8dc0-df7805a83443 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.159629 Oct 15 22:08:35.058783 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4404/6568] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:34 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:36.082546 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:36.083277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:36.090859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:36.091367 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:36.091830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:36.092549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:36.093053 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:36.093507 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:36.230156 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:36.238064 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-83052a36-e729-4f08-8908-74cfd606d72b tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.159839 Oct 15 22:08:36.238635 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2165/6569] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:36 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:37.262059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:37.262668 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:37.269573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:37.270058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:37.270494 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:37.271184 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:37.271648 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:37.272077 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:37.410860 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:37.419463 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0a2f5424-b0d9-4a8c-b4ab-d5097c61aaab tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.161593 Oct 15 22:08:37.420029 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4405/6570] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:37 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:38.443551 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:38.444187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:38.452239 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:38.452690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:38.453175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:38.453893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:38.454371 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:38.454820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:38.590282 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:38.599249 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d25296de-e988-4eda-ae17-67126ef5103a tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.159531 Oct 15 22:08:38.599832 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2166/6571] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:38 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:39.622506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:39.623056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:39.629364 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:39.629864 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:39.630312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:39.631019 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:39.631495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:39.631935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:39.766911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:39.774784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-de09eb69-45b6-4642-958b-90e1acca4d05 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.156636 Oct 15 22:08:39.775358 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4406/6572] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:39 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:40.799065 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:40.799731 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:40.806295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:40.806855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:40.807855 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:40.808587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:40.809092 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:40.809543 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:40.918130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-30d0ec48-ce1b-4996-bd04-f8219fb639fa req-ca75f082-74cb-4d13-8d53-03ce9fbd3eb9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7c63dd3b-40bd-45ea-924e-8470f841db1e", "name": "network-vif-plugged", "status": "completed", "tag": "5a398623-e0ed-4c62-91af-9619a592ae0e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:40.930076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30d0ec48-ce1b-4996-bd04-f8219fb639fa req-ca75f082-74cb-4d13-8d53-03ce9fbd3eb9 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:40.930760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30d0ec48-ce1b-4996-bd04-f8219fb639fa req-ca75f082-74cb-4d13-8d53-03ce9fbd3eb9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:40.931401 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-30d0ec48-ce1b-4996-bd04-f8219fb639fa req-ca75f082-74cb-4d13-8d53-03ce9fbd3eb9 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:40.943803 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-30d0ec48-ce1b-4996-bd04-f8219fb639fa req-ca75f082-74cb-4d13-8d53-03ce9fbd3eb9 service nova] Creating event network-vif-plugged:5a398623-e0ed-4c62-91af-9619a592ae0e for instance 7c63dd3b-40bd-45ea-924e-8470f841db1e on nch2-master-1 Oct 15 22:08:40.946833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:40.950142 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-30d0ec48-ce1b-4996-bd04-f8219fb639fa req-ca75f082-74cb-4d13-8d53-03ce9fbd3eb9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035993 Oct 15 22:08:40.950771 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4407/6573] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:08:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:08:40.956838 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-90d168b9-3bb3-4e82-8465-e13a05111206 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.161796 Oct 15 22:08:40.958393 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2167/6574] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:40 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:41.984509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:41.990003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:41.990705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:41.997218 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:41.997706 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:41.998147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:41.998847 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:41.999308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:41.999741 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:42.135684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:42.143955 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-782dbe99-6d77-47b5-ab94-e638877b7093 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1392 microversion: 2.1 time: 0.160450 Oct 15 22:08:42.144534 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4408/6575] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:41 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1392 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:43.168423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:43.169129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:43.171135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:43.171825 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:08:43.178875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:43.179381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:43.179843 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:43.180563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:43.181067 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:43.181515 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:43.316687 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:43.325133 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-75b2a48b-f24e-4262-8b63-250c307c4365 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1564 microversion: 2.1 time: 0.161384 Oct 15 22:08:43.325696 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2168/6576] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:43 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1564 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:43.348449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:43.349003 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:43.355936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:43.356426 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:43.356870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:43.357600 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:43.358064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:43.358496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:43.495803 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:43.503626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5034bc54-a644-46ee-ba1a-dfa82f8b4bef tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1564 microversion: 2.1 time: 0.159077 Oct 15 22:08:43.504223 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4409/6577] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:43 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1564 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:08:45.088618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:08:45.089348 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:08:45.097915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:45.098670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:45.099135 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:45.099893 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:45.100373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:45.100815 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:45.146059 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d52fc5a9-fe61-4e8f-b3c9-8095fd24d17b req-7fcac095-ff19-439c-a281-b7b67ea66dac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7c63dd3b-40bd-45ea-924e-8470f841db1e", "tag": "5a398623-e0ed-4c62-91af-9619a592ae0e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:08:45.156662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d52fc5a9-fe61-4e8f-b3c9-8095fd24d17b req-7fcac095-ff19-439c-a281-b7b67ea66dac service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:08:45.157180 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d52fc5a9-fe61-4e8f-b3c9-8095fd24d17b req-7fcac095-ff19-439c-a281-b7b67ea66dac service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:08:45.157618 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d52fc5a9-fe61-4e8f-b3c9-8095fd24d17b req-7fcac095-ff19-439c-a281-b7b67ea66dac service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:08:45.171443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d52fc5a9-fe61-4e8f-b3c9-8095fd24d17b req-7fcac095-ff19-439c-a281-b7b67ea66dac service nova] Creating event network-changed:5a398623-e0ed-4c62-91af-9619a592ae0e for instance 7c63dd3b-40bd-45ea-924e-8470f841db1e on nch2-master-1 Oct 15 22:08:45.175482 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d52fc5a9-fe61-4e8f-b3c9-8095fd24d17b req-7fcac095-ff19-439c-a281-b7b67ea66dac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086226 Oct 15 22:08:45.176507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4410/6578] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:08:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:08:45.259612 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:08:45.268806 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d41592e2-8793-43ba-87f2-586e4acf9b61 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1564 microversion: 2.1 time: 0.184551 Oct 15 22:08:45.269523 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2169/6579] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:08:45 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1564 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:16.552389 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:09:16.553075 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:09:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action (ip 10.222.0.16) !!! Oct 15 22:09:16.553501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:16.554249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=40 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:16.560736 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:16.563043 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:16.572765 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:16.573497 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:16.574102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:16.575078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:16.575678 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:16.576109 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:16.632396 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3b34e309-86cc-4ef1-a7f2-e14ad527782f tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action" status: 200 len: 24567 microversion: 2.1 time: 0.082366 Oct 15 22:09:16.633416 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4411/6580] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:09:16 2022] POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action => generated 24567 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:09:47.060822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:47.061611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=40 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:47.065933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:47.066487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:47.073902 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:47.074404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:47.074891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:47.075809 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:47.076290 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:47.076768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:47.100329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lazy-loading 'flavor' on Instance uuid 7c63dd3b-40bd-45ea-924e-8470f841db1e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:09:47.165456 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2120251a-a10c-4026-b73e-e89c9747dac8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action" status: 202 len: 0 microversion: 2.1 time: 0.107240 Oct 15 22:09:47.166091 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2170/6581] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:09:47 2022] POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action => generated 0 bytes in 109 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:09:47.175603 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:47.176315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:47.179130 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:47.179781 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=4 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:09:47.186306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:47.187351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:47.187801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:47.188516 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:47.188980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:47.190078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:47.326578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:47.334558 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3af2da3b-5ff5-48ae-ab77-2cff0ae30a03 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1691 microversion: 2.1 time: 0.163436 Oct 15 22:09:47.335120 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4412/6582] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:47 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1691 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:48.362488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:48.363215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:48.369501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:48.370003 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:48.370463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:48.371176 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:48.371652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:48.372102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:48.509934 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:48.518049 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fec30861-41c9-44af-b17d-8ac2559067d1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1691 microversion: 2.1 time: 0.161663 Oct 15 22:09:48.518616 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2171/6583] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:48 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1691 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:49.540942 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:49.541527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:49.549244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:49.549731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:49.550171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:49.550871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:49.551332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:49.551766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:49.684028 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:49.691931 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dbe6ec3d-d139-452d-8640-6ec364c8d0d7 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1674 microversion: 2.1 time: 0.154975 Oct 15 22:09:49.692507 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4413/6584] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:49 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1674 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:50.788983 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:50.791263 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:50.798278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:50.798786 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:50.799247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:50.799965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:50.800442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:50.800890 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:50.853992 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-818352cf-2216-4986-8d5d-38a4456e3853 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action" status: 200 len: 36373 microversion: 2.1 time: 0.069114 Oct 15 22:09:50.854655 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2172/6585] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:09:50 2022] POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action => generated 36373 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:09:50.866787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:50.867336 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:50.874784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:50.875286 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:50.875727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:50.876423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:50.876884 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:50.877345 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:50.943603 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79964107-e7b2-48fd-be8d-9b7d795b4a06 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action" status: 202 len: 0 microversion: 2.1 time: 0.080534 Oct 15 22:09:50.944239 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4414/6586] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:09:50 2022] POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:09:50.953782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:50.954583 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:50.962596 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:50.963231 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:50.963689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:50.964423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:50.964885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:50.965355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:51.102984 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:51.110904 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4b747152-cbc8-4081-9d7a-5fd289d87634 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1680 microversion: 2.1 time: 0.162099 Oct 15 22:09:51.111481 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2173/6587] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:50 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:52.331824 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:52.332541 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:52.338855 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:52.339347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:52.339792 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:52.340492 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:52.340954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:52.341412 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:52.478582 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:52.486548 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-72b7ac43-bca4-42e2-814d-6bc00a2d6eee tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1680 microversion: 2.1 time: 0.355367 Oct 15 22:09:52.487126 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4415/6588] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:52 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1680 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:53.510295 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:53.510851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:53.519570 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:53.520963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:53.520963 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:53.521358 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:53.521813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:53.522316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:53.663663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:53.671496 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa4034bf-402d-4fb8-8a78-204b49e5fdf1 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1683 microversion: 2.1 time: 0.165163 Oct 15 22:09:53.672069 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2174/6589] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:53 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1683 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:53.767811 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:53.770071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:53.776936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:53.777476 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:53.778295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:53.778634 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:53.779101 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:53.779533 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:53.821168 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f2d4633f-d2d9-4897-96bf-56ae9272b203 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action" status: 200 len: 36373 microversion: 2.1 time: 0.057597 Oct 15 22:09:53.821825 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4416/6590] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:09:53 2022] POST /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e/action => generated 36373 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 15 22:09:55.149364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:55.150005 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:55.152752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-031544e9-4e2d-4813-ac1b-76320bfdbada req-485e70cd-e973-433e-92d1-845756febe4f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7c63dd3b-40bd-45ea-924e-8470f841db1e", "tag": "5a398623-e0ed-4c62-91af-9619a592ae0e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:55.157797 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:55.158858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:55.159678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:55.160880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:55.161452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:55.162084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:55.164858 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-031544e9-4e2d-4813-ac1b-76320bfdbada req-485e70cd-e973-433e-92d1-845756febe4f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:55.165396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-031544e9-4e2d-4813-ac1b-76320bfdbada req-485e70cd-e973-433e-92d1-845756febe4f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:55.165832 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-031544e9-4e2d-4813-ac1b-76320bfdbada req-485e70cd-e973-433e-92d1-845756febe4f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:55.178888 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-031544e9-4e2d-4813-ac1b-76320bfdbada req-485e70cd-e973-433e-92d1-845756febe4f service nova] Creating event network-changed:5a398623-e0ed-4c62-91af-9619a592ae0e for instance 7c63dd3b-40bd-45ea-924e-8470f841db1e on nch2-master-1 Oct 15 22:09:55.184125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-031544e9-4e2d-4813-ac1b-76320bfdbada req-485e70cd-e973-433e-92d1-845756febe4f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035502 Oct 15 22:09:55.185357 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4417/6591] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:09:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:09:55.193829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:09:55.279715 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d78550e8-4e71-4024-9e9a-1ff8f81fd4ae tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 204 len: 0 microversion: 2.1 time: 0.134099 Oct 15 22:09:55.280765 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2175/6592] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:09:55 2022] DELETE /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:09:55.292148 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:55.292863 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:55.299871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:55.300387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:55.300812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:55.302044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:55.302510 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:55.302948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:55.448536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:55.456687 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fa874eb6-26f8-4b18-93fa-f9b4ee11d9c8 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1689 microversion: 2.1 time: 0.169240 Oct 15 22:09:55.457510 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4418/6593] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:55 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1689 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:56.483559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:56.484175 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:56.491558 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:56.492069 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:56.492526 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:56.493271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:56.493750 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:56.494198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:56.642807 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:56.650595 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-fbf751d0-a956-47f1-ad2f-066fea818abd tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1689 microversion: 2.1 time: 0.171461 Oct 15 22:09:56.651158 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2176/6594] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:56 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1689 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:57.487562 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ec909bf4-681f-4eac-9cbb-3aa2937d88fd req-e0b2fb63-c33f-426f-b6a2-a0e2d875fb05 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7c63dd3b-40bd-45ea-924e-8470f841db1e", "name": "network-vif-deleted", "tag": "5a398623-e0ed-4c62-91af-9619a592ae0e"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:09:57.499812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec909bf4-681f-4eac-9cbb-3aa2937d88fd req-e0b2fb63-c33f-426f-b6a2-a0e2d875fb05 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:57.500298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec909bf4-681f-4eac-9cbb-3aa2937d88fd req-e0b2fb63-c33f-426f-b6a2-a0e2d875fb05 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:57.500722 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ec909bf4-681f-4eac-9cbb-3aa2937d88fd req-e0b2fb63-c33f-426f-b6a2-a0e2d875fb05 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:57.513755 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ec909bf4-681f-4eac-9cbb-3aa2937d88fd req-e0b2fb63-c33f-426f-b6a2-a0e2d875fb05 service nova] Creating event network-vif-deleted:5a398623-e0ed-4c62-91af-9619a592ae0e for instance 7c63dd3b-40bd-45ea-924e-8470f841db1e on nch2-master-1 Oct 15 22:09:57.519992 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ec909bf4-681f-4eac-9cbb-3aa2937d88fd req-e0b2fb63-c33f-426f-b6a2-a0e2d875fb05 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038603 Oct 15 22:09:57.520600 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4419/6595] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:09:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:09:57.675398 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:57.676496 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:57.683789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:57.684489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:57.684960 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:57.685760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:57.686200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:57.686659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:57.722130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:57.722766 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:57.723100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:57.723584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lazy-loading 'fault' on Instance uuid 7c63dd3b-40bd-45ea-924e-8470f841db1e {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:09:57.775808 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823ad7d2979042c188525f5341febdff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ea5345e230164cd6ae5b1195425e7ff0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:09:57.786025 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6bb784b7-30f7-449e-bfc5-4fff1668d706 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 200 len: 1356 microversion: 2.1 time: 0.114911 Oct 15 22:09:57.787043 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2177/6596] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:57 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 1356 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:09:58.812415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:09:58.813189 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] [instance: 7c63dd3b-40bd-45ea-924e-8470f841db1e] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:09:58.820923 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:58.821447 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:58.821893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:58.822585 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:09:58.823044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:09:58.823470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:09:58.837824 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] HTTP exception thrown: Instance 7c63dd3b-40bd-45ea-924e-8470f841db1e could not be found. Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Request method failure captured: Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e HTTP/1.1 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSy09-QfPZ-9CNDIIFxEAKgOvMyzS_BGx2YAooBKnz2qBEmy5yraK8uUQYc8ul_plXjHeNR8-6n07eBm4u9lCM3KGAMnrogPl4-d9DQbXxkTgXgwzrU8kkhbIZtFBlypgzMCEjP9tyrjP6od2c2gsCfU2f50omy2qrKGIa7h0T3j2Xp0 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ea5345e230164cd6ae5b1195425e7ff0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: ea5345e230164cd6ae5b1195425e7ff0 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-2018164608 Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 823ad7d2979042c188525f5341febdff Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestNetworkAdvancedServerOps-2018164608-project Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 7c63dd3b-40bd-45ea-924e-8470f841db1e could not be found. Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '7c63dd3b-40bd-45ea-924e-8470f841db1e'} Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7c63dd3b-40bd-45ea-924e-8470f841db1e could not be found. Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7c63dd3b-40bd-45ea-924e-8470f841db1e could not be found. Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:09:58.839432 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7c63dd3b-40bd-45ea-924e-8470f841db1e could not be found. Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Returning 404 to user: Instance 7c63dd3b-40bd-45ea-924e-8470f841db1e could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b1e83072-0a14-41a1-9305-9740361f638e tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e" status: 404 len: 111 microversion: 2.1 time: 0.032773 Oct 15 22:09:58.845205 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4420/6597] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:09:58 2022] GET /compute/v2.1/servers/7c63dd3b-40bd-45ea-924e-8470f841db1e => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:10:01.901488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c0ce57b-67af-48b1-a68a-5bbc2119e996 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] Calling method '.version_select at 0x7fe74ec465e0>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:01.930923 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c0ce57b-67af-48b1-a68a-5bbc2119e996 tempest-TestNetworkAdvancedServerOps-2018164608 tempest-TestNetworkAdvancedServerOps-2018164608-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-815424250" status: 202 len: 0 microversion: 2.1 time: 0.033276 Oct 15 22:10:01.931648 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2178/6598] 10.222.0.16 () {58 vars in 1281 bytes} [Sat Oct 15 22:10:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-815424250 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:10:13.034580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=18 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:13.035806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=44,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:13.209482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-26904255-edb7-4d54-b67c-df6d1457547f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'create', calling method: .version_select at 0x7fe74e063550>, body: {"keypair": {"name": "tempest-TestShelveInstance-531387990", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBddhwxhh0e/7j+epM5+fnVIwm8oJUb68GhuolEM0uVs27j4546r/X9mJDBuOReg7yBG/KgGBV0GKIv7YvDWLuK/2CdgODTR2TqBdT3Cx+vVk0iUj3Md/JU7/pQkZyeMSg=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:10:13.225957 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-26904255-edb7-4d54-b67c-df6d1457547f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:10:13.232107 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-26904255-edb7-4d54-b67c-df6d1457547f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for user 6ddf65a9073b4abba4c8d09fed8a1ea6 and project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:10:13.257354 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-26904255-edb7-4d54-b67c-df6d1457547f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.228923 Oct 15 22:10:13.258192 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4421/6599] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:10:13 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:10:14.342966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:14.343734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:14.348568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1542719221", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-531387990", "security_groups": [{"name": "tempest-secgroup-smoke-689220394"}], "networks": [{"uuid": "c052605f-592c-40e2-90fb-3727407e1ca5"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:10:14.457230 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] validate_networks() for [('c052605f-592c-40e2-90fb-3727407e1ca5', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:10:14.967325 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:10:14.967864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:10:14.968431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:10:14.968930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:10:14.969452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:10:14.979368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:14.979880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:14.980340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:14.981128 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:14.981611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:14.982058 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:15.001878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:10:15.007640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for user 6ddf65a9073b4abba4c8d09fed8a1ea6 and project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:10:15.022059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:10:15.024501 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:10:15.076556 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1e85db6a-7f53-4d3f-a959-fe702f267d28 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.735465 Oct 15 22:10:15.077556 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2179/6600] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:10:14 2022] POST /compute/v2.1/servers => generated 405 bytes in 737 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:10:15.089266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52b649da-9e70-40b3-975d-a5e6bf7005a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:15.089893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52b649da-9e70-40b3-975d-a5e6bf7005a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:15.152996 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52b649da-9e70-40b3-975d-a5e6bf7005a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:15.161688 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52b649da-9e70-40b3-975d-a5e6bf7005a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1246 microversion: 2.1 time: 0.076298 Oct 15 22:10:15.162276 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4422/6601] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:15 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1246 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:16.185739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:16.186541 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:16.195491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:16.196377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:16.197100 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:16.198066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:16.198767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:16.199587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:16.286606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:16.297273 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d6a2eebe-a8c8-43b9-b042-0d8be3550127 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1316 microversion: 2.1 time: 0.114733 Oct 15 22:10:16.298250 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2180/6602] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:16 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1316 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:17.322692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:17.323418 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:17.332892 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:17.333951 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:17.334473 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:17.335391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:17.335973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:17.336483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:17.547472 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:17.556514 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c713151b-1d22-43d1-85a3-c5951e2367c3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.237940 Oct 15 22:10:17.557729 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4423/6603] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:17 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:17.691331 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-babfc678-c992-4634-a096-6f5cde6598de req-4091faa6-2d77-4823-b2b1-34cb139c83f5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:10:17.705304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-babfc678-c992-4634-a096-6f5cde6598de req-4091faa6-2d77-4823-b2b1-34cb139c83f5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:17.706186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-babfc678-c992-4634-a096-6f5cde6598de req-4091faa6-2d77-4823-b2b1-34cb139c83f5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:17.707584 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-babfc678-c992-4634-a096-6f5cde6598de req-4091faa6-2d77-4823-b2b1-34cb139c83f5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:17.724665 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-babfc678-c992-4634-a096-6f5cde6598de req-4091faa6-2d77-4823-b2b1-34cb139c83f5 service nova] Creating event network-changed:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:10:17.730056 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-babfc678-c992-4634-a096-6f5cde6598de req-4091faa6-2d77-4823-b2b1-34cb139c83f5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044829 Oct 15 22:10:17.730687 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2181/6604] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:10:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:10:18.581394 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:18.582007 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:18.589376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:18.589867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:18.590304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:18.591006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:18.591461 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:18.591888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:18.726018 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:18.733944 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6871bd2-d696-4ba5-b53b-ff8dfdbe77dd tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.156988 Oct 15 22:10:18.734543 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4424/6605] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:18 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:19.759992 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:19.760708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:19.769063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:19.769723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:19.770250 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:19.771015 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:19.771491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:19.771937 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:19.915254 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:19.923168 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-82b953f8-8f11-4815-8aa6-ec090437e146 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.168258 Oct 15 22:10:19.923778 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2182/6606] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:19 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:20.946232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:20.946768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:20.953750 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:20.954244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:20.954684 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:20.955888 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:20.956453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:20.956893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:21.086331 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:21.094086 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-544ed34b-9097-48f7-9133-54ec2fbaadf3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.151884 Oct 15 22:10:21.094661 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4425/6607] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:20 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:22.116908 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:22.117485 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:22.124343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:22.124846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:22.125330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:22.126045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:22.126519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:22.127458 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:22.259204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:22.267070 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5afc1c09-13ff-4a5d-a7a5-7b7a083082ca tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.154232 Oct 15 22:10:22.267667 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2183/6608] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:22 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:23.290688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:23.291306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:23.298580 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:23.299112 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:23.299705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:23.300434 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:23.300900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:23.301381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:23.430822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5130b25b-fa11-492e-9afc-2616af74d9ca req-9116a9f7-83f9-48c8-9a22-056642e5046d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "name": "network-vif-plugged", "status": "completed", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:10:23.442767 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5130b25b-fa11-492e-9afc-2616af74d9ca req-9116a9f7-83f9-48c8-9a22-056642e5046d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:23.443672 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5130b25b-fa11-492e-9afc-2616af74d9ca req-9116a9f7-83f9-48c8-9a22-056642e5046d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:23.443891 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5130b25b-fa11-492e-9afc-2616af74d9ca req-9116a9f7-83f9-48c8-9a22-056642e5046d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:23.456365 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5130b25b-fa11-492e-9afc-2616af74d9ca req-9116a9f7-83f9-48c8-9a22-056642e5046d service nova] Creating event network-vif-plugged:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:10:23.459624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:23.462003 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5130b25b-fa11-492e-9afc-2616af74d9ca req-9116a9f7-83f9-48c8-9a22-056642e5046d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109554 Oct 15 22:10:23.463000 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2184/6609] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:10:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:10:23.469665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6de9b795-1c43-43e6-b907-5b34730667bb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.182773 Oct 15 22:10:23.471170 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4426/6610] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:23 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:24.496523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:24.497483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:24.504099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:24.504768 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:24.505264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:24.505996 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:24.506473 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:24.506922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:24.644586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:24.652280 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-25d51bce-2364-43df-a81d-5578300a5c3d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1371 microversion: 2.1 time: 0.160920 Oct 15 22:10:24.652860 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2185/6611] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:24 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:25.677235 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:25.678070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:25.680776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:25.686128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:25.686621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:25.687074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:25.687782 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:25.688249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:25.688682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:25.825963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:25.833910 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2ed77af-c19a-4fd3-9e73-7a1d14bbfedb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1555 microversion: 2.1 time: 0.161124 Oct 15 22:10:25.834527 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4427/6612] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:25 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1555 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:25.856861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:10:25.857647 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:10:25.859629 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:25.860327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:10:25.867813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:25.868321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:25.868776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:25.869520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:25.870002 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:25.870448 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:26.004557 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:10:26.012553 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70371855-14b4-4889-9daa-2bbed62c8864 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1555 microversion: 2.1 time: 0.159545 Oct 15 22:10:26.013156 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2186/6613] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:10:25 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1555 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:10:27.961369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-ee3803c5-013c-4528-a1bd-88b32bf8823d req-c00baea3-4617-4dbe-971e-7e9b144d8c73 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:10:27.971749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee3803c5-013c-4528-a1bd-88b32bf8823d req-c00baea3-4617-4dbe-971e-7e9b144d8c73 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:10:27.972867 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee3803c5-013c-4528-a1bd-88b32bf8823d req-c00baea3-4617-4dbe-971e-7e9b144d8c73 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:10:27.973357 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-ee3803c5-013c-4528-a1bd-88b32bf8823d req-c00baea3-4617-4dbe-971e-7e9b144d8c73 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:10:27.984607 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-ee3803c5-013c-4528-a1bd-88b32bf8823d req-c00baea3-4617-4dbe-971e-7e9b144d8c73 service nova] Creating event network-changed:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:10:27.990537 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-ee3803c5-013c-4528-a1bd-88b32bf8823d req-c00baea3-4617-4dbe-971e-7e9b144d8c73 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033036 Oct 15 22:10:27.991201 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4428/6614] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:10:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:12:53.971891 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 22:12:53.972882 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 22:12:53.973641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:12:53.974542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=36 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:12:53.976146 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:12:53.976393 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 22:12:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7/action (ip 10.222.0.16) !!! Oct 15 22:12:53.977781 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:12:53.977910 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 22:12:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7/action (ip 10.222.0.16) !!! Oct 15 22:12:53.982853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:12:53.983409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:12:53.993986 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:53.994487 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:53.994942 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:53.995651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:53.997189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:53.997640 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:54.022950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lazy-loading 'flavor' on Instance uuid 286509c1-d419-46f5-bf2a-5c7ee6a576f7 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:12:54.215957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9854fc16-8930-4a9d-805d-98dbf06e8aa7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7/action" status: 202 len: 0 microversion: 2.1 time: 0.245671 Oct 15 22:12:54.216556 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2187/6615] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:12:53 2022] POST /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7/action => generated 0 bytes in 247 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:12:54.223782 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [8a327d79-84a5-4eee-8d31-7df1f6cabc7a] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:12:54.224590 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:12:54.225188 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:12:54.226099 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=32 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:12:54.226948 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:12:54.227081 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:12:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 (ip 10.222.0.16) !!! Oct 15 22:12:54.230062 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:12:54.230253 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:12:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 (ip 10.222.0.16) !!! Oct 15 22:12:54.232570 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:12:54.232717 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:12:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 (ip 10.222.0.16) !!! Oct 15 22:12:54.241806 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:12:54.242513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:12:54.254415 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:54.254901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:54.255341 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:54.256040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:54.256500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:54.256926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:54.476785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:12:54.484795 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-153a81c3-e1d5-4a07-b241-906f5ebc67d4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.262855 Oct 15 22:12:54.485421 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4429/6616] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:12:54 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:12:55.513772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:12:55.514403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:12:55.528734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:55.529298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:55.529746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:55.530450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:55.530915 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:55.531352 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:55.532055 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 47908. Oct 15 22:12:55.532861 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 47910. Oct 15 22:12:55.668864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:12:55.676764 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9eeffc97-23d0-415a-acf4-f7603799d95a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.170917 Oct 15 22:12:55.677369 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2188/6617] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:12:55 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:12:56.701731 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:12:56.702348 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:12:56.712154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:56.712643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:56.713114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:56.713810 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:56.714266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:56.714691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:56.719070 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [8a327d79-84a5-4eee-8d31-7df1f6cabc7a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 47914. Oct 15 22:12:56.722750 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 47916. Oct 15 22:12:56.856369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:12:56.864303 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-147f8095-6f15-42d4-a7a1-5d5363f2e850 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.167195 Oct 15 22:12:56.865105 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4430/6618] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:12:56 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:12:57.887829 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:12:57.888343 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:12:57.895435 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:57.895947 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:57.896409 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:57.897173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:57.897656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:57.898106 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:58.031326 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:12:58.039431 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dc9eda09-60f0-40d1-8685-63f924b97da8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.155639 Oct 15 22:12:58.040009 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2189/6619] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:12:57 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:12:59.065222 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:12:59.066064 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:12:59.073445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:59.073928 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:59.074851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:59.075549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:12:59.076006 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:12:59.076429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:12:59.220421 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:12:59.228113 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-390908a8-7046-470e-9048-ce7486572abb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.168278 Oct 15 22:12:59.228687 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4431/6620] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:12:59 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:00.251681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:00.252320 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:00.259191 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:00.259701 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:00.260157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:00.260877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:00.261410 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:00.261919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:00.393816 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:00.401690 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5004ad19-e9ba-4cc0-ad10-8b95df042265 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.154001 Oct 15 22:13:00.402258 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2190/6621] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:00 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:01.426023 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:01.426674 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:01.433625 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:01.434123 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:01.434567 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:01.435275 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:01.435739 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:01.436177 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:01.565900 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:01.573573 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a6bede16-6157-4fe4-a212-d955f8885d1d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.151534 Oct 15 22:13:01.574768 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4432/6622] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:01 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:02.598678 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:02.599236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:02.606988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:02.607489 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:02.607971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:02.608674 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:02.609180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:02.609623 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:02.742961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:02.751804 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52f66f8a-d798-457a-87e1-9069edb46b50 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.157685 Oct 15 22:13:02.752377 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2191/6623] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:02 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:03.774870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:03.775489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:03.783169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:03.783641 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:03.784084 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:03.784784 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:03.785273 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:03.785708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:03.917651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:03.925968 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0d88e240-24a7-4a8b-bee0-aba53fe6fff4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.155066 Oct 15 22:13:03.926539 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4433/6624] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:03 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:04.949845 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:04.950534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:04.957856 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:04.958389 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:04.958972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:04.959699 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:04.960174 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:04.960618 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:05.095259 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:05.103148 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5b2e61dc-0bf0-4a20-a9f0-ea98c9ecc5de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.157861 Oct 15 22:13:05.103717 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2192/6625] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:04 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:06.127063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:06.127621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:06.134650 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:06.135129 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:06.135565 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:06.136256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:06.136827 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:06.137287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:06.270699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:06.278945 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7a7d7f3b-73df-4576-b3f6-94ca58d5ab78 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1701 microversion: 2.1 time: 0.156014 Oct 15 22:13:06.279523 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4434/6626] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:06 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1701 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:07.303719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:07.304381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:07.311879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:07.312386 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:07.312842 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:07.313586 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:07.314068 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:07.314516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:07.449969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:07.457883 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7b71ab1c-0a60-45f7-9f9a-fe2751b67323 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1701 microversion: 2.1 time: 0.158019 Oct 15 22:13:07.458392 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2193/6627] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:07 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1701 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:08.482423 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:08.482970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:08.489839 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:08.490323 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:08.490757 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:08.491458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:08.491916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:08.492344 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:08.622127 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:08.629837 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8367904c-b5c3-4afb-b2c8-e51d2c5024cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1701 microversion: 2.1 time: 0.151992 Oct 15 22:13:08.630409 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4435/6628] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:08 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1701 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:09.654637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:09.655289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:09.662084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:09.662578 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:09.663054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:09.664012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:09.664480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:09.664919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:09.799728 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:09.807617 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-70dd60e9-0867-4431-b6bb-f79c0b237fb6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1701 microversion: 2.1 time: 0.157161 Oct 15 22:13:09.808205 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2194/6629] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:09 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1701 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:10.188236 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e59e76b3-069e-442e-b1ce-2e3a74bb8876 req-4d226838-4650-4f49-9065-590712240286 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "name": "network-vif-plugged", "status": "completed", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:13:10.200154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e59e76b3-069e-442e-b1ce-2e3a74bb8876 req-4d226838-4650-4f49-9065-590712240286 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:10.200686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e59e76b3-069e-442e-b1ce-2e3a74bb8876 req-4d226838-4650-4f49-9065-590712240286 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:10.201120 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e59e76b3-069e-442e-b1ce-2e3a74bb8876 req-4d226838-4650-4f49-9065-590712240286 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:10.213380 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e59e76b3-069e-442e-b1ce-2e3a74bb8876 req-4d226838-4650-4f49-9065-590712240286 service nova] Creating event network-vif-plugged:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:13:10.220097 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e59e76b3-069e-442e-b1ce-2e3a74bb8876 req-4d226838-4650-4f49-9065-590712240286 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036445 Oct 15 22:13:10.220896 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4436/6630] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:13:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:13:10.831877 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:10.832504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:10.839330 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:10.839831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:10.840396 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:10.841139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:10.841619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:10.842061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:10.982804 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:10.990842 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-4361a7cd-df74-4aaa-857e-77d79567408e tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1701 microversion: 2.1 time: 0.162935 Oct 15 22:13:10.991503 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2195/6631] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:10 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1701 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:12.017744 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:12.018452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:12.025081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:12.025651 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:12.026931 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:12.028936 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:12.029431 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:12.030082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:12.177658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:12.186392 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-9209db12-e50a-4d57-9f74-2cf0da1fcbd1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.173910 Oct 15 22:13:12.187086 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4437/6632] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:12 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:13.209690 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:13.211765 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:13.219734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:13.220316 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:13.220874 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:13.221619 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:13.222237 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:13.222655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:13.375235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:13.384141 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-28da19a7-c024-4c4b-a954-905f766cd801 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.178272 Oct 15 22:13:13.397341 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2196/6633] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:13 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:14.408481 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:14.409137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:14.417134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:14.417814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:14.418262 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:14.419175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:14.419640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:14.420074 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:14.576914 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:14.585478 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-615c8921-1bdd-44d7-b77e-136ab1b3cc24 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.180538 Oct 15 22:13:14.587511 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4438/6634] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:14 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:15.612878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:15.613523 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:15.621440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:15.623471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:15.625190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:15.626405 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:15.626957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:15.627533 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:15.772385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:15.781707 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-c7830f05-a6c2-4572-88b7-c17897209721 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.172989 Oct 15 22:13:15.782537 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2197/6635] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:15 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:16.806315 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:16.807243 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:16.815240 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:16.815745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:16.816191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:16.816921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:16.817413 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:16.817851 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:16.969850 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:16.980907 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-22a50585-ddd3-4d12-b182-2b5176ba3e51 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.177982 Oct 15 22:13:16.981865 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4439/6636] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:16 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:18.006718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:18.007998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:18.016193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:18.016918 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:18.017471 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:18.018209 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:18.018684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:18.019327 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:18.175283 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:18.187039 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-dbb0ec6a-7264-467b-a8ab-f08886d77e01 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.184211 Oct 15 22:13:18.187991 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2198/6637] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:18 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:19.212595 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:19.213249 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:19.219891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:19.220399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:19.220869 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:19.221604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:19.222070 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:19.222671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:19.378436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:19.387818 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3deaaaa3-7dce-4396-a948-2052c9821bf0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.180015 Oct 15 22:13:19.388559 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4440/6638] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:19 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:20.411803 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:20.412452 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:20.421159 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:20.422051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:20.422517 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:20.423241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:20.423717 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:20.424167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:20.564442 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:20.572254 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b1985457-9017-418b-8f37-95f81ff633cb tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1696 microversion: 2.1 time: 0.164480 Oct 15 22:13:20.572835 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2199/6639] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:20 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1696 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:21.596527 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:21.597284 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:21.604502 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:21.605498 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:21.605938 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:21.606703 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:21.607110 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:21.607535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:21.955160 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:21.963024 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ecc5cfb0-3496-4d5f-b9cf-4beb239d7dff tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1678 microversion: 2.1 time: 0.370692 Oct 15 22:13:21.963631 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4441/6640] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:21 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1678 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:22.987064 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:22.987669 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:22.994539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:22.995033 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:22.995482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:22.996196 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:22.996661 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:22.997121 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:23.129313 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:23.138013 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-72ca7666-2ee2-460b-a4fc-9bdb01fd6816 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1678 microversion: 2.1 time: 0.154815 Oct 15 22:13:23.138596 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2200/6641] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:22 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1678 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:24.162975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:24.163574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:24.171615 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.172214 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.172664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.173438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.173903 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.174339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.216284 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-79c30103-8a0a-4655-b59d-1a611de5ee69 req-026ab6ed-7e08-462c-85c1-f68c9ac9951f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "name": "network-vif-unplugged", "status": "completed", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:13:24.223539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=29 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:24.228853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-79c30103-8a0a-4655-b59d-1a611de5ee69 req-026ab6ed-7e08-462c-85c1-f68c9ac9951f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.229653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-79c30103-8a0a-4655-b59d-1a611de5ee69 req-026ab6ed-7e08-462c-85c1-f68c9ac9951f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.230160 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-79c30103-8a0a-4655-b59d-1a611de5ee69 req-026ab6ed-7e08-462c-85c1-f68c9ac9951f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.234955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:24.244292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.compute.server_external_events [req-79c30103-8a0a-4655-b59d-1a611de5ee69 req-026ab6ed-7e08-462c-85c1-f68c9ac9951f service nova] Unable to find a host for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7. Dropping event network-vif-unplugged {{(pid=138689) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 15 22:13:24.246046 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-79c30103-8a0a-4655-b59d-1a611de5ee69 req-026ab6ed-7e08-462c-85c1-f68c9ac9951f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.033257 Oct 15 22:13:24.246727 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2201/6642] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:13:24 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 35 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 15 22:13:24.316548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:24.318901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=79 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:24.326674 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a0c9f5b9-e20d-4575-ba36-bb781ee40423 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1621 microversion: 2.1 time: 0.168021 Oct 15 22:13:24.327263 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4442/6643] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:24 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1621 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:24.349542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:13:24.350157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:24.357549 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.357990 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.358441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.359166 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.359637 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.360077 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.402685 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=138689) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 22:13:24.471685 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d9faa19a-fa82-49d9-9c76-d8b94d6756b9 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7/action" status: 202 len: 0 microversion: 2.1 time: 0.125654 Oct 15 22:13:24.472338 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2202/6644] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:13:24 2022] POST /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7/action => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:13:24.481640 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:24.482288 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:24.484381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=29 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:13:24.491626 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.492448 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.492937 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.493990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:24.494453 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:24.494890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:24.639241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:24.647546 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-52851678-bc3e-4621-8637-fdda3a8e20c5 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1629 microversion: 2.1 time: 0.170738 Oct 15 22:13:24.648123 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4443/6645] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:24 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1629 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:25.672604 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:25.673294 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:25.679368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:25.679864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:25.680312 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:25.681010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:25.681498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:25.681936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:25.846830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:25.855296 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-3fedb7b9-3315-4735-b356-21c711783b3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.186400 Oct 15 22:13:25.855878 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2203/6646] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:25 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:26.880256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:26.881689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:26.888821 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:26.889482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:26.890026 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:26.891022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:26.891604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:26.892140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:27.052443 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:27.066277 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-d8b1bd04-f3fc-43bf-ac99-90af7a5ff080 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.190034 Oct 15 22:13:27.067148 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4444/6647] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:26 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:28.091097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:28.091748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:28.099044 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:28.099547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:28.100012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:28.100711 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:28.101220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:28.101659 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:28.252780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:28.262109 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-aa122f1a-1483-4f24-8783-b386b7b3491f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.174785 Oct 15 22:13:28.262975 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2204/6648] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:28 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:29.287572 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:29.288186 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:29.306966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:29.307478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:29.308219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:29.308995 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:29.309490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:29.309927 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:29.459662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:29.468640 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-947d18d0-6b13-42da-b117-2f6226def893 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.184893 Oct 15 22:13:29.469442 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4445/6649] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:29 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:30.494553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:30.495187 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:30.503884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:30.504364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:30.504833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:30.505606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:30.506087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:30.506534 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:30.661520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:30.670334 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-608c9268-b165-4fb7-b1c6-4e3a7101fc6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.179652 Oct 15 22:13:30.670917 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2205/6650] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:30 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:31.694439 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:31.695111 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:31.702209 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:31.702695 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:31.703132 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:31.703836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:31.704295 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:31.704720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:31.846774 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:31.856620 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-0fa12ac9-1933-4910-8ecd-bcafd1f83ec3 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.165946 Oct 15 22:13:31.857452 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4446/6651] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:31 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:32.880441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:32.881153 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:32.890453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:32.890964 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:32.891420 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:32.892277 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:32.892757 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:32.893232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:33.097665 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:33.106841 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-819933ae-4821-41d4-85bd-811fca502a3f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.229570 Oct 15 22:13:33.107669 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2206/6652] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:32 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:34.130692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:34.131395 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:34.138483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:34.138962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:34.139399 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:34.140088 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:34.140544 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:34.140968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:34.274140 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:34.282272 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-af783821-eff0-4ab8-baf4-aed99c2540b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.155448 Oct 15 22:13:34.282848 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4447/6653] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:34 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:35.310309 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:35.310935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:35.317965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:35.318461 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:35.318914 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:35.319625 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:35.320101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:35.320542 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:35.456864 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:35.464714 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-52523eed-d01e-4bad-b611-6b8a2319e38d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.159035 Oct 15 22:13:35.465387 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2207/6654] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:35 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:36.489317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:36.489963 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:36.496823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:36.497347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:36.497785 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:36.498482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:36.498943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:36.499372 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:36.638141 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:36.645794 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3cf6e42d-0d67-41ee-9477-4920121783a2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.160523 Oct 15 22:13:36.646369 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4448/6655] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:36 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:37.669824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:37.670483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:37.677470 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:37.677965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:37.678406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:37.679104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:37.679574 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:37.680004 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:37.810339 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:37.818503 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-df104136-b0c8-4e28-9804-9e8935c56ee7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.153088 Oct 15 22:13:37.819087 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2208/6656] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:37 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:38.842848 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:38.843504 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:38.850636 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:38.851122 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:38.851566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:38.852264 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:38.852730 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:38.853190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:38.990272 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:38.998077 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e96b0d25-6d9f-4414-9e3a-d2fb8a51bdac tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.159233 Oct 15 22:13:38.998659 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4449/6657] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:38 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:40.098256 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:40.098882 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:40.105738 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:40.106236 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:40.107202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:40.107976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:40.108453 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:40.108894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:40.241057 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:40.249453 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bcc5b268-5271-4180-bcce-b06460a727c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.155173 Oct 15 22:13:40.250027 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2209/6658] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:40 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:41.273945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:41.274559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:41.282449 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:41.282932 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:41.283366 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:41.284056 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:41.284514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:41.284945 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:41.420204 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:41.428125 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-95b33fc4-f54e-4304-9357-72c9102ec2db tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.158054 Oct 15 22:13:41.428711 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4450/6659] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:41 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:42.453225 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:42.453781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:42.460606 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:42.461139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:42.461589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:42.462298 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:42.462764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:42.463202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:42.610186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:42.618899 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d14ec2d0-b42e-45a9-8c04-9d7116744bb4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.169802 Oct 15 22:13:42.619801 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2210/6660] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:42 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:43.643282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:43.643912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:43.650670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:43.651156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:43.651598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:43.652292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:43.652752 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:43.653215 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:43.784727 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:43.792700 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ade349cd-f9e7-4003-80d0-6646d496ecbf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.153363 Oct 15 22:13:43.793308 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4451/6661] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:43 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:44.341633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-51cb2493-956b-42d5-9ed4-84ebc833e20b req-b0983e9e-4ccf-485d-936b-b727dfa1fdcd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "name": "network-vif-plugged", "status": "completed", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:13:44.352445 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-51cb2493-956b-42d5-9ed4-84ebc833e20b req-b0983e9e-4ccf-485d-936b-b727dfa1fdcd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:44.352944 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-51cb2493-956b-42d5-9ed4-84ebc833e20b req-b0983e9e-4ccf-485d-936b-b727dfa1fdcd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:44.353423 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-51cb2493-956b-42d5-9ed4-84ebc833e20b req-b0983e9e-4ccf-485d-936b-b727dfa1fdcd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:44.365841 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-51cb2493-956b-42d5-9ed4-84ebc833e20b req-b0983e9e-4ccf-485d-936b-b727dfa1fdcd service nova] Creating event network-vif-plugged:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:13:44.374664 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-51cb2493-956b-42d5-9ed4-84ebc833e20b req-b0983e9e-4ccf-485d-936b-b727dfa1fdcd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037269 Oct 15 22:13:44.375341 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2211/6662] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:13:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:13:44.816073 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:44.816629 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:44.823487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:44.823968 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:44.824404 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:44.825622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:44.826090 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:44.826513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:44.960604 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:44.968951 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-14e859a5-0ae7-42b4-a1d0-ab5625363d9d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.156879 Oct 15 22:13:44.969542 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4452/6663] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:44 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:45.994504 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:45.995338 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:46.002919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:46.003426 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:46.003876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:46.004589 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:46.005096 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:46.005539 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:46.150520 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:46.159777 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-891d0982-d178-4a48-be6b-5646dd2743c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1687 microversion: 2.1 time: 0.170300 Oct 15 22:13:46.166960 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2212/6664] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:45 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1687 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:13:47.185376 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:13:47.186071 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:13:47.193022 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:47.193545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:47.193982 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:47.194689 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:13:47.195151 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:13:47.195578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:13:47.324818 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:13:47.333779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5b6c1726-7259-4f35-9ef5-c95a4378ad86 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1674 microversion: 2.1 time: 0.152782 Oct 15 22:13:47.334352 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4453/6665] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:13:47 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1674 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:23.800408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:23.801404 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:23.803609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=21 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:23.805095 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=67 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:23.807950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:23.808726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:23.816965 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:23.817518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:23.817981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:23.818718 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:23.819198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:23.819648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:23.855758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:15:23.874135 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-e3f14f0e-b1f4-4365-8061-c55fe0fa98e3 req-3d678159-a9ec-4036-a0d6-413aa3bc659a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:23.885167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e3f14f0e-b1f4-4365-8061-c55fe0fa98e3 req-3d678159-a9ec-4036-a0d6-413aa3bc659a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:23.885658 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e3f14f0e-b1f4-4365-8061-c55fe0fa98e3 req-3d678159-a9ec-4036-a0d6-413aa3bc659a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:23.886314 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-e3f14f0e-b1f4-4365-8061-c55fe0fa98e3 req-3d678159-a9ec-4036-a0d6-413aa3bc659a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:23.900351 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-e3f14f0e-b1f4-4365-8061-c55fe0fa98e3 req-3d678159-a9ec-4036-a0d6-413aa3bc659a service nova] Creating event network-changed:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:15:23.906809 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-e3f14f0e-b1f4-4365-8061-c55fe0fa98e3 req-3d678159-a9ec-4036-a0d6-413aa3bc659a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.105743 Oct 15 22:15:23.908455 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4454/6666] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:15:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:15:23.950342 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1a2b45d3-3782-4985-9c56-24a50a83a083 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "DELETE /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 204 len: 0 microversion: 2.1 time: 0.151780 Oct 15 22:15:23.951349 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2213/6667] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:15:23 2022] DELETE /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:15:23.966398 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:23.967154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:23.974489 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:23.974980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:23.975416 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:23.976287 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:23.976746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:23.977197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:24.116045 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:24.124075 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6154227b-0641-4b35-8c6d-0f76223cdb88 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.167184 Oct 15 22:15:24.124651 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4455/6668] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:23 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:25.150154 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:25.150871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:25.157921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:25.158425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:25.159021 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:25.160235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:25.160709 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:25.161204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:25.381657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:25.389541 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b356b267-0fa7-401a-9b6c-67ee960f90aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1680 microversion: 2.1 time: 0.243649 Oct 15 22:15:25.390126 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2214/6669] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:25 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1680 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:26.014970 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2fa31110-2726-4ddb-b698-5c43f339d54d req-2d9d97a2-9769-4ea9-a0e3-e7b1f5059edd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "286509c1-d419-46f5-bf2a-5c7ee6a576f7", "name": "network-vif-deleted", "tag": "c5e841c8-f6a2-43f7-8f1d-060da684136b"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:26.027048 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2fa31110-2726-4ddb-b698-5c43f339d54d req-2d9d97a2-9769-4ea9-a0e3-e7b1f5059edd service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:26.027530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2fa31110-2726-4ddb-b698-5c43f339d54d req-2d9d97a2-9769-4ea9-a0e3-e7b1f5059edd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:26.027960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2fa31110-2726-4ddb-b698-5c43f339d54d req-2d9d97a2-9769-4ea9-a0e3-e7b1f5059edd service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:26.039480 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2fa31110-2726-4ddb-b698-5c43f339d54d req-2d9d97a2-9769-4ea9-a0e3-e7b1f5059edd service nova] Creating event network-vif-deleted:c5e841c8-f6a2-43f7-8f1d-060da684136b for instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 on nch2-master-1 Oct 15 22:15:26.044577 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2fa31110-2726-4ddb-b698-5c43f339d54d req-2d9d97a2-9769-4ea9-a0e3-e7b1f5059edd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033687 Oct 15 22:15:26.045195 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4456/6670] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:15:26.415469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:26.416748 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:26.424054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:26.424597 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:26.425172 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:26.425852 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:26.426333 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:26.427130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:26.463045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:26.463547 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:26.463997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:26.464463 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lazy-loading 'fault' on Instance uuid 286509c1-d419-46f5-bf2a-5c7ee6a576f7 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:15:26.507904 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:26.515914 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-04a4e13c-8cfe-41df-b5b3-ed7aa473ad6a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 200 len: 1336 microversion: 2.1 time: 0.105599 Oct 15 22:15:26.516707 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2215/6671] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:26 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 1336 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:27.542537 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:27.543241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 286509c1-d419-46f5-bf2a-5c7ee6a576f7] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:27.550150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:27.550688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:27.551126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:27.551822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:27.552282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:27.552711 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:27.568936 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] HTTP exception thrown: Instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 could not be found. Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Request method failure captured: Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 HTTP/1.1 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSy-_Bv_3ywF8JbB4Dv--7umTQaAFOjzfAH2Y5y-_j64dVidkWM6mBS8D_ng6pm_QTTavqc_zVMMbzjQuq42Pnw93RwDNvTq3sYacWqQIPYBdi_rEbuRyDIQVQUOxNhCDIWfJC_QbT9ATgZnIPzxRgt9nUEKPO2LrfomIR7lVovVi02g Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 5ccb8936213a46b4804d7bbb43d4042b Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestShelveInstance-1131985420 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestShelveInstance-1131985420 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 5ccb8936213a46b4804d7bbb43d4042b Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestShelveInstance-1131985420 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestShelveInstance-1131985420-project Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6ddf65a9073b4abba4c8d09fed8a1ea6 Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestShelveInstance-1131985420-project Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 could not be found. Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '286509c1-d419-46f5-bf2a-5c7ee6a576f7'} Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 could not be found. Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 could not be found. Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:15:27.570598 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 could not be found. Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Returning 404 to user: Instance 286509c1-d419-46f5-bf2a-5c7ee6a576f7 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-191b2729-386e-4ca5-b37e-3cebfb669b9f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7" status: 404 len: 111 microversion: 2.1 time: 0.035090 Oct 15 22:15:27.579800 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4457/6672] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:27 2022] GET /compute/v2.1/servers/286509c1-d419-46f5-bf2a-5c7ee6a576f7 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:15:27.731676 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-7ec46a7c-049a-4aaa-82dd-6591ddd94a66 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '.version_select at 0x7fe74eca0700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:27.757876 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-7ec46a7c-049a-4aaa-82dd-6591ddd94a66 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-531387990" status: 202 len: 0 microversion: 2.1 time: 0.029553 Oct 15 22:15:27.769088 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2216/6673] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 22:15:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-531387990 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:15:27.778652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-3ad37a30-cce9-418f-b6d9-a75c702f6956 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'create', calling method: .version_select at 0x7fe74dfda310>, body: {"keypair": {"name": "tempest-TestShelveInstance-760352279", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEY8oE7a3Mvm41hES9l60AjGv4+9DtmjIsxlWzYQaejTtor9B/9Rc7k/b1SL3GyWifnDF4vp9KkHrHT4/Ojq+HNV2S6NHs1+umEgA8Feul5dxEyKV5bC/ZgzoUJxkBObKA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:27.795282 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3ad37a30-cce9-418f-b6d9-a75c702f6956 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:15:27.800947 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-3ad37a30-cce9-418f-b6d9-a75c702f6956 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for user 6ddf65a9073b4abba4c8d09fed8a1ea6 and project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:15:27.828195 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-3ad37a30-cce9-418f-b6d9-a75c702f6956 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.053389 Oct 15 22:15:27.828984 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4458/6674] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:15:27 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:15:30.272349 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-431065267", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-760352279", "security_groups": [{"name": "tempest-secgroup-smoke-1228815217"}], "networks": [{"uuid": "c052605f-592c-40e2-90fb-3727407e1ca5"}], "block_device_mapping_v2": [{"uuid": "bfd50710-94da-4f64-98e7-9d785aed02cf", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:30.387776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] validate_networks() for [('c052605f-592c-40e2-90fb-3727407e1ca5', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:15:30.798087 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:15:30.798535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:15:30.799055 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:15:30.799532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:15:30.800206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:15:30.809129 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:30.809657 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:30.810108 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:30.810850 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:30.811321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:30.811762 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:30.831198 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:15:30.836895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Getting quotas for user 6ddf65a9073b4abba4c8d09fed8a1ea6 and project 5ccb8936213a46b4804d7bbb43d4042b. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:15:30.894571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:15:30.896737 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='bfd50710-94da-4f64-98e7-9d785aed02cf',volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:15:31.024821 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-38f52041-db78-4b9f-b09e-04831fabd536 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.757139 Oct 15 22:15:31.025502 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2217/6675] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:15:30 2022] POST /compute/v2.1/servers => generated 406 bytes in 759 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:15:31.038436 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-ea9d8d90-5362-4ed4-9d60-a452147875b4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:31.039082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-ea9d8d90-5362-4ed4-9d60-a452147875b4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:31.097552 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-ea9d8d90-5362-4ed4-9d60-a452147875b4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:31.104731 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-ea9d8d90-5362-4ed4-9d60-a452147875b4 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1087 microversion: 2.1 time: 0.071881 Oct 15 22:15:31.105342 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4459/6676] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:31 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1087 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:32.128656 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:32.129440 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:32.136569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:32.137180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:32.137631 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:32.138342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:32.138811 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:32.139248 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:32.215611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:32.224641 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b748fb9e-8d22-4417-aace-edb9b909cc16 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1203 microversion: 2.1 time: 0.100780 Oct 15 22:15:32.237732 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2218/6677] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:32 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1203 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:33.248621 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:33.249308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:33.257294 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:33.257868 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:33.258383 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:33.259166 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:33.259699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:33.260202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:33.399493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:33.407973 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f91ec058-f698-49a9-ac31-cdcca14d91f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1271 microversion: 2.1 time: 0.163860 Oct 15 22:15:33.408570 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4460/6678] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:33 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1271 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:33.598966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-72afb449-1113-4b6d-b8d8-103a75f282cd req-2a40733c-2107-49b8-9363-946db703b5ca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:33.609406 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72afb449-1113-4b6d-b8d8-103a75f282cd req-2a40733c-2107-49b8-9363-946db703b5ca service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:33.609994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72afb449-1113-4b6d-b8d8-103a75f282cd req-2a40733c-2107-49b8-9363-946db703b5ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:33.610510 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-72afb449-1113-4b6d-b8d8-103a75f282cd req-2a40733c-2107-49b8-9363-946db703b5ca service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:33.623080 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-72afb449-1113-4b6d-b8d8-103a75f282cd req-2a40733c-2107-49b8-9363-946db703b5ca service nova] Creating event network-changed:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch1-master-1 Oct 15 22:15:33.629656 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-72afb449-1113-4b6d-b8d8-103a75f282cd req-2a40733c-2107-49b8-9363-946db703b5ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095489 Oct 15 22:15:33.630385 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2219/6679] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:15:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:15:34.431232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:34.431849 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:34.438943 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:34.439429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:34.440360 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:34.441114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:34.441574 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:34.442001 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:34.573989 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:34.582947 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-47e97d42-72f8-4086-afe9-9825d1af1810 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1259 microversion: 2.1 time: 0.155537 Oct 15 22:15:34.583533 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4461/6680] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:34 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1259 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:35.608155 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:35.608880 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:35.615682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:35.616184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:35.616641 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:35.617413 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:35.617889 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:35.618335 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:35.751421 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:35.760235 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-21418895-8dbd-4a20-b56e-e7a6da5f9b8a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1259 microversion: 2.1 time: 0.157780 Oct 15 22:15:35.760808 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2220/6681] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:35 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:36.784044 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:36.784796 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:36.791885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:36.792374 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:36.792812 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:36.793545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:36.794011 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:36.794446 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:36.930624 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:36.938302 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-7e4caa49-f822-4d49-bf88-f65abf751eb2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1259 microversion: 2.1 time: 0.158315 Oct 15 22:15:36.938876 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4462/6682] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:36 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1259 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:37.962999 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:37.963610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:37.970666 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:37.971726 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:37.972183 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:37.972895 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:37.973391 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:37.973832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:38.113911 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:38.121983 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69c56a5d-6d8d-40bf-a0f4-47936911a2aa tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1259 microversion: 2.1 time: 0.163246 Oct 15 22:15:38.122568 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2221/6683] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:37 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1259 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:39.146082 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:39.146708 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:39.153717 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:39.154206 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:39.154653 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:39.155356 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:39.155823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:39.156258 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:39.291156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:39.298920 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a338265-bf0a-481c-bab7-2a49e7532390 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1259 microversion: 2.1 time: 0.157252 Oct 15 22:15:39.299506 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4463/6684] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:39 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1259 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:40.125910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-d5071cf7-7ecd-49c5-94a4-33d1879e8ce7 req-5d49b2f8-216f-4885-991e-6a89103a524d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "name": "network-vif-plugged", "status": "completed", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:40.137770 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d5071cf7-7ecd-49c5-94a4-33d1879e8ce7 req-5d49b2f8-216f-4885-991e-6a89103a524d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:40.138377 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d5071cf7-7ecd-49c5-94a4-33d1879e8ce7 req-5d49b2f8-216f-4885-991e-6a89103a524d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:40.138695 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-d5071cf7-7ecd-49c5-94a4-33d1879e8ce7 req-5d49b2f8-216f-4885-991e-6a89103a524d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:40.150370 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-d5071cf7-7ecd-49c5-94a4-33d1879e8ce7 req-5d49b2f8-216f-4885-991e-6a89103a524d service nova] Creating event network-vif-plugged:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch1-master-1 Oct 15 22:15:40.155578 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-d5071cf7-7ecd-49c5-94a4-33d1879e8ce7 req-5d49b2f8-216f-4885-991e-6a89103a524d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085700 Oct 15 22:15:40.156209 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2222/6685] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:15:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:15:40.321973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:40.322590 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:40.329437 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:40.329919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:40.330964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:40.331654 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:40.332117 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:40.332542 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:40.464305 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:40.472044 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-273e644c-653e-499c-b6b7-5dd29853b923 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1259 microversion: 2.1 time: 0.153990 Oct 15 22:15:40.472619 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4464/6686] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:40 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1259 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:41.494760 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:41.495483 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:41.497480 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:41.498164 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:41.507310 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:41.507813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:41.508269 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:41.508981 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:41.509491 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:41.509936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:41.640000 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:41.648449 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-2d9cc127-bfc7-467c-bf61-eaf2de32d64f tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1443 microversion: 2.1 time: 0.157975 Oct 15 22:15:41.649026 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2223/6687] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:41 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1443 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:41.671478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:15:41.672093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:15:41.674134 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:15:41.680196 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:41.680680 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:41.681175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:41.681875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:41.682332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:41.682758 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:42.137221 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:15:42.144999 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c57ffa5f-dd8c-422f-b0bf-4ff0eea47e7a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1443 microversion: 2.1 time: 0.477339 Oct 15 22:15:42.145603 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4465/6688] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:15:41 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1443 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:15:44.072913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-f1293dac-88a2-4994-932f-7990631b593b req-0f51361f-043d-4a53-8bd8-272d70eb6519 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:15:44.083950 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f1293dac-88a2-4994-932f-7990631b593b req-0f51361f-043d-4a53-8bd8-272d70eb6519 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:15:44.084441 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f1293dac-88a2-4994-932f-7990631b593b req-0f51361f-043d-4a53-8bd8-272d70eb6519 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:15:44.084886 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f1293dac-88a2-4994-932f-7990631b593b req-0f51361f-043d-4a53-8bd8-272d70eb6519 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:15:44.096092 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-f1293dac-88a2-4994-932f-7990631b593b req-0f51361f-043d-4a53-8bd8-272d70eb6519 service nova] Creating event network-changed:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch1-master-1 Oct 15 22:15:44.101179 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-f1293dac-88a2-4994-932f-7990631b593b req-0f51361f-043d-4a53-8bd8-272d70eb6519 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031904 Oct 15 22:15:44.102042 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2224/6689] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:15:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:17:49.113960 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:17:49.114768 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=48 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:17:49.115918 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:17:49.116075 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:17:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318/action (ip 10.222.0.16) !!! Oct 15 22:17:49.121450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:17:49.121990 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:49.134592 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:49.135087 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:49.135532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:49.136244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:49.136709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:49.137427 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:49.161191 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.objects.instance [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lazy-loading 'flavor' on Instance uuid 7f35fe9b-8402-400f-b4fc-24fba08a6318 {{(pid=138688) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:17:49.233675 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-05e5df08-67ac-48bd-aa42-d3e845d6d2a7 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318/action" status: 202 len: 0 microversion: 2.1 time: 0.122262 Oct 15 22:17:49.234271 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4466/6690] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:17:49 2022] POST /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318/action => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:17:49.241133 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] AMQP server on 10.222.0.16:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Oct 15 22:17:49.242754 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] AMQP server on 10.222.0.16:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Oct 15 22:17:49.243610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:17:49.244454 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:17:49.250516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:49.251342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:49.259910 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:49.260416 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:49.260870 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:49.261620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:49.262097 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:49.262548 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:49.400392 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:49.409092 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-69b7e8ba-6230-4513-b0e4-d25fec92d2b6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.169535 Oct 15 22:17:49.410249 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2225/6691] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:49 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:50.434955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:50.435663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:50.442114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:50.442670 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:50.443147 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:50.443838 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:50.444304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:50.444737 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:50.581355 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:50.589206 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-36ef8b6c-f390-4207-860d-469175343004 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.159496 Oct 15 22:17:50.589784 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4467/6692] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:50 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:51.614482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:51.615039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:51.625571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:51.626071 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:51.626519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:51.628342 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:51.628833 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:51.629304 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:51.633298 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 48510. Oct 15 22:17:51.634026 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 48512. Oct 15 22:17:51.764859 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:51.772869 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cd13af0d-b238-4841-af6e-aee227f131a0 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.163296 Oct 15 22:17:51.773482 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2226/6693] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:51 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:52.796566 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:52.797202 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:52.804787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:52.805306 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:52.805740 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:52.806435 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:52.806891 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:52.807317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:52.941157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:52.950524 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a879b720-0355-45b2-82d3-315794fc0f80 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.158154 Oct 15 22:17:52.951109 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4468/6694] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:52 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:53.975745 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:53.976302 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:53.983145 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:53.983642 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:53.984091 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:53.984792 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:53.985286 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:53.985727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:54.119146 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:54.126881 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ef47a3e5-7078-4209-bf03-ac5127fc66a6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.155199 Oct 15 22:17:54.127446 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2227/6695] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:53 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:55.150579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:55.151219 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:55.158167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:55.158692 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:55.159144 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:55.159854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:55.160319 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:55.161308 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:55.342673 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:55.350844 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e7b6b6ab-48c8-45e4-8220-f1779f0b8669 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.204106 Oct 15 22:17:55.351552 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4469/6696] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:55 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:17:56.375723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:56.376271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:56.383781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:56.384278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:56.384724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:56.385460 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:56.385930 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:56.386368 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:56.518224 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:56.526768 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-78fb527a-9ead-4853-89e6-1f19731c6252 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.155039 Oct 15 22:17:56.527351 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2228/6697] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:56 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:57.550854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:57.551536 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:57.557952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:57.558441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:57.558879 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:57.559573 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:57.560033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:57.560460 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:57.689656 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:57.697665 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8fa745f7-b9af-43ea-9c65-427ad53e90d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.150815 Oct 15 22:17:57.698238 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4470/6698] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:57 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:58.721204 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:58.721776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:58.729271 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:58.729776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:58.730232 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:58.730952 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:58.731430 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:58.731875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:58.872961 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:17:58.880957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-84f471f8-4014-4486-9043-8bed859c9a2d tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.163753 Oct 15 22:17:58.881553 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2229/6699] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:58 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:17:59.904881 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:17:59.905558 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:17:59.912555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:59.913069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:59.913522 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:17:59.914293 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:17:59.914760 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:17:59.915190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:00.049477 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:00.058174 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c38981f3-d268-4b9a-9b00-2e04415d71e2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.157319 Oct 15 22:18:00.058752 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4471/6700] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:17:59 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:01.082431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:01.083192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:01.090157 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:01.090652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:01.091101 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:01.091805 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:01.092272 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:01.092708 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:01.231538 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:01.240381 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-cb72d1b3-a202-486c-95fd-4a97a5db6f61 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.162502 Oct 15 22:18:01.240953 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2230/6701] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:01 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:02.264387 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:02.265002 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:02.272014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:02.272500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:02.272944 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:02.273667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:02.274128 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:02.274560 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:02.403569 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:02.412081 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-70db7bb7-2710-48e5-bdad-4b134fb7059a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1509 microversion: 2.1 time: 0.151611 Oct 15 22:18:02.412689 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4472/6702] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:02 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1509 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:02.435575 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:18:02.436201 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:02.444431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:02.444943 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:02.445439 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:02.446168 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:02.446652 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:02.447103 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:02.479084 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=138689) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 15 22:18:02.543985 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [7545fcb5-d814-488b-b74f-e0013db97cfc] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:18:03.551808 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [7545fcb5-d814-488b-b74f-e0013db97cfc] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 48542. Oct 15 22:18:03.555657 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5793ce99-ae16-4c98-84d1-f68dea6b1538 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "POST /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318/action" status: 202 len: 0 microversion: 2.1 time: 1.124158 Oct 15 22:18:03.556278 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2231/6703] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:18:02 2022] POST /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318/action => generated 0 bytes in 1125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:18:03.564304 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:03.564916 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:03.572384 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:03.572875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:03.573342 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:03.574041 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:03.574501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:03.574933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:03.756078 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:03.764666 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-5ce9af51-129d-415a-97f7-3750ee7a1259 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1517 microversion: 2.1 time: 0.204189 Oct 15 22:18:03.765271 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4473/6704] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:03 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1517 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:18:04.794823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-76b308c1-2c57-4b67-b86b-6348ea31cf61 req-8e4faedd-4a18-47ff-8d9e-161fbad19eab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "name": "network-vif-unplugged", "status": "completed", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:18:04.801238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:04.802336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:04.815429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-76b308c1-2c57-4b67-b86b-6348ea31cf61 req-8e4faedd-4a18-47ff-8d9e-161fbad19eab service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:04.815921 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-76b308c1-2c57-4b67-b86b-6348ea31cf61 req-8e4faedd-4a18-47ff-8d9e-161fbad19eab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:04.816897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-76b308c1-2c57-4b67-b86b-6348ea31cf61 req-8e4faedd-4a18-47ff-8d9e-161fbad19eab service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:04.820926 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:04.821498 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:04.821957 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:04.822689 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:04.823167 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:04.823617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:04.852950 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.compute.server_external_events [req-76b308c1-2c57-4b67-b86b-6348ea31cf61 req-8e4faedd-4a18-47ff-8d9e-161fbad19eab service nova] Unable to find a host for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318. Dropping event network-vif-unplugged {{(pid=138688) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 15 22:18:04.853899 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-76b308c1-2c57-4b67-b86b-6348ea31cf61 req-8e4faedd-4a18-47ff-8d9e-161fbad19eab service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.063098 Oct 15 22:18:04.854517 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4474/6705] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:18:04 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 64 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 15 22:18:04.978516 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:04.986717 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-817c9dc0-e8a5-44e8-bc6b-b3cf5ebde799 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1519 microversion: 2.1 time: 0.202840 Oct 15 22:18:04.987502 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2232/6706] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:04 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1519 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:06.011216 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:06.011833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:06.019093 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:06.019856 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:06.020292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:06.021017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:06.021486 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:06.021913 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:06.279205 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:06.287532 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-cb439a3f-4962-42d6-aa13-8fa4f459b6d6 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.280148 Oct 15 22:18:06.288130 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4475/6707] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:06 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:06.489488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-f7be2674-dbbe-4a26-93b0-2f90774f2026 req-f1ba3657-4c77-4a06-a78d-977c51e86555 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:18:06.499933 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f7be2674-dbbe-4a26-93b0-2f90774f2026 req-f1ba3657-4c77-4a06-a78d-977c51e86555 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:06.500431 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f7be2674-dbbe-4a26-93b0-2f90774f2026 req-f1ba3657-4c77-4a06-a78d-977c51e86555 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:06.500969 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f7be2674-dbbe-4a26-93b0-2f90774f2026 req-f1ba3657-4c77-4a06-a78d-977c51e86555 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:06.513372 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-f7be2674-dbbe-4a26-93b0-2f90774f2026 req-f1ba3657-4c77-4a06-a78d-977c51e86555 service nova] Creating event network-changed:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch2-master-1 Oct 15 22:18:06.518646 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-f7be2674-dbbe-4a26-93b0-2f90774f2026 req-f1ba3657-4c77-4a06-a78d-977c51e86555 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033339 Oct 15 22:18:06.519537 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2233/6708] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:18:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:18:07.312506 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:07.313058 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:07.320826 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:07.321347 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:07.321787 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:07.322482 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:07.322940 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:07.323375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:07.457450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:07.465252 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-eef0172f-b4f1-49ac-bbad-7b924ffd723c tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.156734 Oct 15 22:18:07.465827 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4476/6709] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:07 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:08.490012 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:08.490858 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:08.497297 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:08.497824 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:08.498280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:08.498994 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:08.499464 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:08.499901 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:08.640051 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:08.648483 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-114202d6-2c3e-41f6-899c-3ef511a63983 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.164376 Oct 15 22:18:08.649078 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2234/6710] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:08 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:09.672751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:09.673396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:09.681025 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:09.681539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:09.681980 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:09.682686 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:09.683149 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:09.683579 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:09.812390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:09.820427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a234175e-35a4-4b2d-b067-321a02511a2a tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.151608 Oct 15 22:18:09.821001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4477/6711] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:09 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:18:10.843955 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:10.844588 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:10.851653 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:10.852158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:10.852611 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:10.853364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:10.853841 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:10.854289 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:11.031817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:11.039765 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-bb00a966-5329-4668-b0dd-0d68b4604180 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.199873 Oct 15 22:18:11.040326 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2235/6712] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:10 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:12.063788 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:12.064403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:12.071501 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:12.071986 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:12.072424 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:12.073157 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:12.073617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:12.074040 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:12.220445 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:12.229119 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-e2c31dac-ec2d-450a-ba9a-ec1f2f080057 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.169164 Oct 15 22:18:12.229726 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4478/6713] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:12 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:13.253152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:13.253966 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:13.261569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:13.262171 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:13.262777 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:13.263697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:13.264238 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:13.264769 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:13.396476 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:13.404267 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6d7a8ce7-af2c-463b-85fa-1966f9eb9edf tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.155174 Oct 15 22:18:13.404838 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2236/6714] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:13 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:13.637102 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b4b36368-3b01-43f3-9234-8e8ac96e0256 req-382466ce-137f-4a10-9f6e-bf6deb968f0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "name": "network-vif-plugged", "status": "completed", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:18:13.647843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b4b36368-3b01-43f3-9234-8e8ac96e0256 req-382466ce-137f-4a10-9f6e-bf6deb968f0f service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:13.648327 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b4b36368-3b01-43f3-9234-8e8ac96e0256 req-382466ce-137f-4a10-9f6e-bf6deb968f0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:13.648808 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b4b36368-3b01-43f3-9234-8e8ac96e0256 req-382466ce-137f-4a10-9f6e-bf6deb968f0f service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:13.660225 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b4b36368-3b01-43f3-9234-8e8ac96e0256 req-382466ce-137f-4a10-9f6e-bf6deb968f0f service nova] Creating event network-vif-plugged:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch2-master-1 Oct 15 22:18:13.665283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b4b36368-3b01-43f3-9234-8e8ac96e0256 req-382466ce-137f-4a10-9f6e-bf6deb968f0f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031914 Oct 15 22:18:13.665875 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4479/6715] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:18:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:18:14.429562 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:14.430189 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:14.437173 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:14.437668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:14.438113 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:14.438813 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:14.439278 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:14.439714 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:14.568102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:14.575996 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6cf1bb05-3cfe-4b2d-b9a5-e78d6cdc4360 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1575 microversion: 2.1 time: 0.151190 Oct 15 22:18:14.576566 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2237/6716] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:14 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1575 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:18:15.601200 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:18:15.601862 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:18:15.608187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:15.608671 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:15.609339 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:15.610116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:18:15.611539 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:18:15.611971 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:18:15.747763 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:18:15.756055 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-104f72a1-7a84-4665-9416-af4f43f745ed tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1562 microversion: 2.1 time: 0.160015 Oct 15 22:18:15.756623 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4480/6717] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:18:15 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1562 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:19:36.842462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:36.846833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=30 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:36.847870 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:36.848694 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:19:36.848847 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:19:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 15 22:19:36.849723 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=78,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:36.851047 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:19:36.851193 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 22:19:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 (ip 10.222.0.16) !!! Oct 15 22:19:36.855688 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-62d947c0-429d-4c8a-aac9-5a63c8873912 req-919cfd1c-eda7-4963-9871-cefb094d39eb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:19:36.866162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:19:36.866668 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:19:36.884549 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-62d947c0-429d-4c8a-aac9-5a63c8873912 req-919cfd1c-eda7-4963-9871-cefb094d39eb service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:36.885079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-62d947c0-429d-4c8a-aac9-5a63c8873912 req-919cfd1c-eda7-4963-9871-cefb094d39eb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:36.885490 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-62d947c0-429d-4c8a-aac9-5a63c8873912 req-919cfd1c-eda7-4963-9871-cefb094d39eb service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:36.892008 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:36.892509 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:36.892958 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:36.893715 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:36.894180 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:36.894617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:36.907149 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-62d947c0-429d-4c8a-aac9-5a63c8873912 req-919cfd1c-eda7-4963-9871-cefb094d39eb service nova] Creating event network-changed:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch2-master-1 Oct 15 22:19:36.912806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-62d947c0-429d-4c8a-aac9-5a63c8873912 req-919cfd1c-eda7-4963-9871-cefb094d39eb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068453 Oct 15 22:19:36.913552 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4481/6718] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:19:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:19:36.923359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:19:37.021309 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5e8aa83c-7cb5-438d-8984-96aed4637901 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 204 len: 0 microversion: 2.1 time: 0.180778 Oct 15 22:19:37.022268 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2238/6719] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:19:36 2022] DELETE /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:19:37.035086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:19:37.035814 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:19:37.043746 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:37.044250 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:37.044699 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:37.045444 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:37.045912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:37.046351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:37.188299 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:19:37.196734 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-30eea320-e096-4019-8bd2-6886e313c9e8 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.168529 Oct 15 22:19:37.201615 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4482/6720] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:19:37 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:19:38.221117 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:19:38.221872 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:19:38.229503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:38.230014 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:38.230469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:38.231190 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:38.231663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:38.232200 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:38.478746 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:19:38.486741 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-9c357c9d-c69b-4d89-8a79-084e1e3533de tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1568 microversion: 2.1 time: 0.269985 Oct 15 22:19:38.487318 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2239/6721] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:19:38 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1568 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:19:39.136578 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-56d27d27-da50-4852-b89e-4e71719b16d1 req-a80ab304-ed43-4c95-b53c-55da55adf786 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f35fe9b-8402-400f-b4fc-24fba08a6318", "name": "network-vif-deleted", "tag": "73dd3949-465d-42a8-b4c7-fdbdf02ea55d"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:19:39.147381 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-56d27d27-da50-4852-b89e-4e71719b16d1 req-a80ab304-ed43-4c95-b53c-55da55adf786 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:39.147882 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-56d27d27-da50-4852-b89e-4e71719b16d1 req-a80ab304-ed43-4c95-b53c-55da55adf786 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:39.148298 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-56d27d27-da50-4852-b89e-4e71719b16d1 req-a80ab304-ed43-4c95-b53c-55da55adf786 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:39.159630 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-56d27d27-da50-4852-b89e-4e71719b16d1 req-a80ab304-ed43-4c95-b53c-55da55adf786 service nova] Creating event network-vif-deleted:73dd3949-465d-42a8-b4c7-fdbdf02ea55d for instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 on nch2-master-1 Oct 15 22:19:39.164784 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-56d27d27-da50-4852-b89e-4e71719b16d1 req-a80ab304-ed43-4c95-b53c-55da55adf786 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032410 Oct 15 22:19:39.166306 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4483/6722] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:19:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:19:39.512922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:19:39.513801 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:19:39.522328 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:39.522876 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:39.523364 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:39.524099 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:39.525528 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:39.526152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:39.568837 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:39.569433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:39.569978 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:39.570459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lazy-loading 'fault' on Instance uuid 7f35fe9b-8402-400f-b4fc-24fba08a6318 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:19:39.627535 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6ddf65a9073b4abba4c8d09fed8a1ea6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ccb8936213a46b4804d7bbb43d4042b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:19:39.636265 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-02daa406-8dc5-40d4-8a16-242dc13596c1 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 200 len: 1223 microversion: 2.1 time: 0.128259 Oct 15 22:19:39.636895 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2240/6723] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:19:39 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 1223 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:19:40.661908 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:19:40.662622 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] [instance: 7f35fe9b-8402-400f-b4fc-24fba08a6318] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:19:40.669876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:40.670375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:40.670822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:40.671530 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:19:40.671997 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:19:40.672433 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:19:40.686648 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] HTTP exception thrown: Instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 could not be found. Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Request method failure captured: Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 HTTP/1.1 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSy-_Bv_3ywF8JbB4Dv--7umTQaAFOjzfAH2Y5y-_j64dVidkWM6mBS8D_ng6pm_QTTavqc_zVMMbzjQuq42Pnw93RwDNvTq3sYacWqQIPYBdi_rEbuRyDIQVQUOxNhCDIWfJC_QbT9ATgZnIPzxRgt9nUEKPO2LrfomIR7lVovVi02g Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: 5ccb8936213a46b4804d7bbb43d4042b Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestShelveInstance-1131985420 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5ccb8936213a46b4804d7bbb43d4042b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestShelveInstance-1131985420 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: 5ccb8936213a46b4804d7bbb43d4042b Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestShelveInstance-1131985420 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestShelveInstance-1131985420-project Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 6ddf65a9073b4abba4c8d09fed8a1ea6 Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestShelveInstance-1131985420-project Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 could not be found. Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': '7f35fe9b-8402-400f-b4fc-24fba08a6318'} Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 could not be found. Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 could not be found. Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:19:40.688271 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 could not be found. Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Returning 404 to user: Instance 7f35fe9b-8402-400f-b4fc-24fba08a6318 could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-27b52928-75ad-4f77-b31d-d7381e42f9f2 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318" status: 404 len: 111 microversion: 2.1 time: 0.032961 Oct 15 22:19:40.691664 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4484/6724] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:19:40 2022] GET /compute/v2.1/servers/7f35fe9b-8402-400f-b4fc-24fba08a6318 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:19:40.854921 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0c57aabd-c413-49aa-b77c-683cc1066052 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] Calling method '.version_select at 0x7fe74ec36700>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:19:40.883164 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0c57aabd-c413-49aa-b77c-683cc1066052 tempest-TestShelveInstance-1131985420 tempest-TestShelveInstance-1131985420-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-760352279" status: 202 len: 0 microversion: 2.1 time: 0.033097 Oct 15 22:19:40.883729 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2241/6725] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Oct 15 22:19:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-760352279 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 15 22:19:58.268890 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=8 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:58.269964 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=12 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:58.448150 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25b495ed-c7ba-430f-90e1-72575e5ce7f0 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Action: 'create', calling method: .version_select at 0x7fe74fb3b5e0>, body: {"keypair": {"name": "tempest-TestSnapshotPattern-935573672", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBODyGJNXRzGp6mol606ufXIYpu9PasB8huSB715n+gYYsSDPX6emBhZ85Rf6FKDDvqB5MLFmDx5yZgyaZWwp6ZFdqPC8Z1vMfnLVv23TZkr73OnheeuxX8cNOrqG8gUlmA=="}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:19:58.463659 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-25b495ed-c7ba-430f-90e1-72575e5ce7f0 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Getting quotas for project f02d6a82dfc941c9a0f19bce198d76ab. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:19:58.469055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-25b495ed-c7ba-430f-90e1-72575e5ce7f0 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Getting quotas for user 4879d2bce837495baf7411ee27934ece and project f02d6a82dfc941c9a0f19bce198d76ab. Resources: {'key_pairs'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:19:58.499701 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25b495ed-c7ba-430f-90e1-72575e5ce7f0 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.236962 Oct 15 22:19:58.500316 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4485/6726] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Oct 15 22:19:58 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:19:59.563045 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestSnapshotPattern-server-603640024", "imageRef": "77903f56-0555-4c00-82ae-bbf42e11a2a9", "flavorRef": "452", "key_name": "tempest-TestSnapshotPattern-935573672", "security_groups": [{"name": "tempest-secgroup-smoke-1210505345"}], "networks": [{"uuid": "9cd3f7fa-8987-4e6a-b207-e6ea5b5fa568"}]}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:19:59.573919 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:59.574580 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:19:59.681443 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.network.neutron [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] validate_networks() for [('9cd3f7fa-8987-4e6a-b207-e6ea5b5fa568', None, None, None, None, None)] {{(pid=138689) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:20:00.096772 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Flavor limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:20:00.097388 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Image limits 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:20:00.097881 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Flavor pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:20:00.098345 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Image pref 0:0:0 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:20:00.098730 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.virt.hardware [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138689) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:20:00.108662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:00.109347 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:00.109684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:00.110447 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:00.111066 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:00.111519 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:00.130736 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Getting quotas for project f02d6a82dfc941c9a0f19bce198d76ab. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:20:00.139654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.quota [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Getting quotas for user 4879d2bce837495baf7411ee27934ece and project f02d6a82dfc941c9a0f19bce198d76ab. Resources: {'instances', 'ram', 'cores'} {{(pid=138689) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:20:00.154054 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Going to run 1 instances... {{(pid=138689) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:20:00.156503 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='77903f56-0555-4c00-82ae-bbf42e11a2a9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138689) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:20:00.212188 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e57911ff-1adc-4f38-aab3-017990e19d96 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.652556 Oct 15 22:20:00.217110 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2242/6727] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:19:59 2022] POST /compute/v2.1/servers => generated 406 bytes in 654 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:20:00.237907 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-25eecebe-bfc8-4be9-9982-4b2d612a8ba2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:00.238425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-25eecebe-bfc8-4be9-9982-4b2d612a8ba2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:00.332876 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-25eecebe-bfc8-4be9-9982-4b2d612a8ba2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:00.341516 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-25eecebe-bfc8-4be9-9982-4b2d612a8ba2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1247 microversion: 2.1 time: 0.121073 Oct 15 22:20:00.343236 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4486/6728] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:00 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1247 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:01.369336 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:01.370724 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:01.380518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:01.381049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:01.381513 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:01.382300 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:01.382785 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:01.383235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:01.458524 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:01.468030 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-a60055b7-793d-4337-bf5f-185d27e9c914 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1317 microversion: 2.1 time: 0.106788 Oct 15 22:20:01.469071 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2243/6729] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:01 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1317 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:02.492519 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:02.494487 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:02.502270 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:02.502906 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:02.503478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:02.504561 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:02.505292 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:02.506033 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:02.675241 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:02.684553 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-10793681-f6ba-4c58-9221-0e7f9c063a09 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.196173 Oct 15 22:20:02.685515 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4487/6730] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:02 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:02.785193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-5a987375-46b8-4d89-826a-e2cc7584dd58 req-3bad45aa-f70f-49ed-9239-1ab3fe911197 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:20:02.807355 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a987375-46b8-4d89-826a-e2cc7584dd58 req-3bad45aa-f70f-49ed-9239-1ab3fe911197 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:02.808131 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a987375-46b8-4d89-826a-e2cc7584dd58 req-3bad45aa-f70f-49ed-9239-1ab3fe911197 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:02.808924 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-5a987375-46b8-4d89-826a-e2cc7584dd58 req-3bad45aa-f70f-49ed-9239-1ab3fe911197 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:02.825300 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-5a987375-46b8-4d89-826a-e2cc7584dd58 req-3bad45aa-f70f-49ed-9239-1ab3fe911197 service nova] Creating event network-changed:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:20:02.834147 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-5a987375-46b8-4d89-826a-e2cc7584dd58 req-3bad45aa-f70f-49ed-9239-1ab3fe911197 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053433 Oct 15 22:20:02.835480 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2244/6731] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:20:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:20:03.709969 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:03.710854 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:03.718584 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:03.719089 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:03.719509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:03.720210 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:03.720667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:03.721124 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:03.854317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:03.862185 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6b134610-c8db-4e66-9645-7f650d643b48 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.156349 Oct 15 22:20:03.862788 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4488/6732] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:03 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:04.887649 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:04.888373 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:04.895782 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:04.896357 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:04.896972 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:04.897727 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:04.898206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:04.898651 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:05.049016 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:05.060108 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-195818ff-cff3-4621-bf08-a78954c9c326 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.176935 Oct 15 22:20:05.061013 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2245/6733] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:04 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:20:06.089819 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:06.090429 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:06.097269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:06.097751 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:06.098190 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:06.098885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:06.099343 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:06.099822 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:06.246032 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:06.254036 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-8f416bb7-741b-4bea-80b1-19574416180f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.168770 Oct 15 22:20:06.254625 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4489/6734] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:06 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:07.278488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:07.279162 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:07.286467 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:07.286976 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:07.287427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:07.288152 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:07.288621 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:07.289226 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:07.432571 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:07.440612 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-0fec3749-9fb4-4e3c-90b6-193a986d6241 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.166796 Oct 15 22:20:07.441220 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2246/6735] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:07 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:08.464252 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:08.464871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:08.471954 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:08.472442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:08.472889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:08.473617 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:08.474086 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:08.474514 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:08.611450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:08.619284 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b6f3675f-ed9a-409e-82ec-d0430f0ed1db tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.159419 Oct 15 22:20:08.619863 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4490/6736] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:08 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:09.327764 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-c4d18b50-4e39-43f0-b59b-d9f48599ee87 req-baa51c6c-b48e-4488-9c32-62291e9e01c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "name": "network-vif-plugged", "status": "completed", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:20:09.340861 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c4d18b50-4e39-43f0-b59b-d9f48599ee87 req-baa51c6c-b48e-4488-9c32-62291e9e01c3 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:09.341425 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c4d18b50-4e39-43f0-b59b-d9f48599ee87 req-baa51c6c-b48e-4488-9c32-62291e9e01c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:09.341875 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-c4d18b50-4e39-43f0-b59b-d9f48599ee87 req-baa51c6c-b48e-4488-9c32-62291e9e01c3 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:09.353384 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-c4d18b50-4e39-43f0-b59b-d9f48599ee87 req-baa51c6c-b48e-4488-9c32-62291e9e01c3 service nova] Creating event network-vif-plugged:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:20:09.358391 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-c4d18b50-4e39-43f0-b59b-d9f48599ee87 req-baa51c6c-b48e-4488-9c32-62291e9e01c3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035035 Oct 15 22:20:09.358980 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2247/6737] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:20:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:20:09.642553 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:09.643167 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:09.650485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:09.650967 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:09.651523 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:09.652227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:09.652691 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:09.653187 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:09.783060 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:09.791084 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b3132ed8-d1b6-4b45-be30-250a812d1873 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1373 microversion: 2.1 time: 0.152420 Oct 15 22:20:09.791660 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4491/6738] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:09 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1373 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:10.815832 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:10.816563 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:10.818639 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:20:10.819433 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=6 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:20:10.827450 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:10.827962 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:10.828427 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:10.829184 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:10.829663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:10.830122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:10.967126 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:10.975108 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8b707b7c-15e2-48df-a228-263f771df3cf tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1558 microversion: 2.1 time: 0.163864 Oct 15 22:20:10.975801 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2248/6739] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:10 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:20:10.998370 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:20:10.998983 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:20:11.000825 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:20:11.006893 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:11.007373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:11.007809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:11.008495 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:11.008952 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:11.009456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:11.143382 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:20:11.151786 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-b0bdfd41-5aca-4ad7-9661-3c9c90ea7f30 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1558 microversion: 2.1 time: 0.157380 Oct 15 22:20:11.152349 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4492/6740] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:20:10 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:20:13.157885 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-e81aadbe-8cc8-452d-a811-ba2d50767d85 req-15d24cee-a5df-4440-8e1c-2e87794b4453 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:20:13.168662 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e81aadbe-8cc8-452d-a811-ba2d50767d85 req-15d24cee-a5df-4440-8e1c-2e87794b4453 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:20:13.169477 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e81aadbe-8cc8-452d-a811-ba2d50767d85 req-15d24cee-a5df-4440-8e1c-2e87794b4453 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:20:13.169922 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-e81aadbe-8cc8-452d-a811-ba2d50767d85 req-15d24cee-a5df-4440-8e1c-2e87794b4453 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:20:13.181492 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-e81aadbe-8cc8-452d-a811-ba2d50767d85 req-15d24cee-a5df-4440-8e1c-2e87794b4453 service nova] Creating event network-changed:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:20:13.186508 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-e81aadbe-8cc8-452d-a811-ba2d50767d85 req-15d24cee-a5df-4440-8e1c-2e87794b4453 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032329 Oct 15 22:20:13.187090 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2249/6741] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:20:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:22:07.360212 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 22:22:07.361328 ncdv-master-1 devstack@n-api.service[138688]: ERROR oslo.messaging._drivers.impl_rabbit [-] [8a327d79-84a5-4eee-8d31-7df1f6cabc7a] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Oct 15 22:22:07.361871 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:07.362807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=36 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:07.363875 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:22:07.364031 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:22:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574/action (ip 10.222.0.16) !!! Oct 15 22:22:07.366157 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:22:07.366306 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:22:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574/action (ip 10.222.0.16) !!! Oct 15 22:22:07.371885 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestSnapshotPatternsnapshot-660183760"}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:07.374256 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:07.389776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:07.390269 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:07.390712 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:07.391425 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:07.391896 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:07.392332 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:07.594409 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-79da1a62-4358-4cdc-9a18-9309a6af6662 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "POST /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574/action" status: 202 len: 0 microversion: 2.1 time: 0.235809 Oct 15 22:22:07.595079 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4493/6742] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:22:07 2022] POST /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574/action => generated 0 bytes in 237 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 22:22:13.859465 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:22:13.860457 ncdv-master-1 devstack@n-api.service[138689]: ERROR oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 15 22:22:13.861178 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:13.862370 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=32 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:13.864362 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 15 22:22:13.864846 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 22:22:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 15 22:22:13.964719 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-229dac0e-bf57-4791-a500-665c50b83099 req-357e5f31-228d-490a-a1d9-4bce08e706ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "name": "network-vif-plugged", "status": "completed", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:13.975049 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-229dac0e-bf57-4791-a500-665c50b83099 req-357e5f31-228d-490a-a1d9-4bce08e706ea service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:13.975648 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-229dac0e-bf57-4791-a500-665c50b83099 req-357e5f31-228d-490a-a1d9-4bce08e706ea service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:13.976105 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-229dac0e-bf57-4791-a500-665c50b83099 req-357e5f31-228d-490a-a1d9-4bce08e706ea service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:13.988767 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-229dac0e-bf57-4791-a500-665c50b83099 req-357e5f31-228d-490a-a1d9-4bce08e706ea service nova] Creating event network-vif-plugged:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:22:13.994018 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-229dac0e-bf57-4791-a500-665c50b83099 req-357e5f31-228d-490a-a1d9-4bce08e706ea service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.136642 Oct 15 22:22:13.994631 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2250/6743] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:22:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:22:21.156548 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:21.157351 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:21.163889 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestSnapshotPattern-server-37999778", "imageRef": "4e9d1dde-d204-4b0d-964c-ebc03dd3dd66", "flavorRef": "452", "key_name": "tempest-TestSnapshotPattern-935573672", "security_groups": [{"name": "tempest-secgroup-smoke-1210505345"}], "networks": [{"uuid": "9cd3f7fa-8987-4e6a-b207-e6ea5b5fa568"}]}} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:21.193440 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [7bbd84c2-53a6-428b-b4f9-22b084b23373] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 49082. Oct 15 22:22:21.194247 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] [8a327d79-84a5-4eee-8d31-7df1f6cabc7a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 49084. Oct 15 22:22:21.285063 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.network.neutron [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] validate_networks() for [('9cd3f7fa-8987-4e6a-b207-e6ea5b5fa568', None, None, None, None, None)] {{(pid=138688) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 15 22:22:21.712511 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Flavor limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 15 22:22:21.712959 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Image limits 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 15 22:22:21.713470 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Flavor pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 15 22:22:21.713887 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Image pref 0:0:0 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 15 22:22:21.714302 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.virt.hardware [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138688) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 15 22:22:21.723912 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:21.724396 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:21.724835 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:21.725607 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:21.726069 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:21.726496 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:21.743065 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Getting quotas for project f02d6a82dfc941c9a0f19bce198d76ab. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 15 22:22:21.748517 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.quota [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Getting quotas for user 4879d2bce837495baf7411ee27934ece and project f02d6a82dfc941c9a0f19bce198d76ab. Resources: {'instances', 'ram', 'cores'} {{(pid=138688) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 15 22:22:21.760843 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Going to run 1 instances... {{(pid=138688) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 15 22:22:21.763897 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='4e9d1dde-d204-4b0d-964c-ebc03dd3dd66',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138688) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 15 22:22:21.812426 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-f3bd342e-c702-436f-980c-bc65f668e1d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.659149 Oct 15 22:22:21.813108 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4494/6744] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Oct 15 22:22:21 2022] POST /compute/v2.1/servers => generated 406 bytes in 661 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 15 22:22:21.826734 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8a8e367e-67de-41a1-a6f8-8974a1c67918 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:21.827462 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8a8e367e-67de-41a1-a6f8-8974a1c67918 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:21.855518 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [3c8d0cb1-713c-4aa0-b55f-0e28211cc67c] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 49100. Oct 15 22:22:21.856643 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] [daa1cfef-5cc8-4c99-bc95-45f89ca2211a] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 49102. Oct 15 22:22:21.890220 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8a8e367e-67de-41a1-a6f8-8974a1c67918 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:21.898182 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8a8e367e-67de-41a1-a6f8-8974a1c67918 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1246 microversion: 2.1 time: 0.077433 Oct 15 22:22:21.898745 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2251/6745] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:21 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1246 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:22.927452 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:22.928217 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:22.935762 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:22.936268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:22.936714 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:22.937466 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:22.937966 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:22.938405 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:23.014126 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:23.022880 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-68b215ab-954c-44dd-9eb7-e805c5ac2303 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1316 microversion: 2.1 time: 0.101601 Oct 15 22:22:23.023791 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4495/6746] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:22 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1316 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:22:23.927078 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-90f36fdf-6de6-47be-afd5-8a10b6df4e56 req-878aeed5-956f-4607-ac38-a5204a1756e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "name": "network-vif-unplugged", "status": "completed", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:23.947177 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-90f36fdf-6de6-47be-afd5-8a10b6df4e56 req-878aeed5-956f-4607-ac38-a5204a1756e5 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:23.950059 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-90f36fdf-6de6-47be-afd5-8a10b6df4e56 req-878aeed5-956f-4607-ac38-a5204a1756e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:23.950776 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-90f36fdf-6de6-47be-afd5-8a10b6df4e56 req-878aeed5-956f-4607-ac38-a5204a1756e5 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:23.974114 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-90f36fdf-6de6-47be-afd5-8a10b6df4e56 req-878aeed5-956f-4607-ac38-a5204a1756e5 service nova] Creating event network-vif-unplugged:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:22:23.992202 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-90f36fdf-6de6-47be-afd5-8a10b6df4e56 req-878aeed5-956f-4607-ac38-a5204a1756e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.068580 Oct 15 22:22:23.993431 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2252/6747] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:22:23 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:22:24.048335 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:24.049079 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:24.062759 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:24.064922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:24.065555 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:24.066526 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:24.070509 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:24.071137 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:24.257704 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:24.266133 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-a2a55111-0631-4aba-a5b4-8c2e02fc5ec2 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.222676 Oct 15 22:22:24.266718 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4496/6748] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:24 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:25.291892 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:25.293206 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:25.300243 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:25.301102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:25.301846 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:25.302959 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:25.303733 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:25.304459 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:25.463267 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:25.471944 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-abd4b2b3-897a-4351-b3e7-c350cb8b73fc tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.185053 Oct 15 22:22:25.472536 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2253/6749] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:25 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:26.495606 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:26.496483 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:26.503513 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:26.504000 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:26.504438 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:26.505169 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:26.505628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:26.506055 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:26.648705 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:26.657443 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-bbd26ca7-a873-4be7-8c68-0a7f67b0f5e8 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.165823 Oct 15 22:22:26.658041 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4497/6750] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:26 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:27.682761 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:27.684663 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:27.692417 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:27.692935 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:27.693418 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:27.694158 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:27.694638 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:27.695089 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:27.844466 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:27.852512 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-e547a8b0-f794-4b15-804b-58cb4e97e219 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.174320 Oct 15 22:22:27.853166 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2254/6751] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:27 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:28.876532 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:28.877334 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:28.884246 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:28.884749 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:28.885251 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:28.885962 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:28.886432 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:28.886901 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:29.062081 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:29.070487 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-dfb06894-d8eb-44d1-b508-0cf2c36e29d6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.197806 Oct 15 22:22:29.071059 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4498/6752] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:28 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:30.094249 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:30.094913 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:30.101831 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:30.102329 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:30.102780 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:30.103488 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:30.103953 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:30.104393 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:30.245587 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:30.255500 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-5779d21f-f666-4843-948b-3ce6cd59d23a tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.165207 Oct 15 22:22:30.256389 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2255/6753] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:30 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:31.279948 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:31.280683 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:31.287707 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:31.288197 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:31.288638 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:31.289373 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:31.289836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:31.290266 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:31.429799 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:31.437512 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-51aedbbf-09d6-4b93-956b-d35bf8a89106 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.162519 Oct 15 22:22:31.438103 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4499/6754] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:31 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:32.461758 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:32.462381 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:32.465215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell0:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:32.471193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:32.471686 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:32.472130 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:32.472830 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:32.473318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:32.473763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:32.609867 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:32.617605 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ebf0a18e-b97a-4a93-b05e-98f4631e4d21 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.159716 Oct 15 22:22:32.618187 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2256/6755] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:32 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:33.639924 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:33.640564 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:33.642478 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:33.643312 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell0:SELECT=6 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:22:33.649742 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:33.650227 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:33.650662 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:33.651354 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:33.651809 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:33.652232 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:33.788687 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:33.797562 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-09c925a0-450d-4e89-8783-e92d64655c17 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.161646 Oct 15 22:22:33.798134 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4500/6756] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:33 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:34.821268 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:34.821911 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:34.829310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:34.830534 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:34.830993 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:34.831709 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:34.832380 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:34.832609 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:34.844729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-0241ebce-b092-4c92-a488-5f0db06b0300 req-4a7c240f-ca60-4d56-a16c-e61dd494ac8b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1f05571-1f9e-4ea8-8af1-a53672d0beba", "tag": "414e20e5-e456-4dcb-82dd-8db120f98dac"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:34.854408 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0241ebce-b092-4c92-a488-5f0db06b0300 req-4a7c240f-ca60-4d56-a16c-e61dd494ac8b service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:34.854971 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0241ebce-b092-4c92-a488-5f0db06b0300 req-4a7c240f-ca60-4d56-a16c-e61dd494ac8b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:34.855340 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-0241ebce-b092-4c92-a488-5f0db06b0300 req-4a7c240f-ca60-4d56-a16c-e61dd494ac8b service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:34.867795 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-0241ebce-b092-4c92-a488-5f0db06b0300 req-4a7c240f-ca60-4d56-a16c-e61dd494ac8b service nova] Creating event network-changed:414e20e5-e456-4dcb-82dd-8db120f98dac for instance d1f05571-1f9e-4ea8-8af1-a53672d0beba on nch2-master-1 Oct 15 22:22:34.873319 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-0241ebce-b092-4c92-a488-5f0db06b0300 req-4a7c240f-ca60-4d56-a16c-e61dd494ac8b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.104082 Oct 15 22:22:34.874066 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2257/6757] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:22:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:22:35.072154 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:35.080790 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-1a92e35d-839f-4c8b-b86c-b3abf9ab8d0d tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.263316 Oct 15 22:22:35.081392 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4501/6758] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:34 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:36.107085 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:36.107936 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:36.114143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:36.114644 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:36.115104 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:36.115817 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:36.116292 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:36.116735 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:36.259632 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:36.267795 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d8e6d3f1-b2db-4e30-a6f4-4fb943b08e37 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.166614 Oct 15 22:22:36.268372 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2258/6759] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:36 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:37.291925 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:37.292535 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:37.300403 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:37.301192 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:37.301663 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:37.302457 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:37.302973 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:37.303456 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:37.528467 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:37.536319 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-234039a8-479a-4660-b1eb-4b1f767d72a4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.248336 Oct 15 22:22:37.536897 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4502/6760] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:37 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:38.561305 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:38.561853 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:38.568693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:38.569241 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:38.569698 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:38.570407 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:38.570884 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:38.571321 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:38.704681 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:38.712445 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ce394a53-077d-4a23-adbd-47f899f5866c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.155693 Oct 15 22:22:38.713016 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2259/6761] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:38 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:39.736369 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:39.736981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:39.743922 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:39.744400 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:39.744836 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:39.745563 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:39.746017 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:39.746441 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:39.882828 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:39.890779 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-53342c7b-5e99-4d2c-aa28-cb671fa2cbef tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.158317 Oct 15 22:22:39.891345 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4503/6762] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:39 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:40.886684 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [req-f81ea29f-90b0-4acf-be78-e2157ddf621c req-cf0eb573-70f7-4559-8d96-c42d3301720e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1f05571-1f9e-4ea8-8af1-a53672d0beba", "name": "network-vif-plugged", "status": "completed", "tag": "414e20e5-e456-4dcb-82dd-8db120f98dac"}]} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:40.897545 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f81ea29f-90b0-4acf-be78-e2157ddf621c req-cf0eb573-70f7-4559-8d96-c42d3301720e service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:40.898039 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f81ea29f-90b0-4acf-be78-e2157ddf621c req-cf0eb573-70f7-4559-8d96-c42d3301720e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:40.898482 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [req-f81ea29f-90b0-4acf-be78-e2157ddf621c req-cf0eb573-70f7-4559-8d96-c42d3301720e service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:40.911156 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.compute.server_external_events [req-f81ea29f-90b0-4acf-be78-e2157ddf621c req-cf0eb573-70f7-4559-8d96-c42d3301720e service nova] Creating event network-vif-plugged:414e20e5-e456-4dcb-82dd-8db120f98dac for instance d1f05571-1f9e-4ea8-8af1-a53672d0beba on nch2-master-1 Oct 15 22:22:40.915014 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:40.917096 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:40.918238 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [req-f81ea29f-90b0-4acf-be78-e2157ddf621c req-cf0eb573-70f7-4559-8d96-c42d3301720e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035158 Oct 15 22:22:40.920582 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2260/6763] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:22:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:22:40.925977 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:40.926493 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:40.926926 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:40.927652 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:40.928114 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:40.928720 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:41.132776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:41.141339 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-6d020b61-696f-4b13-9af7-9d55da47cb1e tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.230581 Oct 15 22:22:41.141911 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4504/6764] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:40 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 15 22:22:42.165474 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:42.166192 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:42.174419 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:42.174938 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:42.175387 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:42.176093 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:42.176559 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:42.176997 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:42.314553 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:42.322422 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-8bef40c9-8562-423a-9ace-c81c884b0ba6 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1372 microversion: 2.1 time: 0.161210 Oct 15 22:22:42.324708 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2261/6765] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:42 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1372 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:43.346919 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:43.347538 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:43.354391 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:43.354875 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:43.355310 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:43.356485 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:43.356935 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:43.357385 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:43.496719 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:43.504626 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-fbaefbde-50d8-41ad-a00c-2e3335735003 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1556 microversion: 2.1 time: 0.161620 Oct 15 22:22:43.505243 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4505/6766] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:43 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1556 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:43.528568 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:22:43.529247 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:22:43.536122 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:43.536620 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:43.537088 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:43.537795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:43.538264 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:43.538702 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:43.669011 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:22:43.676992 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-322e1191-ee05-4486-8f99-6acc74e17fc4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1556 microversion: 2.1 time: 0.152888 Oct 15 22:22:43.677589 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2262/6767] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:22:43 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1556 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:22:45.597682 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-030af661-3b52-4478-acaf-aa73bfb48918 req-221a6846-5c53-427e-a85e-c901c18202e0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1f05571-1f9e-4ea8-8af1-a53672d0beba", "tag": "414e20e5-e456-4dcb-82dd-8db120f98dac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:22:45.608872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-030af661-3b52-4478-acaf-aa73bfb48918 req-221a6846-5c53-427e-a85e-c901c18202e0 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:22:45.609390 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-030af661-3b52-4478-acaf-aa73bfb48918 req-221a6846-5c53-427e-a85e-c901c18202e0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:22:45.609823 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-030af661-3b52-4478-acaf-aa73bfb48918 req-221a6846-5c53-427e-a85e-c901c18202e0 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:22:45.621236 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-030af661-3b52-4478-acaf-aa73bfb48918 req-221a6846-5c53-427e-a85e-c901c18202e0 service nova] Creating event network-changed:414e20e5-e456-4dcb-82dd-8db120f98dac for instance d1f05571-1f9e-4ea8-8af1-a53672d0beba on nch2-master-1 Oct 15 22:22:45.626970 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-030af661-3b52-4478-acaf-aa73bfb48918 req-221a6846-5c53-427e-a85e-c901c18202e0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032921 Oct 15 22:22:45.627576 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4506/6768] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:22:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:24:46.128820 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=28 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:24:46.129532 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=56 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:24:46.130715 ncdv-master-1 devstack@n-api.service[138689]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:24:46.130888 ncdv-master-1 devstack@n-api.service[138689]: Sat Oct 15 22:24:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba/action (ip 10.222.0.16) !!! Oct 15 22:24:46.136143 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestSnapshotPatternsnapshot-1597104164"}} {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:24:46.138469 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:24:46.146700 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:24:46.147202 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:24:46.147654 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:24:46.148385 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:24:46.148851 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:24:46.149318 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:24:46.561531 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-77eafa1a-e62b-40c3-95a2-fb2feb329d38 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "POST /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba/action" status: 202 len: 0 microversion: 2.1 time: 0.434659 Oct 15 22:24:46.562187 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2263/6769] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Oct 15 22:24:46 2022] POST /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba/action => generated 0 bytes in 436 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Oct 15 22:24:51.597981 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=3 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:24:51.598866 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=70 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:24:51.600323 ncdv-master-1 devstack@n-api.service[138688]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 15 22:24:51.600573 ncdv-master-1 devstack@n-api.service[138688]: Sat Oct 15 22:24:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 15 22:24:51.605783 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-b99624ef-b1c5-41a5-9576-4a37100ac6e4 req-2e1c3ee2-861f-4b60-8c13-bc46228d328a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1f05571-1f9e-4ea8-8af1-a53672d0beba", "name": "network-vif-plugged", "status": "completed", "tag": "414e20e5-e456-4dcb-82dd-8db120f98dac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:24:51.620156 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b99624ef-b1c5-41a5-9576-4a37100ac6e4 req-2e1c3ee2-861f-4b60-8c13-bc46228d328a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:24:51.620643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b99624ef-b1c5-41a5-9576-4a37100ac6e4 req-2e1c3ee2-861f-4b60-8c13-bc46228d328a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:24:51.621106 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-b99624ef-b1c5-41a5-9576-4a37100ac6e4 req-2e1c3ee2-861f-4b60-8c13-bc46228d328a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:24:51.634836 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-b99624ef-b1c5-41a5-9576-4a37100ac6e4 req-2e1c3ee2-861f-4b60-8c13-bc46228d328a service nova] Creating event network-vif-plugged:414e20e5-e456-4dcb-82dd-8db120f98dac for instance d1f05571-1f9e-4ea8-8af1-a53672d0beba on nch2-master-1 Oct 15 22:24:51.640422 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-b99624ef-b1c5-41a5-9576-4a37100ac6e4 req-2e1c3ee2-861f-4b60-8c13-bc46228d328a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044397 Oct 15 22:24:51.641119 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4507/6770] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:24:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:25:05.402822 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_api:SELECT=2 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:05.403664 ncdv-master-1 devstack@n-api.service[138689]: DEBUG dbcounter [-] [138689] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:05.407155 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_api:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:05.408116 ncdv-master-1 devstack@n-api.service[138688]: DEBUG dbcounter [-] [138688] Writing DB stats nova_cell1:SELECT=2 {{(pid=138688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 22:25:05.409446 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:05.410123 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:05.412807 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-2aed215a-0458-4288-97f4-19a4175c327d req-220f8cd2-ff62-4a5f-97a9-b4f06eef3f8a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1f05571-1f9e-4ea8-8af1-a53672d0beba", "tag": "414e20e5-e456-4dcb-82dd-8db120f98dac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:25:05.418670 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:05.419444 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:05.420165 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:05.420682 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:05.421182 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:05.421878 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:05.424817 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2aed215a-0458-4288-97f4-19a4175c327d req-220f8cd2-ff62-4a5f-97a9-b4f06eef3f8a service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:05.425500 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2aed215a-0458-4288-97f4-19a4175c327d req-220f8cd2-ff62-4a5f-97a9-b4f06eef3f8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:05.425933 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-2aed215a-0458-4288-97f4-19a4175c327d req-220f8cd2-ff62-4a5f-97a9-b4f06eef3f8a service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:05.438101 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-2aed215a-0458-4288-97f4-19a4175c327d req-220f8cd2-ff62-4a5f-97a9-b4f06eef3f8a service nova] Creating event network-changed:414e20e5-e456-4dcb-82dd-8db120f98dac for instance d1f05571-1f9e-4ea8-8af1-a53672d0beba on nch2-master-1 Oct 15 22:25:05.445427 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-2aed215a-0458-4288-97f4-19a4175c327d req-220f8cd2-ff62-4a5f-97a9-b4f06eef3f8a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040784 Oct 15 22:25:05.446221 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4508/6771] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:25:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 15 22:25:05.452306 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:25:05.540276 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-6cf8e7fc-9a33-4582-9b12-d8db6ca3e245 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 204 len: 0 microversion: 2.1 time: 0.139273 Oct 15 22:25:05.541000 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2264/6772] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:25:05 2022] DELETE /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:25:05.550667 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:05.551643 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:05.558776 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:05.559628 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:05.560233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:05.560955 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:05.561480 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:05.561894 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:05.774244 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:25:05.782333 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-c7a18935-49d4-43e8-aa46-7ed8491751d5 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1681 microversion: 2.1 time: 0.236458 Oct 15 22:25:05.782907 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4509/6773] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:05 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1681 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:25:06.807280 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:06.807988 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:06.814747 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:06.815246 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:06.815693 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:06.816403 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:06.816871 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:06.817383 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:07.018917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:25:07.027502 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-d5fe2d9c-34f3-4dea-8f10-9e2200fa1c45 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1681 microversion: 2.1 time: 0.224575 Oct 15 22:25:07.028089 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2265/6774] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:06 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1681 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:25:07.659183 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-d2bba9b2-16a6-4084-8a0e-1e6545195474 req-9968c9fb-9378-45ae-921e-2828793bd19d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1f05571-1f9e-4ea8-8af1-a53672d0beba", "name": "network-vif-deleted", "tag": "414e20e5-e456-4dcb-82dd-8db120f98dac"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:25:07.670559 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2bba9b2-16a6-4084-8a0e-1e6545195474 req-9968c9fb-9378-45ae-921e-2828793bd19d service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:07.671050 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2bba9b2-16a6-4084-8a0e-1e6545195474 req-9968c9fb-9378-45ae-921e-2828793bd19d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:07.671766 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-d2bba9b2-16a6-4084-8a0e-1e6545195474 req-9968c9fb-9378-45ae-921e-2828793bd19d service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:07.684780 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-d2bba9b2-16a6-4084-8a0e-1e6545195474 req-9968c9fb-9378-45ae-921e-2828793bd19d service nova] Creating event network-vif-deleted:414e20e5-e456-4dcb-82dd-8db120f98dac for instance d1f05571-1f9e-4ea8-8af1-a53672d0beba on nch2-master-1 Oct 15 22:25:07.689704 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-d2bba9b2-16a6-4084-8a0e-1e6545195474 req-9968c9fb-9378-45ae-921e-2828793bd19d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034561 Oct 15 22:25:07.690334 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4510/6775] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:25:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:25:08.051879 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:08.052633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:08.060569 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:08.061556 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:08.062009 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:08.062743 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:08.063215 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:08.063655 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:08.100729 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:08.101139 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:08.101781 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:08.102193 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lazy-loading 'fault' on Instance uuid d1f05571-1f9e-4ea8-8af1-a53672d0beba {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:25:08.148752 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:25:08.156514 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-b9c546a9-23a8-4cad-af8b-7db872c368b9 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 200 len: 1336 microversion: 2.1 time: 0.108613 Oct 15 22:25:08.157113 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2266/6776] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:08 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 1336 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:25:09.183458 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:09.184171 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: d1f05571-1f9e-4ea8-8af1-a53672d0beba] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:09.190442 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:09.190934 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:09.191375 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:09.192076 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:09.192545 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:09.192978 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:09.206604 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.wsgi [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] HTTP exception thrown: Instance d1f05571-1f9e-4ea8-8af1-a53672d0beba could not be found. Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Request method failure captured: Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: request: GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba HTTP/1.1 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Accept: application/json Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Accept-Encoding: identity Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Connection: close Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Content-Length: 0 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Content-Type: application/json Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Host: 10.222.0.16 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: Openstack-System-Scope: None Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: User-Agent: python-urllib3/1.26.12 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Auth-Token: gAAAAABjSzIJWQm6stML_eLeB622U5tO9KNWqNWoybuMZt0-Z4kiNPrEeZwcPThTi6xrdx86HtAEOr52N08-AGpM5cH_6iBqxy7T0TAws5KA7Hw9wN4Z0n8rzdxseukS5Qrbd2CNQF_7eVbrVr9lUMhw03i57wZiH0bXrLhEm0GidPo4p_SXKVo Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Id: None Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Domain-Name: None Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Identity-Status: Confirmed Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Is-Admin-Project: True Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Id: default Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Domain-Name: Default Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Id: f02d6a82dfc941c9a0f19bce198d76ab Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Project-Name: tempest-TestSnapshotPattern-1556217073 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Role: reader,member Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Roles: reader,member Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant: tempest-TestSnapshotPattern-1556217073 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Id: f02d6a82dfc941c9a0f19bce198d76ab Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-Tenant-Name: tempest-TestSnapshotPattern-1556217073 Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-User: tempest-TestSnapshotPattern-1556217073-project Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Id: default Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-User-Domain-Name: Default Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-User-Id: 4879d2bce837495baf7411ee27934ece Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: X-User-Name: tempest-TestSnapshotPattern-1556217073-project Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: method: > Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: exception: Instance d1f05571-1f9e-4ea8-8af1-a53672d0beba could not be found. Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: action_args: {'id': 'd1f05571-1f9e-4ea8-8af1-a53672d0beba'} Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1f05571-1f9e-4ea8-8af1-a53672d0beba could not be found. Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1f05571-1f9e-4ea8-8af1-a53672d0beba could not be found. Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:25:09.208248 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1f05571-1f9e-4ea8-8af1-a53672d0beba could not be found. Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: ERROR nova.api.openstack.wsgi Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Returning 404 to user: Instance d1f05571-1f9e-4ea8-8af1-a53672d0beba could not be found. {{(pid=138688) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-741b4c1a-ae5b-4ce7-99a8-1d257722e14f tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba" status: 404 len: 111 microversion: 2.1 time: 0.031383 Oct 15 22:25:09.212801 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4511/6777] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:09 2022] GET /compute/v2.1/servers/d1f05571-1f9e-4ea8-8af1-a53672d0beba => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:25:10.861102 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:10.867195 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-55f63964-921d-4c3d-8d4b-bad9e0e0d2f1 req-d85e63f1-70f4-47da-a4cd-9d9ee206d0bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:25:10.867790 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:10.879449 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:10.879617 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:10.880697 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:10.882061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:10.882061 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:10.882233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-55f63964-921d-4c3d-8d4b-bad9e0e0d2f1 req-d85e63f1-70f4-47da-a4cd-9d9ee206d0bf service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:10.882633 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:10.882753 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-55f63964-921d-4c3d-8d4b-bad9e0e0d2f1 req-d85e63f1-70f4-47da-a4cd-9d9ee206d0bf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:10.883213 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-55f63964-921d-4c3d-8d4b-bad9e0e0d2f1 req-d85e63f1-70f4-47da-a4cd-9d9ee206d0bf service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:10.897354 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-55f63964-921d-4c3d-8d4b-bad9e0e0d2f1 req-d85e63f1-70f4-47da-a4cd-9d9ee206d0bf service nova] Creating event network-changed:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:25:10.903283 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-55f63964-921d-4c3d-8d4b-bad9e0e0d2f1 req-d85e63f1-70f4-47da-a4cd-9d9ee206d0bf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039845 Oct 15 22:25:10.905869 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4512/6778] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:25:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:25:10.911375 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Going to try to terminate instance {{(pid=138689) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 15 22:25:11.005856 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-1831ba4e-25bc-44a6-80c4-a27606f95e9b tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 204 len: 0 microversion: 2.1 time: 0.147940 Oct 15 22:25:11.006462 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2267/6779] 10.222.0.16 () {58 vars in 1239 bytes} [Sat Oct 15 22:25:10 2022] DELETE /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 15 22:25:11.015613 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:11.016745 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.compute.api [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138688) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:11.024175 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:11.024872 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:11.026133 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:11.027732 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:11.028450 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:11.029233 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:11.172062 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.policy [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138688) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:25:11.180192 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-67d2eb2e-aec0-47d1-b2bf-36447330e5da tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1683 microversion: 2.1 time: 0.169151 Oct 15 22:25:11.181001 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4513/6780] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:11 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1683 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:25:12.205789 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:12.206518 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:12.215010 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:12.215572 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:12.216063 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:12.217276 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:12.217763 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:12.218235 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:12.253917 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:12.254424 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:12.255437 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:12.255998 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.objects.instance [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lazy-loading 'fault' on Instance uuid e3605236-a1e4-469a-aee2-39dc799c3574 {{(pid=138689) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 15 22:25:12.300400 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.policy [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4879d2bce837495baf7411ee27934ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f02d6a82dfc941c9a0f19bce198d76ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138689) authorize /opt/stack/nova/nova/policy.py:199}} Oct 15 22:25:12.308959 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-ec6318ea-5598-4d25-9988-febf20ea30c4 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 200 len: 1337 microversion: 2.1 time: 0.108425 Oct 15 22:25:12.309561 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2268/6781] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:12 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 1337 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 15 22:25:12.918392 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [req-516eb49a-ce88-4800-9c2c-f19ee70a3057 req-45bdc5d4-9c07-4c3c-ac45-a728d2c40b22 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e3605236-a1e4-469a-aee2-39dc799c3574", "name": "network-vif-deleted", "tag": "873197d9-1353-45b0-ad75-fe9bc0bff341"}]} {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 15 22:25:12.929833 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-516eb49a-ce88-4800-9c2c-f19ee70a3057 req-45bdc5d4-9c07-4c3c-ac45-a728d2c40b22 service nova] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:12.930317 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-516eb49a-ce88-4800-9c2c-f19ee70a3057 req-45bdc5d4-9c07-4c3c-ac45-a728d2c40b22 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:12.930756 ncdv-master-1 devstack@n-api.service[138688]: DEBUG oslo_concurrency.lockutils [req-516eb49a-ce88-4800-9c2c-f19ee70a3057 req-45bdc5d4-9c07-4c3c-ac45-a728d2c40b22 service nova] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138688) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:12.941619 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.compute.server_external_events [req-516eb49a-ce88-4800-9c2c-f19ee70a3057 req-45bdc5d4-9c07-4c3c-ac45-a728d2c40b22 service nova] Creating event network-vif-deleted:873197d9-1353-45b0-ad75-fe9bc0bff341 for instance e3605236-a1e4-469a-aee2-39dc799c3574 on nch1-master-1 Oct 15 22:25:12.946806 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [req-516eb49a-ce88-4800-9c2c-f19ee70a3057 req-45bdc5d4-9c07-4c3c-ac45-a728d2c40b22 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032137 Oct 15 22:25:12.947376 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4514/6782] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Oct 15 22:25:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 15 22:25:13.333615 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '>' {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:13.334229 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.compute.api [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] [instance: e3605236-a1e4-469a-aee2-39dc799c3574] Fetching instance by UUID {{(pid=138689) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 15 22:25:13.341244 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:13.341739 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:13.342186 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:13.342894 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Acquiring lock "680a3962-acf9-40a3-9cba-9f0a72927170" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 15 22:25:13.343359 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 15 22:25:13.343795 ncdv-master-1 devstack@n-api.service[138689]: DEBUG oslo_concurrency.lockutils [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Lock "680a3962-acf9-40a3-9cba-9f0a72927170" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138689) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 15 22:25:13.356957 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.wsgi [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] HTTP exception thrown: Instance e3605236-a1e4-469a-aee2-39dc799c3574 could not be found. Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Request method failure captured: Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: request: GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 HTTP/1.1 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Accept: application/json Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Accept-Encoding: identity Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Connection: close Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Content-Length: 0 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Content-Type: application/json Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Host: 10.222.0.16 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: Openstack-System-Scope: None Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: User-Agent: python-urllib3/1.26.12 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Auth-Token: gAAAAABjSzIJWQm6stML_eLeB622U5tO9KNWqNWoybuMZt0-Z4kiNPrEeZwcPThTi6xrdx86HtAEOr52N08-AGpM5cH_6iBqxy7T0TAws5KA7Hw9wN4Z0n8rzdxseukS5Qrbd2CNQF_7eVbrVr9lUMhw03i57wZiH0bXrLhEm0GidPo4p_SXKVo Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Id: None Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Domain-Name: None Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Identity-Status: Confirmed Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Is-Admin-Project: True Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Id: default Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Domain-Name: Default Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Id: f02d6a82dfc941c9a0f19bce198d76ab Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Project-Name: tempest-TestSnapshotPattern-1556217073 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Role: reader,member Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Roles: reader,member Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f02d6a82dfc941c9a0f19bce198d76ab"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}] Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant: tempest-TestSnapshotPattern-1556217073 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Id: f02d6a82dfc941c9a0f19bce198d76ab Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-Tenant-Name: tempest-TestSnapshotPattern-1556217073 Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-User: tempest-TestSnapshotPattern-1556217073-project Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Id: default Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-User-Domain-Name: Default Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-User-Id: 4879d2bce837495baf7411ee27934ece Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: X-User-Name: tempest-TestSnapshotPattern-1556217073-project Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: method: > Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: exception: Instance e3605236-a1e4-469a-aee2-39dc799c3574 could not be found. Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: action_args: {'id': 'e3605236-a1e4-469a-aee2-39dc799c3574'} Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: {{(pid=138689) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e3605236-a1e4-469a-aee2-39dc799c3574 could not be found. Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e3605236-a1e4-469a-aee2-39dc799c3574 could not be found. Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 15 22:25:13.358610 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e3605236-a1e4-469a-aee2-39dc799c3574 could not be found. Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: ERROR nova.api.openstack.wsgi Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: DEBUG nova.api.openstack.wsgi [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Returning 404 to user: Instance e3605236-a1e4-469a-aee2-39dc799c3574 could not be found. {{(pid=138689) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: INFO nova.api.openstack.requestlog [None req-14835ff2-ee61-47e6-b2a9-2ce95686475c tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574" status: 404 len: 111 microversion: 2.1 time: 0.030311 Oct 15 22:25:13.362495 ncdv-master-1 devstack@n-api.service[138689]: [pid: 138689|app: 0|req: 2269/6783] 10.222.0.16 () {58 vars in 1236 bytes} [Sat Oct 15 22:25:13 2022] GET /compute/v2.1/servers/e3605236-a1e4-469a-aee2-39dc799c3574 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 15 22:25:13.526975 ncdv-master-1 devstack@n-api.service[138688]: DEBUG nova.api.openstack.wsgi [None req-65aa2cbe-3b31-4b53-b94e-b0f0aaf64078 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] Calling method '.version_select at 0x7fe74e11f310>' {{(pid=138688) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 15 22:25:13.549430 ncdv-master-1 devstack@n-api.service[138688]: INFO nova.api.openstack.requestlog [None req-65aa2cbe-3b31-4b53-b94e-b0f0aaf64078 tempest-TestSnapshotPattern-1556217073 tempest-TestSnapshotPattern-1556217073-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestSnapshotPattern-935573672" status: 202 len: 0 microversion: 2.1 time: 0.026571 Oct 15 22:25:13.550412 ncdv-master-1 devstack@n-api.service[138688]: [pid: 138688|app: 0|req: 4515/6784] 10.222.0.16 () {58 vars in 1254 bytes} [Sat Oct 15 22:25:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestSnapshotPattern-935573672 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)